Enhancing Kubernetes Security with Kyverno, RuntimeClass, and Kata Containers

5 min readJul 21, 2023

Flexibility without Compromise

As more organizations adopt Kubernetes, security has become a major concern. Kubernetes has many built-in security features, but ensuring security policies are properly enforced across a large and complex cluster can be challenging. At the same time, Kubernetes also needs to provide the flexibility required for developers to work efficiently. This article will explore three technologies to help you achieve this balance: Kyverno Kubernetes Policy Engine, Kubernetes RuntimeClass, and Kata Containers.

Kyverno Policy Engine

Policy engines are a powerful tool that Kubernetes uses to manage and enforce policies across the entire cluster. Policy Engines enable cluster administrators to configure and enforce policies for a wide range of actions, including network access, resource usage, and pod placement. Examples of policy engines are Open Policy Agent (OPA) and Kyverno.

Kyverno is a Kubernetes policy engine that allows you to enforce policies across your Kubernetes clusters. With Kyverno, you can create policies restricting resource access, enforcing naming conventions, limiting network access, and more. You can also define policies that trigger on specific events, such as when a new deployment is created. When a policy violation occurs, Kyverno can take automatic action to remediate the issue, such as blocking a deployment or removing a pod.

One of the benefits of Kyverno is that it integrates seamlessly with Kubernetes, so you can define policies using familiar Kubernetes objects like ConfigMaps and CRDs. Kyverno also supports policy validation and generation, so you can easily ensure that your policies work as expected and generate policies based on your existing Kubernetes resources.

RuntimeClasses in Kubernetes

RuntimeClass is a Kubernetes feature that allows you to specify the container runtime for your workloads. With RuntimeClass, you can choose between different container runtimes or different configurations of the same runtime. This is useful for security purposes because different runtimes have different security properties. For example, you might use a lightweight runtime for some workloads that require fast startup times while using a more secure runtime (but with additional overheads) for workloads requiring higher isolation levels.

Another advantage of RuntimeClass is that it allows you to switch runtimes without modifying your workloads. For example, if you need to upgrade to a more secure runtime, you can simply update the RuntimeClass in the workload deployment manifest, and Kubernetes will automatically use the new runtime associated with that RuntimeClass.

Overall, RuntimeClass provides a powerful tool for managing the runtime environment of your Kubernetes workloads, giving you greater control over security and flexibility.

Kata Containers

Kata Containers is an open-source project that provides a lightweight virtualization layer to isolate Kubernetes pods from the host system. Standard container runtimes like “runc” provides cgroup isolation and share the same kernel with all the pods. On the other hand, Kata runtime (depicted as containerd-shim-kata-v2 in the above diagram) provides an additional isolation layer by creating a dedicated virtual machine for every Kubernetes pod. This is depicted as Pod VM in the above diagram. Every pod uses a separate kernel.

The Benefits of Using Policy Engines, RuntimeClasses and Kata Together

While Policy engines, RuntimeClasses, and Kata Containers are powerful on their own, they become even more potent when used together. Here are some benefits of using Policy Engines, RuntimeClasses, and Kata Containers together:

Enhanced Security
Administrators can use Policy Engines, RuntimeClasses, and Kata Containers to automatically ensure that workloads from specific users or workloads requesting privileged capabilities are run using Kata containers’ runtime. This is made possible by the Kyverno policy engine automatically adding the correct RuntimeClass in the pod definition before it gets deployed.
Greater Flexibility
RuntimeClasses and Kata Containers enable administrators to choose from different container runtimes, providing greater flexibility in managing pods. This flexibility allows administrators to choose the runtime and container technology that best meets their specific requirements, including security requirements, without compromising developer productivity.
Developer and User Productivity
The developer or the user need not worry about RuntimeClasses or adding specific entries in the pod definition. The user continues to deploy the pods as usual. The magic happens behind the scenes through the policy engine.

The following section will guide you on setting up Kyverno and looking at a few example policies using RuntimeClass and Kata.

Setup and examples

Assuming you have an existing Kubernetes cluster with helm setup. Install the Kyverno policy engine using the instructions given below.

helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
helm install kyverno kyverno/kyverno -n kyverno - create-namespace - set replicaCount=1

Note that you can use any policy engine providing a mutation admission controller

Following is an example policy to allow running pods in a specific namespace using Kata containers runtime. The RuntimeClass configured for using Kata containers runtime is named “kata” (surprise ;) ) :

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: run-pod-using-kata
spec:
  background: false
  rules:
  - name: "Run pods in specific namespace using Kata containers runtime"
    match:
      any:
      - resources:
          kinds:
          - Pod
          namespaces:
          - test
     mutate:
      patchStrategicMerge:
        spec:
          +(runtimeClassName): kata

Deploy the policy:

kubectl apply -f pod-policy.yaml

Following is another policy example that ensures privilege pods or pods requiring a “root” user in all namespaces except “kube-system” are run using Kata containers runtime:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: run-priv-pod-using-kata
spec:
  background: false
  rules:
  - name: "Run privilege pods using Kata containers runtime"
    match:
      all:
      - resources:
          kinds:
          - Pod    
    exclude:
        any:
        - resources:
            namespaces:
            - kube-system
    preconditions:
     any:
      - key: "{{request.object.spec.containers[?securityContext.privileged] | length(@)}}"
        operator: Equals
        value: 1
      - key: "{{request.object.spec.containers[?securityContext.runAsRoot] | length(@)}}"
        operator: Equals
        value: 1
    mutate:
      patchStrategicMerge:
        spec:
          +(runtimeClassName): kata

For more examples, refer to the official Kyverno documentation.

Conclusion

Kubernetes has revolutionized the way software applications are deployed and managed. However, securing Kubernetes clusters can be challenging. Policy Engines and RuntimeClasses are two powerful features that Kubernetes provides to enhance cluster security.

By combining these with additional container runtimes like Kata containers, you can create a highly secure and flexible Kubernetes environment. Kyverno enforces security policies, RuntimeClass specifies the runtime environment for your workloads, and Kata Containers provides an extra layer of security by isolating pods from each other and the host system. This approach allows you to maintain a high level of security without compromising on the flexibility developers need to work efficiently.