Admission Controllers for kubernetes clusters with Kyverno

Kyverno runs as a dynamic admission controller in a Kubernetes cluster. Kyverno receives validating and mutating admission webhook HTTP callbacks from the kube-apiserver and applies matching policies to return results that enforce admission policies or reject requests.

Install Kyverno using Helm:

Add Kyverno repository.

helm3 repo add kyverno https://kyverno.github.io/kyverno/

Update kyverno repo

helm3 repo update

Use helm 3 to create a namespace and install kyverno

helm3 install kyverno kyverno/kyverno --namespace kyverno --create-namespace

Installation is completed now in kubernetes cluster.

kubectl get pods -n kyverno
NAME                       READY   STATUS    RESTARTS   AGE
kyverno-554ccb78c8-9mjr5   1/1     Running   0          6h1m
kyverno-554ccb78c8-w79ks   1/1     Running   5          5h34m

Creation of policies:

Now create policies for validating the admission controller. You can get so many sample policies in kyverno documentation.

Here is one sample policy for adding annotation to pods except kube-system namespace file name as safe-to-evict.yaml. This policy is used for autoscaling the GKE.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: add-safe-to-evict
  annotations:
    pod-policies.kyverno.io/autogen-controllers: none
spec:
  rules:
  - name: "add-safe-to-evict-to-pods"
    match:
      resources:
        kinds:
        - Pod
    exclude:
      resources:
        namespaces:
        - kube-system
    mutate:
      patchStrategicMerge:
        metadata:
          annotations:
            cluster-autoscaler.kubernetes.io/safe-to-evict: "true"

Now create policy in kubernetes cluster.

kubectl create -f safe-to-evict.yaml

Policy has been created in cluster

kubectl get clusterpolicies.kyverno.io
NAME                BACKGROUND   ACTION   READY
add-safe-to-evict   true         audit    true

For verification purpose lets delete one of the pod in cluster and see it is annotating the pods or not.

kubectl get pod busybox-7bf4f97ff-7dzpv -o yaml
apiVersion: v1
kind: Pod
metadata:
  annotations:
    cluster-autoscaler.kubernetes.io/safe-to-evict: "true"
    policies.kyverno.io/last-applied-patches: |
      add-safe-to-evict-to-pods.add-safe-to-evict.kyverno.io: added /metadata/annotations/cluster-autoscaler.kubernetes.io~1safe-to-evict

Here it is adding the annotations to the pod.