Definitive Guide to Kubernetes Admission Controller

This blog was originally published by ARMO here.

Written by Leonid Sandler, CTO & Co-founder, ARMO.

What is Kubernetes Admission Controller?

Kubernetes Admission Controller is an advanced plugin for gating and governing the configuration changes and workload deployment in a cluster.

Admission Controller enables DevOps and Security personnel to enforce deployment requirements and restrictions in the cluster upon every workload start and any configuration change. Think of an Admission Controller as an Advanced Resource manager with a shield. It provides a very efficient way to govern and control what can be deployed in order to limit risk and stop many security problems from occurring in the future.

An example of admission control can be a policy that prevents privileged pods from execution in a cluster. Running a pod in privileged mode means that the pod has access to the host's resources and kernel capabilities. A hacker will be extra motivated to exploit this pod and access the host's resources, which causes a significant amount of damage to the cluster.

Benefits of Kubernetes Admission Controller

Extended Security Controls

Kubernetes Admission Controllers offer an additional line of defense on top of other mechanisms. RBAC offers security, however, it is not sufficiently granular and does not have control over, for example, image vulnerability status, and cannot enforce network policy definition, privilege level, and other important characteristics.

Company-Wide Guardrails

Admission controllers may enforce company-wide rules on multiple clusters without changing existing RBAC and workload deployment configuration.

Preventive Capabilities

As opposed to audit log analysis tools and configuration scanning, admission controllers not only detect problems but also prevent them from entering into a cluster.

Highly customizable

Unlike built-in Kubernetes security mechanisms, dynamic admission controllers (described below) are adjustable to many different user-specific scenarios and environments. There are many open-source and commercial admission controller implementations that customers can choose from and enforce their special constraints.

Static Admission Controllers

Static admission controllers are admission controllers that are built-in and provided by Kubernetes itself. Not all of them are enabled by default. Some of them are also blocked or taken by the cloud vendors for their own use. If you own your Kubernetes deployment, you can enable and use them.

Let's look at a few examples of static controllers and see what they do:

• LimitRanger: ensures that incoming requests don’t violate any of the constraints enumerated in the LimitRange object in a namespace. If you are using LimitRange objects in your Kubernetes deployment, you MUST use this admission controller to enforce those constraints. LimitRanger can also be used to apply default resource requests to pods that don't specify any.
• AlwaysPullImages: changes the image pull policy for every new Pod. This is useful, for example, in multitenant clusters to ensure that only those with the credentials to fetch private images can access them. Without this admission controller, after an image has been pulled to a node, any pod from any user can use it just by knowing the image's name without any authorization checks. This feature must be enabled in the cluster.
• PodSecurityPolicy: sometimes referred to as PSP, this is an essential controller that is responsible for deciding whether or not pods should be admitted to the cluster depending on the security policies, pod configuration and container posture. However, this feature was deprecated in Kubernetes v1.21 and will be removed in v1.25.

Dynamic Admission Controllers

There will be scenarios when the native built-in options and functionality just don’t suffice. The requirements may demand the development of additional functionalities that will help us meet our business objectives. Dynamic admission control is a mechanism in which we inject our custom-built business logic into the admission control pipeline.

The purpose of the admission controller is to approve, reject or even modify the API requests before executing them or persisting them in the etcd. There are slight differences between self-managed and cloud-hosted Kubernetes.

In cloud-hosted Kubernetes, the control plane and components such as etcd and Kubernetes API server are completely handled by the provider, such as AWS. While the complete management infrastructure operates in the background across several availability zones, AWS will automatically cure any unhealthy nodes and keep the service available.

When using self-hosted Kubernetes, however, users will have full control over the management components of the cluster. This means that you are fully responsible to keep Kubernetes management nodes and services up-to-date and properly configured.

Dynamic admission controllers can be of two types - validating and mutating. They can be used separately and together.

To implement dynamic admission controllers, you must configure the API server first with a change to the --enable-admission-plugins.

--enable-admission-plugins=...,MutatingAdmissionWebhook,\
 ValidatingAdmissionWebhook

Validating Admission Controller

Validating admission controllers are responsible for gating the API requests and enforcing custom policies prior to request execution or persistence of the new object in the etcd.

Based upon predefined policies expressing customers' business logic, the validating admission controller will accept or reject users’ requests. In the event of a rejection, an appropriate HTTP status code is returned to the API server and stored in the Kubernetes events so that users can read the detailed reason for the failure.

The API server uses an HTTP REST interface to communicate with dynamic admission controllers. It sends a JSON-encoded object that is about to be persisted in the etcd which allows the admission controller to validate the minimal requirements compliance and policy violations and prevent this object from entering the cluster in case of a failure. For example, block all pods that request to run as root. Obviously, such a policy requires exceptions. One object can be validated against multiple admission policy controls.

More sophisticated admission controllers can assess a risk score based on many different criteria rather than a single validation failure. For example, block privileged pods from starting if they point to a container image in the repository that is not explicitly permitted.

Admission controllers usually offer a basic set of policies that users may activate and allow users to define their own custom policies.

One of the very important qualities of admission controllers is their diagnostics capabilities. Admission controller preventing pods from starting may be considered as frustrating, even if it is done for a greater good. Therefore diagnostics with a precise description of the denial reasoning and maybe even suggested remediation is the most important thing to look for in the admission controllers.

Mutating Admission Controller

Mutating admission control is an additional step offered by Kubernetes API servers to help automate policy compliance processes. It is performed by a mutating webhook interface which is logically similar to the validating admission that we have described above, but it also allows changing the object that is being admitted instead of rejecting it.

The mutating step is performed prior to the validating call. Admission controllers that support mutation functionality may perform the mutation and validation in one step or still separate them and act as two independent capabilities. It is important to note that Kubernetes cluster supports multiple mutating and validating webhooks. Therefore, it is more reliable to separate the mutation from the validation because the validation could cover mutations that are performed by other admission controllers.

While mutating capability is very powerful and has a great potential for auto-remediation, it also has a downside. The actual objects that are going to be persisted in the cluster are not the same that the user has designed. It may cause complexities in the debugging and troubleshooting and may even break the desired application logic as the automation is not always right. Therefore, the actual usage of the mutating admission is pretty limited today.

In the contemporary DevOps environments where deployments are constructed from HELM charts and parameters and the pods are created from deployments, mutating admission may cause a conflict with HELM if it will modify the deployment object. Therefore, the most common approach is to modify the pods, but this only further increases the distance between what the user has designed and what is deployed.

Having said that, I believe that mutating admission will grow and help DevOps and security teams to improve their cluster posture.

Conclusion

Admission controllers are a very important mechanism for maintaining an organization's policies and compliance with security standards and regulations. Moreover, webhooks enhance the functionality of the Kubernetes API server, making it simpler to monitor, not just enforce the desired rules.

Working together with other security and deployment best practices, admission controllers can be applied not only in the production environments but also in the development stage to provide early indications of potential violations. In turn, this can help fix issues earlier in the pipeline, speeding up the deployment process and helping developers understand the rules and avoid repetitive mistakes in the future.