Tanzu Pod Security Admission (PSA)

Starting with vSphere with Tanzu Kubernetes releases v1.25 and onwards. Pod Security Admission (PSA) replaces the original method for pod security using Pod Security Policy (PSP). Tanzu Kubernetes v1.25 had the default security mode set to warn. From Tanzu Kubernetes release v1.26 and onwards, the default security mode changed to enforce with the pod standard set to restricted.

Both Kubernetes.io and VMware by Broadcom have some good documentation that details PSA. The process is very easy to use and configured by defining lables on namespaces. This new method to define pod security feels much more simple and logical than the older PSP method to use, which I really like.

Without going too deep into PSA on Tanzu, and quite honestly I’m still learning, there are just a few things to note. First there are PSA Modes. We have three modes to choose from, enforce, audit, and warn. This will define what action the control plane will take when a policy violation is detected.

Next we have Pod Security Standards (PSS). We have three profiles or levels, priviledged, baseline, and restricted. These will define what the pods are allowed to do. Kubernetes.io has a good detailed list of what each of these profiles encompasses.

Now why do we care about all of this on Tanzu Kubernetes. Well, basically, nothing will want to run out of the box when you deploy a new pod on a TKr v1.26+ cluster. Even the most simplest pods will be forbidden from running due to violating PodSecurity.

kubectl run test --image=nginxdemos/hello -n test

Error from server (Forbidden): pods “test” is forbidden: violates PodSecurity “restricted:latest”: allowPrivilegeEscalation != false (container “test” must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container “test” must set securityContext.capabilities.drop=[“ALL”]), runAsNonRoot != true (pod or container “test” must set securityContext.runAsNonRoot=true), seccompProfile (pod or container “test” must set securityContext.seccompProfile.type to “RuntimeDefault” or “Localhost”)

This security stuff is all quite frustrating isn’t it. 🙂

We have a few options to getting our pods running though. We can configure some security context on our pod or we can change the security level on a namespace. By far the namespace is the quickest and easiest.

It’s just a matter of providing a label on our namespace that defines our enforcement level from the default restricted to either baseline or priviledged. You can initially start with baseline and then move to priviledged if that doesn’t work.

For me, straight to priviledged all the way. 🤠

kubectl label --overwrite ns NAMESPACE pod-security.kubernetes.io/enforce=privileged

When you next go to run a pod in your namespace with this label it should start up fine. Horay

kubectl run test --image=nginxdemos/hello -n test

pod/test created

There’s one frustrating things that I’ve run into time and time again with this new Pod Security Admission method. And that is you obviously need a namespace first to define a label on it, right. Now that might not sound that frustrating until you try to deploy something like an external HELM chart or third party app and realise that it doesn’t want to run because of the default PSA level. Unfortunately I don’t know of an easily (and supported) way to make this change at the cluster level. Please let me know if you know of a way! I’ve seen references to do it against TKGm but not vSphere with Tanzu. And we don’t talk about TKGm right!

Anyway in conclusion, you’re going to want to most likely modify Pod Security Admission on your namespaces after you deploy a TKG cluster on vSphere with Tanzu. So don’t forget.