Azure-docs: default:privileged is not a clusterrolebinding

[tesharp]

Documentation refers to a default:privileged clusterrolebinding that is installed by default. It looks like this is a rolebinding in namespace kube-system, not a clusterrolebinding

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: a7dc4598-18d3-bdcc-e21a-f838fb752268
• Version Independent ID: edf2d399-9bbb-1c1e-e4cc-d69bf2ce0f05
• Content: Use pod security policies in Azure Kubernetes Service (AKS)
• Content Source: articles/aks/use-pod-security-policies.md
• Service: container-service
• GitHub Login: @mlearned
• Microsoft Alias: mlearned

[mimckitt]

Thanks for the feedback! We are currently investigating and will update you shortly.

[mimckitt]

@tesharp can you elaborate on how you came to that conclusion? I don't see kube-system mentioned in this document.

[DreamRivulet]

I can't see the clusterrolebindings “default:priviledged“ either after psp enabled:
~$ kubectl get clusterrolebindings default:priviledged -o yaml
Error from server (NotFound): clusterrolebindings.rbac.authorization.k8s.io "default:priviledged" not found

however there is a "default:privileged" rolebindings created in kube-system:
~$ kubectl get rolebindings -n kube-system
NAME AGE
default:privileged 93s

[DreamRivulet]

Is it expected to bind the clusterrole "psp:privileged" with rolebinding "default:privileged" in kube-system namespace?

[DreamRivulet]

Another issue, when I tried to create nginx-privileged and nginx-unprivileged nginx pods, both failed with the error:
$ kubectl-nonadminuser apply -f nginx-privileged.yaml
Error from server (Forbidden): error when creating "nginx-privileged.yaml": pods "nginx-privileged" is forbidden: unable to validate against any pod security policy: []
$ kubectl-nonadminuser apply -f nginx-unprivileged.yaml
Error from server (Forbidden): error when creating "nginx-unprivileged.yaml": pods "nginx-unprivileged" is forbidden: unable to validate against any pod security policy: []

I think this is as my expected, because this service account doesn't have permission to use podsecuritypolicies.

But in this doc the situation is different, it says that privileged pod creation is denied by default AKS PSP, and unprivileged pod is allowed to be created but not scheduled.
I didn't see this in my cluster, and i think without adding psp to the serviceaccount "psp-aks:nonadmin-user", it will all fail with the same error:
unable to validate against any pod security policy: []

[tesharp]

As DreamRivulet mentioned the documentation refers to a clusterrolebinding that should exist, but doesn’t. Kube-system is not mentioned in the docs but there is a rolebinding in kube-system namespace called same as clusterrolebinding should be called. So I assume the docs are wrong and it should refer to a rolebinding in kube-system instead if clusterrolebinding

[mimckitt]

@mlearned @MicrosoftDocs/aks-pm can anyone confirm if this is a doc issue or a bug? If doc, I can update no problem.

[palma21]

@sauryadas

[DreamRivulet]

any update on this issue? @MicahMcKittrick-MSFT @sauryadas

[sauryadas]

A rolebinding instead of a clusterrolebinding is created for the kube-system namespace and bound it to the cluster role for it to apply only to the kube-system namespace and not the entire cluster. priviledged psp is used for service accounts in kube-system namespace. @MicahMcKittrick-MSFT Can you please update the docs?

[DreamRivulet]

@sauryadas could you also check my comments: https://github.com/MicrosoftDocs/azure-docs/issues/40300#issuecomment-539821598 ？I think this part may also needs to be updated.

[mimckitt]

Thanks @sauryadas I will update the doc today

[sauryadas]

@DreamRivulet you are right on the #40300 (comment) . can you copy/paste both the yaml files just to be double sure?

[DreamRivulet]

nginx-unprivileged.yaml

apiVersion: v1
kind: Pod
metadata:
name: nginx-unprivileged
spec:
containers:
- name: nginx-unprivileged
image: nginx:1.14.2

nginx-privileged.yaml

apiVersion: v1
kind: Pod
metadata:
name: nginx-privileged
spec:
containers:
- name: nginx-privileged
image: nginx:1.14.2
securityContext:
privileged: true

[DreamRivulet]

@MicahMcKittrick-MSFT May I know when this doc will be updated? As this inaccurate part of the doc may cause confusion and frustration when users try psp on AKS.

[mimckitt]

We are working it now. Still getting clarification on if only a single part needs to change or multiple.

[DreamRivulet]

@sauryadas could you please help to confirm whether other parts need to be updated?

[sauryadas]

you are correct @DreamRivulet . Please feel free to make a PR

@MicahMcKittrick-MSFT Please clarify in the doc that the following applies to only to service accounts that have permission to use the PSP applied

privileged pod creation is denied by default AKS PSP, and unprivileged pod is allowed to be created but not scheduled

[DreamRivulet]

@MicahMcKittrick-MSFT May I know when this doc will be updated? Is there an ETA of this task?

[mimckitt]

Myself of @mlearned will look into updating this week. We currently have a hold on documentation updates due to Ignite so nothing can be published for a few days.

[kmoussa]

Hi @MicahMcKittrick-MSFT , I just tried the same tutorial and it's till not updated and I get the same issues when i try to run the privileged and unprivileged pods?
when i run kubectl-nonadminuser apply -f nginx-privileged.yaml it should return to me this error "

Error from server (Forbidden): error when creating "nginx-privileged.yaml": pods "nginx-privileged" is forbidden: unable to validate against any pod security policy: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]

instead this is what i get which indicates that It can't relate to which security policy I'm violating?

Error from server (Forbidden): error when creating "nginx-privileged.yaml": pods "nginx-privileged" is forbidden: unable to validate against any pod security policy: []

[truealex81]

Hi @kmoussa,
Last error tells you that you didn't bind psp clusterrole to your serviceaccount ("default" serviceacoount is created automatically in every namespace).

kubectl create clusterrolebinding psp-deny-privileged-clusterrolebinding --clusterrole=psp-deny-privileged-clusterrole --group=system:serviceaccounts

The command above will bind psp to every serviceaccount (including "default") in every namespace, so the psp defined in clusterrole psp-deny-privileged-clusterrole became default for all cluster.

Of course you could bind it any serviaccount instead of whole cluster, see details here: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#service-account-permissions

[kmoussa]

Hi @truealex81 , thanks for your reply however I actually did create a rolebinding to my serviceaccount as per this screenshot but i still get the above result, what am I missing?

[truealex81]

@kmoussa you need to create 3 things:

1. restricted psp

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
----cut----

1. ClusterRole

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: psp:restricted
rules:
- apiGroups:
  - extensions
  resources:
  - podsecuritypolicies
  resourceNames:
  - restricted
  verbs:
  - use

1. ClusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: psp:restricted-to-serviceacounts
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp:restricted
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts

This is working config if you want to attach restricted psp to default sa's (to all pods except pods in kube-system namespace which has privileged psp attached)
If you want to attach restricted psp to certain sa, then try to run this comand instead p.3:
kubectl create rolebinding psp:restricted-to-sa \ --clusterrole=psp:restricted \ --serviceaccount=my-namespace:my-sa \ --namespace=my-namespace

[DreamRivulet]

@MicahMcKittrick-MSFT , may I know the latest status of this doc issue? It seems that doc is not updated yet.

[AlexanderViken]

Any eta on document update?

[pdev1989]

Doc is still buggy. Can this be fixed soon, please ?

[sheeeng]

Could someone provide the complete workaround steps on this issue while waiting for the documentation to be updated?

[sheeeng]

@truealex81, could you share with us the rest of pod-security-policy.yaml?

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted

kubectl apply --filename pod-security-policy.yaml
The PodSecurityPolicy "restricted" is invalid:
* spec.runAsUser.rule: Unsupported value: "": supported values: "MustRunAs", "MustRunAsNonRoot", "RunAsAny"
* spec.seLinux.rule: Unsupported value: "": supported values: "MustRunAs", "RunAsAny"
* spec.supplementalGroups.rule: Unsupported value: "": supported values: "MayRunAs", "MustRunAs", "RunAsAny"
* spec.fsGroup.rule: Unsupported value: "": supported values: "MayRunAs", "MustRunAs", "RunAsAny"

[truealex81]

@sheeeng, you could find examples in Kubernetes docs

[zr-msft]

thank you for the feedback @tesharp ! I've updated the document and the changes should be live soon.

please-close