Kubeadm: Enable ValidatingAdmissionWebhook and MutatingAdmissionWebhook

[tamalsaha]

Since 1.9, ValidatingAdmissionWebhook and MutatingAdmissionWebhook are beta. Kubeadm should enable them by default. There are two parts to this:

• Setting the --admission-control flag to api server.
• Providing / Managing kubeconfigFile for webhook. My understanding is that
  --admission-control-config-file flag is for kube api server and then this flag points to a file like
  { "webhookAdmission": { "kubeConfigFile": "actual-kubeconfig-path" } }
  In the admission control config file, we have to point to a kubeConfigFile path. That kubeconfig will have the client certs for api server to use. This kubeconfig should use clients certs issued from --requestheader-client-ca-file.

So, the question is should Kubeadm do these by default ?

[tamalsaha]

cc: @sttts @deads2k @luxas @caesarxuchao

[deads2k]

In the admission control config file, we have to point to a kubeConfigFile path. That kubeconfig will have the client certs for api server to use. This kubeconfig should use clients certs issued from --requestheader-client-ca-file.

@kubernetes/sig-auth-api-reviews

That is the wrong certificate to use. It is confusing different trust domains under one cert and doesn't resolve problems around certificate distribution. The kubeconfig should not use that client certificate.

The client certificates signed by that CA are the most trusted client certificates in the entire cluster. That CA is used to trust front proxies which are allowed to assert identity as any user in the cluster.

Instead, you can create an admission server which requires zero config at the kube-apiserver and zero config from every extension apiserver. Such a server is fully portable across every kubernetes deployment and secure by default. See how we do this inside of https://github.com/openshift/generic-admission-server and how we make a fully working admission webhook server in less than 200 lines protected by cluster authentication and RBAC here: https://github.com/openshift/kubernetes-namespace-reservation/blob/master/cmd/namespacereservationserver/main.go.

The overall picture looks like this:

You can see that we actually control access using RBAC and avoid exposure of SA tokens.

[deads2k]

In the admission control config file, we have to point to a kubeConfigFile path.

You don't have to reference a kubeconfig file. You only have to do that if you need to manually distribute client keys or tokens. Some topologies (see above) don't require that and avoid having to establish a second, uniform authentication and authorization domain.

[luxas]

Setting the --admission-control flag to api server.

Yeah, we should definitely enable those by default as they are beta, looks like we missed the train on https://github.com/kubernetes/kubernetes/pull/54892 :(

Do folks feel like enabling these two admission controllers would be acceptable to do in a patch release as they ought to be enabled in v1.9.0 as-is? I totally missed that PR so hadn't a chance to comment on it.

[sttts]

Do folks feel like enabling these two admission controllers would be acceptable to do in a patch release as they ought to be enabled in v1.9.0 as-is?

They won't change anything if there are no webhooks defined. So at least it is safe to do so.

[luxas]

Ok, then I think we should backport such a change. @dixudx do you want to send that PR against the master branch and then backport to v1.9 :smile:?

[dixudx]

/assign

@luxas Yeah, why not.