Containers-roadmap: [EKS] [request]: Enable Pod Preset for EKS

[ajohnstone]

Tell us about your request
Enable Pod Preset for EKS - https://kubernetes.io/docs/concepts/workloads/pods/podpreset/

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Need to transparently inject configuration into deployments and other objects, such as proxy configuration.

Whilst this is also achievable with mutating admission controllers. It would be ideal to use a native implementation.

Are you currently working around this issue?
Patching deployments such as aws-node to inject proxy configuration to reach AWS endpoints.

[christopherhein]

@ajohnstone thanks for adding this request, given the feature is still v1alpha1 we'll keep an eye on this until it moves to a beta version before adding support for this into EKS.

[atheiman]

@christopherhein - Is it possible to configure the apiserver in EKS to change the list of enabled admission controller plugins? I havent found a way to do it with eksctl or the eks Terraform resource. I dont see anything like that in the EKS createcluster api method, so Im guessing no?

https://docs.aws.amazon.com/eks/latest/APIReference/API_CreateCluster.html

[christopherhein]

@atheiman there is not as of today, are there specific (besides Pod Preset) admission controllers that you want requested?

[atheiman]

Is there a list of which plugins are enabled on eks clusters?

[christopherhein]

Is there a list of which plugins are enabled on eks clusters?

Yes, check out https://docs.aws.amazon.com/eks/latest/userguide/platform-versions.html it maps k8s version (and our internal eks version) to the list of enabled Admission Controllers.

[atheiman]

Thank you for the responses!

From that doc I think the only one I am currently concerned with that is missing is PodPreset. I do understand that it is in alpha state, but I think consumers should be able to turn on alpha admission controller plugins via create-cluster (or a new modify-cluster) api call.

[ksexton]

Could the same features of PodPreset be accomplished with the MutatingAdmissionWebhook that was added to support dynamic admission controllers mentioned here: https://aws.amazon.com/about-aws/whats-new/2018/10/amazon-eks-enables-support-for-kubernetes-dynamic-admission-cont/ ?

[christopherhein]

@atheiman and @ksexton in the interim you could use Open Policy Agent to do this via a MutatingAdmissionWebhook although this example specifically uses ValidatingAdmissionWebhook's conceptually it's the same with a different manifest and policy. Check out the blog I just wrote on using it w/ EKS https://aws.amazon.com/blogs/opensource/using-open-policy-agent-on-amazon-eks/

[o6uoq]

+1 for AWS EKS being able to enable k8s alpha settings.

[Jeffwan]

Instead of tracking individual feature gate, I would suggest someone create a issue to talk about create EKS cluster with alpha features.

[szelenka]

Found this CRD which works okay in AWS .. the selector portion doesn't seem to be fully compatible with the native PodPreset selector (yet), but it can inject env into Pods pretty nicely.
https://github.com/redhat-cop/podpreset-webhook

Would be nice to have this natively in AWS though.