Containers-roadmap: [EKS] [request]: Managed Node Groups Custom Userdata support

[shanmugakarna]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
EKS Managed Node Groups should support adding additional custom user-data and additional bootstrap/kubelet args

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

1. We can't enable docker0 bridge easily in Managed Node Groups, when we want to deploy CI/CD tools
2. Any other customs tasks we need to do on worker nodes

Are you currently working around this issue?
Old way of launching worker nodes

[atheiman]

We have to install certain agents onto EC2 instances for security compliance. This would give us an option to do that on startup without going down the path and headache of managing our own ami.

[bhops]

This would also allow us to set taints on various nodegroups by passing kubelet arguments, which does not appear to be possible in the current state of nodegroups.

[dschaaff]

I’m specifically looking to add taints to nodes, whether via specifying user data or direct api support.

[girikuncoro]

we have exact same usecase with @atheiman where we need to install agent and configure system level to integrate with our central auth service through PAM for SSH access

[drewhemm]

Userdata is the standard, AWS-native way to modify an instance post-launch.

We perform a considerable amount of dynamic instance customisation in userdata for things like networking. Baking a custom AMI is the only alternative.

[3oris]

Also, Userdata is the only current recommended way of "Restricting Access to Amazon EC2 Instance Profile Credentials" (https://docs.aws.amazon.com/eks/latest/userguide/restrict-ec2-credential-access.html ).

If you don't do this it leaves your cluster vulnerable by default, allowing any pod using an un-annotated service account (e.g. default) to interfere with ENIs and cluster IPs on the host.

[splieth]

@tabern any change to get this on the roadmap?

[funkswing]

Also looking for the ability to taint nodes in the managed worker node autoscaling group.

Explicitly stated here under the Feature parity with unmanaged nodegroups section is:

_"Control over the node bootstrapping process and customization of the kubelet are not supported. This includes the following fields: maxPodsPerNode, taints, targetGroupARNs, preBootstrapCommands, overrideBootstrapCommand, clusterDNS and kubeletExtraConfig."_

Exposing the kubelet-extra-args flag during the worker node ami bootstrapping could be a path forward...https://aws.amazon.com/blogs/opensource/improvements-eks-worker-node-provisioning/

[johncblandii]

This is a big blocker for our usage of node groups. We need to set enable-docker-bridge on one of the groups in our cluster and can't so we have to go back to managing autoscaling groups, etc.

[vadapalliravikumar]

we need the ability to use custom user data to configure http proxy for the docker daemon and kubelet.

[bneelima84]

Any update on this feature request ? This is required for our internal security compliance

[MatteoMori]

Hey guys,
Is there any updates on this one?

[mikestef9]

Hi @MatteoMori, this feature will come as part of #585. We are working on it, but no timeline that I can share right now.

[jgoeres]

This would also allow us to set taints on various nodegroups by passing kubelet arguments, which does not appear to be possible in the current state of nodegroups.

I also came here looking for a way to add taints to nodegroups, any news?

[0verc1ocker]

System.aws.out.println("¡Ándale, Ándale AWS!")

GKE already has the support for this for years...

I'm not here to troll. I'm just really pissed off. This is basic security we are talking about...

[im-erikjohansson]

We'd need this to be able to configure additional services for logrotation, for example ipamd.

[AmarkFox]

need add args for bootstrap.sh --kubelet-extra-args , make the node support --allowed-unsafe-sysctls=net.ipv4.*

[odg0318]

@mikestef9 Can you tell when this feature will be released in production?

[mikestef9]

Can't give specific dates in this forum, but with that said, I'd highly recommend checking out the AWS container day presentations at Kubecon next week :)

https://awscontainerday.splashthat.com/

[dgarbus]

Can't give specific dates in this forum, but with that said, I'd highly recommend checking out the AWS container day presentations at Kubecon next week :)

https://awscontainerday.splashthat.com/

I also noticed in the CloudFormation release history that AWS::EKS::Nodegroup now has a LaunchTemplate property.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/ReleaseHistory.html
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-eks-nodegroup-launchtemplatespecification.html

[bgsilvait]

Is HERE!

https://aws.amazon.com/blogs/containers/introducing-launch-template-and-custom-ami-support-in-amazon-eks-managed-node-groups/

[mikestef9]

Closing as this feature request is addressed by launch template support. See #585 for details!

An important note is that user data must in MIME multi-part archive format, as by default, EKS will merge the bootstrapping command required for nodes to join the cluster with your user data. If you use a custom AMI in your launch template, this merging will not happen and you are responsible for nodes joining the cluster. See docs for more details.