Amazon-eks-ami: SSM agent

[davidkarlsen]

Environment:

• AWS Region: eu-north-1
• Instance Type(s): t3.micro / t3.small
• EKS Platform version (use aws eks describe-cluster --name <name> --query cluster.platformVersion): eks.7
• Kubernetes version (use aws eks describe-cluster --name <name> --query cluster.version): 1.14
• AMI Version: amazon-eks-node-1.14-v20190927 (ami-0c1dc1b58d2c9db01)
• Kernel (e.g. uname -a): Linux ip-10-20-102-207.eu-north-1.compute.internal 4.14.146-119.123.amzn2.x86_64 #1 SMP Mon Sep 23 16:58:43 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
• Release information (run cat /etc/eks/release on a node):

cat: /etc/eks/release: No such file or directory

Would it make sence to include SSM agent by default? While these type of instances are more like cattle, it could be handy to be able to connect from time to time through Systems Manager Session. I want to avoid running ssh at the nodes

[davidkarlsen]

Also found this: https://github.com/aws/containers-roadmap/issues/593

[max-rocket-internet]

Would it make sence to include SSM agent by default?

No please don't 😃

Many people don't want it and also want a lean and clean AMI.

https://github.com/awslabs/amazon-eks-ami/issues/127

[davidkarlsen]

OK, I can understand that - I guess focus should be on https://github.com/aws/containers-roadmap/issues/596 then - as today you are only left with the option to build custom AMI - which kind of defeats the purpose of managed node-groups in the first place.

[dlaidlaw]

With this one you are going to disappoint people no matter which way you go. A compromise would be putting the SSM Agent code in the AMI, then only starting the agent based on a flag. As long as your IAM instance role does not allow the SSM agent, even starting it will cause no harm, since it will fail to start.

[max-rocket-internet]

Comrades,
There's no need to disappoint anyone, there is already ways to do this. Just create a daemonset, this is mentioned in that issue I linked to. This method is already used for many other management tools like SSM Agent. For example Datadog, many logging tools, kube-proxy, spot termination handler, the CNI etc. There's even repo with the config from a very reputable developer: https://github.com/mumoshu/kube-ssm-agent

Otherwise just use user-data, that's exactly what it's for. It's easy in the CFN templates. Easy in Terraform 😅

[dlaidlaw]

That daemonset uses some unsafe practices. Host networking and privileged container. It also allows access to the docker socket. These things will not be allowed by many.
Yes, using UserData is exactly what I do. I would prefer if it was already installed so that my instance startup time is faster though.

[max-rocket-internet]

Does it not run as root in the AMI though?

EDIT: yes it does:

root@xxxxxx:~# ps aux| grep ssm
root     25861  0.0  0.1  13136  1040 pts/0    S+   15:37   0:00 grep --color=auto ssm
root     28961  0.0  1.6 553804 16764 ?        Ssl   2019   0:56 /snap/amazon-ssm-agent/1480/amazon-ssm-agent

[slimm609]

user data is not accessible when using node groups, you can not specify AMI or anything when using managed node groups.

[shenrie]

Seems like there should be two versions of the EKS AMI, one with the SSM agent enabled and one without. Seems like one additional jenkins job could make everyone happy.

[juniorz]

@slimm609 Amazon Elastic Kubernetes Service (EKS) now supports EC2 Launch Templates and custom AMIs for managed node groups, which allows specifying user data.

See: https://aws.amazon.com/blogs/containers/introducing-launch-template-and-custom-ami-support-in-amazon-eks-managed-node-groups/

[MyannaHarris]

Hi all,

We've released new EKS Optimized Amazon Linux AMIs with the SSM agent pre-installed and enabled by default.

https://github.com/awslabs/amazon-eks-ami/releases/tag/v20210621