Containers-roadmap: [EKS] [CNI]: Optional Default CNI Plugin Installation

[TigerC10]

Tell us about your request
I would like an option to disable default installation of the AWS VPC CNI Plugin during create cluster.

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
When an EKS cluster is created, the AWS VPC CNI Plugin is automatically installed to the cluster as the default CNI plugin. In order to change the plugin, the AWS VPC CNI plugin must be uninstalled _before_ worker nodes join the cluster, or else all existing worker nodes must be destroyed after removing the default and installing a different CNI plugin.

Alternatively, a custom AMI must be supplied to the worker nodes which use a lower CNI plugin prefix than 10 so that the worker node loads it before loading the AWS VPC CNI Plugin.

In either of these cases, clusters become less "deployable" with automated tooling. In one case there are manual steps involved in the deployment of a cluster (and possibly a race condition), and in the other case a custom AMI is required which could become out-dated over time as Amazon releases new AMI base images.

Are you currently working around this issue?
Currently building a custom AMI with an alternate CNI plugin installed with 00- prefix.

Additional context
https://github.com/awslabs/amazon-eks-ami/issues/117
https://github.com/aws/amazon-vpc-cni-k8s/issues/214
https://github.com/aws/amazon-vpc-cni-k8s/issues/176

[gaurav-dalvi]

looks like this might be working for some people : https://github.com/awslabs/amazon-eks-ami/issues/117

[TigerC10]

@gaurav-dalvi did the workaround work for you? It didn’t work for me.

[gaurav-dalvi]

yeah same here. did not work for me either.

[lgg42]

Any updates on this? Just wondering... it really sucks having to hack some stuff just to opt out...

[lgg42]

Damn, I really think this should be top priority. In forums, chats, slacks, IRCs, telegram groups, whatever, you name it... there are so many people/companies not choosing EKS because of the pod LIMIT. This simply erases the whole point of kubernetes.

Also... have any of you tried CNI-Genie?? : https://github.com/Huawei-PaaS/CNI-Genie

[TigerC10]

Yeah, I'm using cni-genie at the moment. if you install it to the master node and configure the default to a different CNI before you create the ec2 worker nodes, it works like a charm. However, automating this is kinda difficult.

[lgg42]

W-O-W, even if automating it is a PITA, knowing that it works with EKS is something great to hear! so many thanks!

[TigerC10]

I mean.... yeah... it _works_. I'd still rather not have to stop my cloudformation scripts to run a few kubectl commands before continuing them to create the worker nodes.

[jwenz723]

Would love to see some movement on this issue so that I can use a CNI other than AWS VPC CNI.

[agung-kargo]

Yeah, I'm using cni-genie at the moment. if you install it to the master node and configure the default to a different CNI before you create the ec2 worker nodes, it works like a charm. However, automating this is kinda difficult.

@TigerC10 do you have any article that describes the process? I would love to try this out.