Eksctl: support for containerd (or CRIO) CRI-runtime

[egernst]

Why do you want this feature?
In order to use additional runtimeClasses in Kubernetes, it is necessary to use either containerd or CRIO instead of Dockershim. Example runtimes include Kata Containers and gVisor.

Today, there isn't a way to start a cluster using eksctl which will utilize a non-dockershim based CRI implementation.

What feature/behavior/change do you want?
Ideally, eksctl will support being able to create a cluster with a specified CRI implementation. The default could be dockershim still, but a user should be able to specify something like:
"containerRuntime": "containerd",

[egernst]

This would be similar to what can be done with aks-engine via https://github.com/Azure/aks-engine/blob/master/examples/kubernetes-containerd.json

[whereisaaron]

I think the Ubuntu AMI's you can use with EKS and eksctl maybe using containerd?

Ref: https://medium.com/errnothxbye/centos-7-with-cri-o-on-eks-ae9684aff764

[egernst]

Trying to take a look now....

One thing I noticed is that I cannot start a cluster utilizing the --node-ami-family Ubuntu1804:

eksctl create cluster -n kata-eksctl-cluster -r us-west-2  --version 1.12  -N 1 --ssh-access --node-ami-family Ubuntu1804
[ℹ]  using region us-west-2
[ℹ]  setting availability zones to [us-west-2b us-west-2a us-west-2c]
[ℹ]  subnets for us-west-2b - public:192.168.0.0/19 private:192.168.96.0/19
[ℹ]  subnets for us-west-2a - public:192.168.32.0/19 private:192.168.128.0/19
[ℹ]  subnets for us-west-2c - public:192.168.64.0/19 private:192.168.160.0/19
[✖]  Unable to determine AMI to use: Unable to determine AMI for region us-west-2, version 1.12, instance type m5.large and image family Ubuntu1804

I'm happy to open a separate issue for this; I expect this to work after reading the eksctl README.md .

[whereisaaron]

@egernst I don't think are any 1.12 AMIs for ubuntu yet - at least I can't see any. Try deploying the nodegroup with 1.11.

[egernst]

Same story:

$ eksctl create cluster -n kata-eksctl-ubuntu  --node-ami-family Ubuntu1804
[ℹ]  using region us-west-2
[ℹ]  setting availability zones to [us-west-2a us-west-2b us-west-2d]
[ℹ]  subnets for us-west-2a - public:192.168.0.0/19 private:192.168.96.0/19
[ℹ]  subnets for us-west-2b - public:192.168.32.0/19 private:192.168.128.0/19
[ℹ]  subnets for us-west-2d - public:192.168.64.0/19 private:192.168.160.0/19
[✖]  Unable to determine AMI to use: Unable to determine AMI for region us-west-2, version 1.11, instance type m5.large and image family Ubuntu1804

[egernst]

Cluster control plane starts without error when just explicitly setting an Ubuntu AMI, a`la:

$ eksctl create cluster -n kata-eksctl-ubuntu  --node-ami ami-00a7ce72a2fe4fcf8 

But eventually times out wating for a node from the nodegroup to join.

 [✖]  timed out (after 25m0s) waiting for at least 2 nodes to join the cluster and become ready in "ng-ba5f5368"

[pawelprazak]

Interesting use case: one nodepool with kata-containers (hard isolation) and one with Docker/containerd (soft isolation)

[errordeveloper]

Indeed this will need to be nodegroup-level option. Supposedly one could make it work with a custom AMI to start with, which may require tweaks to nodebootstrap package. I don't think I'll find time for this anytime soon, but happy to guide someone who wants to have a go at this (either as a proof-of-concept with custom AMI, or as a proper feature).

[whereisaaron]

@errordeveloper @egernst was trying to create a eksctl cluster with Ubuntu nodes but --node-ami-family Ubuntu1804 isn’t working for him?

[errordeveloper]

As I mentioned, the official Ubuntu AMI hasn't seen updates for a while. If
you are keen to help with a community AMI of some sort, I would be happy to
help!

On Wed, 3 Apr 2019, 7:03 pm Aaron Roydhouse, [e-mail]
wrote:

@errordeveloper https://github.com/errordeveloper @egernst
https://github.com/egernst was trying to create a eksctl cluster with
Ubuntu nodes but --node-ami-family Ubuntu1804 isn’t working for him?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/weaveworks/eksctl/issues/696#issuecomment-479597315,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPWSzB34V_tFGo-VoVaq337juM32c26ks5vdO0HgaJpZM4cYyOx
.

[ebcarty]

Was there an update to adding support to select additional runtimes (e.g. containerd)?

[whereisaaron]

@ebcarty there is a related issue in the AWS containers road map that confirms AWS intends to move to containerd at some point. Possibly with the option to choose either docker or containerd runtime.

Someone also suggested that Fargate (for EKS?) is already using containerd.

Ref: https://github.com/aws/containers-roadmap/issues/313

[inductor]

Hi,
I wrote that comment and yes it is confirmed that on Fargate worker nodes containerd runs as the container runtime.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.