Amazon-eks-ami: Support for t3a, m5ad and r5ad instance types is missing

Created on 8 May 2019 · 10Comments · Source: awslabs/amazon-eks-ami

What would you like to be added:

Support for t3a, m5ad and r5ad instance types.

Why is this needed:

AWS had added new instance types, and the AMI does not currently support them.

Source

drewhemm

👍3

Most helpful comment

According the chart @VojtechVitek references, a t3a.small only supports 2 x ENIs, for a total of 8 IP addresses. At that ratio, --max-pods needs to be 7 or less, as far as I can discern.

dayglojesus on 24 Jul 2019

👍3

All 10 comments

drewhemm on 8 May 2019

What happens currently if the CNI is used on one of these instances?

max-rocket-internet on 13 May 2019

It is not able to allocate IP addresses to all the pods on the instance (I'll have to get the exact error message).

I thought it was because it does not know what the pod limit should be for these instance types, but curiously, I get the same problem with this PR.

drewhemm on 13 May 2019

It is not able to allocate IP addresses to all the pods

But some will run? Or none?

max-rocket-internet on 14 May 2019

It is not able to allocate IP addresses to all the pods

But some will run? Or none?

i ran into this problem yesterday; i was able to start three pods on a t3a.small, and then no additional pods could be scheduled b/c CNI was unable to give them network interfaces. switching back to t3.small enabled me to work around the issue.

hakamadare on 15 May 2019

It is not able to allocate IP addresses to all the pods

But some will run? Or none?

Sorry for the late response. Same as reported by @hakamadare: It runs a few pods, but then no more, without actually hitting the ENI limit.

drewhemm on 15 May 2019

I think the CNI version just needs to be updated to v1.5. See my comment here: https://github.com/aws/amazon-vpc-cni-k8s/issues/59#issuecomment-504972238

sc250024 on 24 Jun 2019

We're hitting the CNI errors when trying to schedule more pods on t3a.small instance too.

I think the number of maxPods t3a.small 11 (see eni-max-pods.txt) doesn't correspond to this table:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI

I think the number of --max-pods on t3a.small instance should be less then 11.

Can anyone confirm?

VojtechVitek on 24 Jul 2019

According the chart @VojtechVitek references, a t3a.small only supports 2 x ENIs, for a total of 8 IP addresses. At that ratio, --max-pods needs to be 7 or less, as far as I can discern.

dayglojesus on 24 Jul 2019

👍3

Thanks for reporting, I've created a PR for the CNI, will fix for the AMI build as well.