Containers-roadmap: [EKS/Fargate] request: Improve Fargate Node Startup Time

[lilley2412]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
Improve the startup time of Fargate nodes on EKS

Which service(s) is this request for?
EKS Fargate

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
I haven't done extensive benchmarks yet, but anecdotal "kicking the tires" of Fargate for EKS shows 30-45 seconds for a node to be created and registered; because it's node-per-pod, I then have to wait for image pull and container start time, so in total it's taking over a minute to start a new pod.

This is problematic for obvious reasons; for some use-cases it's not a show-stopper, like HPA-based scaled deployments I'm OK with the startup time. For others, like a CI cluster for gitlab, the startup time is painful; each CI job spawns a new pod which takes "forever".

Are you currently working around this issue?
Currently just eating the startup time.

[unthought]

Same usecase. Would love to migrate some Github Action workloads to EKS/Fargate for ease of integration, but current pod boot time is a showstopper. In our case it's even more exacerbated, because we'd use Argo Workflows, which launches a pod per workflow step. And Github Action VMs launch incredibly fast, we even had to abandon AWS CodeBuild because of it's VM launch time delay, especially on medium and large instances.

[jgoeres]

Same here - we are trying to put spark jobs on EKS/Fargate. For long-running spark jobs, this is not a big deal, but to streamline our stack, we also have many shorter spark jobs, for which this is effectively unacceptable.

[booleanbetrayal]

Is this an issue with AWS not having the appropriate EC2 capacity ready and warmed ahead of time for node spooling? Certainly feels like it could be an EC2 initialization happening behind the scenes during each and every pod deploy.

[mreferre]

@booleanbetrayal this is not about having a EC2 instances up and running beforehand. There is a lot of mechanics happening behind the scenes that adds up (e.g. connecting the ENI to the customer VPC, etc). Also, Kubernetes doesn't really have a "serverless" mode of operation so the instance we use needs to be initialized with the Kubernetes client code, it needs to virtually connect to the cluster and show up as a fully initialized worker node and only then the pod can be scheduled on it. So while we try to make the user experience as "serverless-ly" as possible, the mechanic behind the scene is more complex than taking a running EC2 to deploy the container. We do appreciate that for a long running task this initialization isn't a big deal but for short running tasks it adds up. We are working hard to reduce the time it takes for these steps to execute and/or removing some of these steps (where possible).

[booleanbetrayal]

Thanks for the clarification @mreferre !

[spicysomtam]

It seems very inefficient. 1 pod = 1 fargate node? I was just testing this and it takes 60s in eu-west-1 to find a fargate node to run a pod on and then it starts running. If you are used to the normal pod spin up on k8s its a second or less depending on whether the node has the image cached. Would it not be better to implement a virtual kubelet for a node which can then spin up lots of pods and thus run much quicker? Or am I missing something and doing something terribly wrong? I did get caught out by the fargate private subnets needing a NAT route to get out to the internet to pull images. I'll continue my testing to see if I can get fargate running faster!

[mreferre]

@spicysomtam I think 50/60 seconds is just about right and you won't be able to reduce it substantially (also, it depends on the image size but even tiny images won't take less than 45 seconds because of everything above). We are working to reduce this timing over time. If all you need is fast single pod startup time than EKS managed node groups are a great solution. Fargate's value isn't in single pod start time but rather in the additional security that running pods in dedicated os kernels bring and the fact you no longer have to manage/scale/life-cycle nodes (you can read more here). A couple of years ago we did look into the virtual Kubelet project to run Fargate pods but 1) this won't have a different effect on pod startup time given all the virtual Kubelet does is proxy requests to a backend (Fargate in this case) and the timing experience would have been similar and 2) virtual Kubelet is a (heavy) fork of the Kubelet and we did not want to go down that path.

[spicysomtam]

I did testing with node groups and Cluster Autoscaling (CA) and the pod spin up times are what I am used with k8s (most of my experience is with Openshift and Rancher). The only slight issue, which isn't AWS related, is spin up time for nodes via the CA when pods are stuck in pending, but you could work around that via placement pods with a lower pod priority; k8s would kill these and replace them with your real pods pending and in the background a new node would be spun up.

[mreferre]

Sure. If you want to over-index on optimizing pods startup times and you are not constrained by the additional costs of idle resources that is the right thing to do. With Fargate we aim at removing a lot of undifferentiated heavy lifting associated to managing the underlying infrastructure but, depending on your use case and objectives, it may not (yet) be the right service for you. We are working on this.

[nyouna]

we are experiencing the same issue with Fargate in ECS... It doesn't seems to be exclusively related to EKS.

[nyouna]

@mikestef9 can you please add ECS label?