Containers-roadmap: [EKS] [request]: Enable TTLAfterFinished in alpha on control plane

[geerlingguy]

Tell us about your request

Kubernetes 1.12 added an awesome new feature that makes working with Jobs so much simpler: TTL Controller for Finished Resources.

It is currently in alpha, so anyone who wants to be able to run Jobs but then have the Jobs go away after some period of time (or right after they complete) has to build their own Job garbage collection system.

Can EKS allow this feature (feature gate TTLAfterFinished) on the EKS Control Plane so we can use it on EKS 1.12 or later?

Which service(s) is this request for?

EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

I want to not have to write and maintain my own Job garbage collection system since a simple, easy-to-use one is already built into Kubernetes 1.12 and later.

Are you currently working around this issue?

I have to maintain my own Job garbage collection system instead of using the alpha one that's built-in.

Additional context

Related issue: https://github.com/awslabs/amazon-eks-ami/issues/238

Attachments

N/A

[acesir]

Any update on this with Kubernetes 1.13 version?

[bart613]

Any update on this with Kubernetes 1.13 version?

It's not working on EKS 1.13.

[whereisaaron]

I’d like this feature too. It entered alpha in 1.12 but it has not graduated to beta, it is still alpha in 1.15. That means there is almost no chance we’ll see it enabled on managed k8s before next year at the earliest.
https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/

The way to promote it is to work with the k8s project to get the feature over the line to a beta release. Then it will be available on managed k8s 3-6 months later.

The feature is an active controller with a watch, queue, and workers. So not much different than an operator. You could try https://github.com/lwolf/kube-cleanup-operator

[doprdele]

I would like this feature as well.

[whereisaaron]

Also, even simpler than an operator, CronJobs with something like:

kubectl delete job $(kubectl get job -o=jsonpath='{.items[?(@.status.succeeded==1)].metadata.name}')

While we wait for native support.

[imdhruva]

Also, even simpler than an operator, CronJobs with something like:

kubectl delete job $(kubectl get job -o=jsonpath='{.items[?(@.status.succeeded==1)].metadata.name}')

While we wait for native support.

that is very simply put and is not a solution; it will almost break a lot of things than solving a problem. I would like something asynchronous for this as a work-around. so after job deployment we start the watch and then after say n seconds we run trigger kubectl delete job $(kubectl get job -o=jsonpath='{.items[?(@.status.succeeded==1)].metadata.name}')

Currently though I am using helmfile for performing all the deployments so I am leveraging postSync hooks to watch for a job completion (my job only takes <40 secs to run) and then perform

echo Logs:  $(kubectl logs job/abc) && kubectl del job/abc

Leveraging helm-hooks would be ideal. In that we run a post-install job to perform job-deletion. In my case though that's a deployment overhead because the post-installjob would need a deployment container which is heavy and will create a needless resource overload on eks when I can very well run the helmfile hook locally in the deployment pipeline

[TonyBarganski]

I would like this feature too which doesn't seem to be working on EKS version v1.14.6 either.

[seb-steuer]

Yep I would like to see that added too. Very useful for anyone that launch jobs frequently and then need to do cleanups. TTL controller would solve that

[whereisaaron]

Unfortunately this is still an alpha feature in 1.16. So it will be at least 9 months or more before it is enabled in EKS, that is assuming it even graduates to beta in 1.17. So could a year or more in the future. So for now you should consider an alternative such as a CronJob or an operator.

https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/

[antoniotamer]

Would be really nice if EKS allowed customers to enable K8S alpha features, at our own risk. I hope the number of votes on this issue is an indication that we are willing to live with the risk.

[eightlimbed]

+1

[jaygorrell]

Put +1s on the top post rather than replies

[mageyuki]

👍

[bryanisgreat]

has this been updated? it appears to work on EKS v1.14.8

[stanislav-zaprudskiy]

has this been updated? it appears to work on EKS v1.14.8

Doesn't appear to be working on EKS 1.14.9

[bryanisgreat]

Yeah, i was mistaken. the spec validated fine when applied, but the setting had no effect on ttl

[aramse]

Here's a quick solution that provides the same functionality via a ttl annotation. Feedback/PRs are welcome!

https://github.com/aramse/k8s-job-reaper

[casidiablo]

[Schnitzel]

waiting on this as well, in the meantime found https://github.com/lwolf/kube-cleanup-operator and happy with it.

[whereisaaron]

TTLAfterFinished is still alpha in 1.18, and that doesn't appear to be changing in 1.19, so I think it will be at least 2021 before we could hope to see this feature in EKS. So for now you should consider an alternative like https://github.com/aramse/k8s-job-reaper

https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates