Containers-roadmap: [EKS] [Fargate] [request]: fargate daemon set support

[runningman84]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
Allow us to run daemon sets like kube2iam, fluentbit and others in a fargate env.

What do you want us to build?
Improve the scheduler to allow running whitelisted daemon sets on fargate nodes.

Which service(s) is this request for?
Fargate, EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Right it is very difficult to run fargate as a replacement for ec2 nodes because you do not have exporter logs, iam integration and so on...

Are you currently working around this issue?
Use fargate only for non ciritical tasks which do not need iam, logs or anything else...

[mikestef9]

Hey @runningman84,

You can track #701 for a logging solution for EKS/Fargate.

Can you explain more on the IAM integration point and why you are using kube2iam? IAM roles for service accounts works today with EKS/Fargate.

[runningman84]

Yes IAM stuff can be solved using the native integration. But there is still software which runs an old sdk.

I think there should be a general solution for daemon sets in fargate.

[dalgibbard]

FWIW; we would like daemon-sets in EKS Fargate for two main reasons:

• Parity between mixed workloads on EC2 + Fargate (daemonsets that can perform the same actions irrelevant of underlying node type, for consistancy/compliance)
• To offer k8s as a 'platform as a service' to our internal users: specifically, allowing them to deploy whatever they need/want, without needing to worry about things like datadog sidecars (just for example) in their configurations.

This has the potential added advantage that the Datadog Daemonset in particular supports log exporting too (as well as container metrics and events etc), so removes our current blocker of log centralisation for compliance as an extra benefit.

We looked at https://github.com/tumblr/k8s-sidecar-injector for example to auto-inject sidecars, but really, this is exactly the purpose of a daemonset for our use case, and sidecars have limited scope compared to daemonsets.