Containers-roadmap: [EKS] [request]: EKS works with Istio with auto sidecar-injection.

[hustshawn]

Tell us about your request
What do you want us to build?
Currently, EKS cannot work with Istio with auto sidecar-injection. We need EKS is able to work well with Istio without limitation. Since a cluster with kops does not have such issue. This issue could be a serious concern for us to decide to build k8s with kops or EKS.

Which service(s) is this request for?
This could be Fargate, ECS, EKS, ECR
EKS

*Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
*
What outcome are you trying to achieve, ultimately, and why is it hard/impossible to do right now? What is the impact of not having this problem solved? The more details you can provide, the better we'll be able to understand and solve the problem.
There should be some limitation on EKS about Dynamic Admission Webhook. eg. label a namespace for istio sidecar auto-injection, and all pods in the namespace will be automatically injected with a proxy sidecar when creating pods. This feature is quite important for us, especially using helm to perform creation/updates.

Are you currently working around this issue?
How are you currently solving this problem?
Currently, I can only disable the auto-injection when installing Istio, and manual inject the sidecar every time. As what you demo here

Additional context
Anything else we should know?
Please refer to this issue in Istio.

Attachments
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

[hustshawn]

Link related story here:

• https://github.com/istio/istio/issues/16434
• https://github.com/istio/istio/issues/13840#issuecomment-532954308

[nandhyala]

@hustshawn ,

Even i am facing same issue for auto sidecar-injection if istio-injection=enabled for namespace on (EKS + ISTIO + Weave).

Is there any solution is there to fix webhook issue ?

[hustshawn]

@nandhyala
Unfortunately no. That's why I linked the related stories here, and hopefully to get noticed by the EKS team.

[phlukman]

thanks @hustshawn for this update!

[phlukman]

to whoever google this issue, Istio 1.2.2 with eks running k8s v 1.13 fails in the following points:

• istio policy pod keeps crashing
• Istio installation times out after 15 mins even, due to problem no. 1
• Automatic injection proxy does not work as now there is a 503 error in the readiness probe for the Envoy proxy.

[hustshawn]

@phlukman Good summary above.

B.T.W. I have also tried on the latest EKS 1.14, the issue still there. As I mentioned https://github.com/istio/istio/issues/13840#issuecomment-532954308, it seems a compatibility issue from EKS with Istio instead of the K8S itself.

This issue becomes the only one reason that blocks me to use EKS.

[phlukman]

@hustshawn have you tried using GKE? I'd be curious to find out about it...

[hustshawn]

@phlukman Nope. All my infrastructure leverage on AWS. I only use kops and EKS as the K8S setup solution.

I suppose GKE is Kubernetes native platform. It should be working fine, and I seldom see such problems from the community.

[somaliz]

@hustshawn I'm facing the same issue EKS1.14 + istio 1.2.2 + Weave and i think the issue is weave's "npc" network policy + istio . How did you configure weave with EKS ? do you have it with genie-cni installed ? im trying to hack weave-cni + eks 1.14 without vpc-cni & genie-cni any tips or suggestions on how to achieve that ! Thank you ..

[hustshawn]

@somaliz Yes. I installed weave on top of genie-cni. You can refer to this post https://medium.com/codeops/installing-weave-cni-on-aws-eks-51c2e6b7abc8

[nandhyala]

If we go with EKS1.14 + istio 1.2.3 + Weave + genie-cni components Metric server is not working with genie-cni on EKS cluster,AWS support team is confirmed it is there out of scope for genie-cni.

[chadlwilson]

I don't think this issue accurately describes the current state of the world in a way that will help problems get resolved.

EKS generally works fine with Istio and auto sidecar injection (we've been using it since January 2019 throughout Istio 1.1 -> 1.3 and EKS versions 1.11 -> 1.14) with the regular AWS CNI plugin.

https://github.com/istio/istio/issues/13840 describes a specific issue where replicationsets can get stuck with timeouts when the sidecar injector is not available. Workaround is keeping multiple replicas of the sidecar injector distributed across nodes to support rolling node bounces. Perhaps the symptoms are the same when the admission controller is not able to play nicely with the sidecar injector webhook, but the root issue is probably different.

If there is a problem specific to Weave, I think we should try and note that specifically in the relevant issues, as that poses quite a different technical context to a default EKS + AWS CNI + Istio installation.