Containers-roadmap: [EKS] [request]: EKS managed node group support for ASG target group

Created on 21 Jan 2020 · 12Comments · Source: aws/containers-roadmap

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
The ability to attach a load balancer to the ASG created by the EKS managed node group at cluster creation with cloudformation.

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
We used to create an unmanaged node group with ASG and a classic load balancer in the same cloudformation stack. We used !Ref to attach the load balancer to the ASG using TargetGroupARNs. However, the configuration is not available in eks managed node group at cluster creation today.

Are you currently working around this issue?
We need to separate the creation of cluster and the load balancer into two stacks while they have the same lifecycle. Besides, we are not sure if this modification to ASG is allowed and supported since the ASG is managed by EKS.

EKS EKS Managed Nodes

Source

chingyi-lin

👍48

Most helpful comment

We also need this to support services of type: NodePort.

adamrbennett on 18 Sep 2020

👍3

All 12 comments

@chingyi-lin can you help clarify your use case for this configuration vs. creating a Kubernetes service type=LoadBalancer?

tabern on 2 Mar 2020

@tabern Unless I'm mistaken, one can not use a single NLB for several K8s services of type load balancer. For example, we want to be able to point ports 80 and 443 to our ingress controller service, but we also want port 22 to the SSH service of our GitLab.

Also we want to be able to share our NLB between classic EC2 instances and EKS cluster to be able to do a zero downtime migration from the stateless application running on EC2 instances to the same application running on an EKS cluster.

And the last use case we have is sharing a NLB between two EKS clusters (blue and green) to be able to seamlessly switch from one to the other (in case we have big changes to bring to our cluster, we prefer spawning a new cluster and switching to it after having tested that it works as intended).

yann-soubeyrand on 2 Mar 2020

👍1

I have a workaround in terraform (a bit tricky but it works):

resource "aws_autoscaling_attachment" "autoscaling_attachment" {
  autoscaling_group_name = lookup(lookup(lookup(aws_eks_node_group.node_group, "resources")[0], "autoscaling_groups")[0], "name")
  alb_target_group_arn   = var.TARGET_GROUP_ARN
}

dawidmalina on 3 Apr 2020

@dawidmalina your workaround works for adding the autoscaling instances to the load balancer target group, however, the ALB can't reach the node group.

HTTP/2 504 
server: awselb/2.0
date: Thu, 09 Apr 2020 18:53:46 GMT
content-type: text/html
content-length: 148

<html>
<head><title>504 Gateway Time-out</title></head>
<body bgcolor="white">
<center><h1>504 Gateway Time-out</h1></center>
</body>
</html>

guigo2k on 9 Apr 2020

Another workaround I plan to test is to add postStart and preStop lifecycle event on the pod (https://kubernetes.io/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/) with a little command that register / deregister from the target group using aws cli. You can easily get instanceId from within the container (wget -q -O - http://169.254.169.254/latest/meta-data/instance-id) and use it on aws elbv2 register-targets .

jodem on 27 Apr 2020

Hey all, please take a look at the TargetGroupBinding CRD included in the v2 release candidate of the ALB ingress controller

https://github.com/kubernetes-sigs/aws-alb-ingress-controller/releases/tag/v2.0.0-rc0

We believe this will address the feature request described in this issue, and are looking for feedback.

mikestef9 on 16 Sep 2020

Hi @mikestef9, thanks for the update. Unfortunately, this does not address our use cases outlined in this comment https://github.com/aws/containers-roadmap/issues/709#issuecomment-593322971.

yann-soubeyrand on 17 Sep 2020

We also need this to support services of type: NodePort.

adamrbennett on 18 Sep 2020

👍3

@yann-soubeyrand are you tring to use multiple ASG in a single TargetGroup? otherwise, TargetGroupBinding should solve it.

M00nF1sh on 5 Oct 2020

@M00nF1sh isn't TargetGroupBinding meant for use with ALB ingress controller only? We use NLB with Istio ingress gateway. And yes, we need to put two ASG in a single target group for certain operations requiring zero downtime.

yann-soubeyrand on 5 Oct 2020

@yann-soubeyrand
it supports both ALB/NLB targetGroups. we'll rename ALB ingress controller to AWS LoadBalanacer controller soon.
Currently, when using instanceType TargetGroups, it only supports using all nodes in your cluster as backend, but we'll add some nodeSelector in the future, so if your two ASG will be in same cluster, it will be support. (but we won't support two ASG in different cluster)

M00nF1sh on 5 Oct 2020

👍1

@M00nF1sh sorry for the late reply. We need to be able to put two ASGs from different clusters in a single target group. This is how we do certain migrations requiring rebuilding a whole cluster.

yann-soubeyrand on 15 Nov 2020

Was this page helpful?

0 / 5 - 0 ratings