Terraform-aws-eks: Not able to attach security group in Managed node group

[dhineshbabuelango]

When we try to use eks managed worker nodes, we dont find any option to attach security group to the ec2 instances created as part of the managed worker nodes.
The reason being, we normally don't attach any NAT gateway to the worker nodes subnets, instead we will create VPC endpoint and give https access to the eks worker node security group ids.
now since the SG is random and can be known only after we create we are not able to restrict access only to that SG.

I tried mentioning the SG in the resources section like below

resources = [
{
"autoscaling_groups" = [
{
"name" = "eksmanaged"
},
]
"remote_access_security_group_id" = "var.worker_sg"
},
]

but terraform throws Error: "resources": this field cannot be set

[max-rocket-internet]

MNGs is not even in a release yet.

[dhineshbabuelango]

Thanks @max-rocket-internet.
Is there any future request page where I can track this, currently i am able to use terraform to launch it, have some issues like attaching the security group to MNG, how to update the MNG through terraform.

[max-rocket-internet]

Sure, here: https://github.com/terraform-aws-modules/terraform-aws-eks/pull/662

[philipbankier]

@dhineshbabuelango Have you figured out a work around for attaching a security group to an MNG?

[pandres95]

@dhineshbabuelango @max-rocket-internet

I also need to attach a SG to an MNG, but there just isn't a way to do so.

[dhineshbabuelango]

@pandres95 this is the default behaviour, even with aws-cli or aws console you cannot attach the security group, I asked the AWS support person to raise a future request for this

[gpkc]

I don't know exactly how they do that, but eksctl apparently allows to attach security groups to a node group. From eksctl create nodegroup --help:

--node-security-groups strings   Attach additional security groups to nodes, so that it can be used to allow extra ingress/egress access from/to pods

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[evankicks]

Hi, i'm currently facing the same issues when attaching a sg to MNG, Is there any update on this?

[achisolomon]

having the same issue..

[dpiddockcmp]

This is still impossible with the MNG system released by AWS. Use classic worker groups if you need to assign custom security groups at creation.

There is a request on the container team's roadmap for this feature: https://github.com/aws/containers-roadmap/issues/609

I don't know exactly how they do that, but eksctl apparently allows to attach security groups to a node group. From eksctl create nodegroup --help:

Careful of naming conventions here. A "node group" in eksctl is a classic manually created autoscaling group. This flag is not for managed node groups.

[nikhils5501]

@max-rocket-internet @dpiddockcmp Now that AWS released support for custom security groups for Managed NG using AWS Launch Templates, any update on this?
I see managed_node_groups module on this repo. But that does not support additional_security_group_ids yet.

[galindro]

Well, for this to being able to work, the node_groups module would need to be extended to accept custom launch templates. There is already and issue open for that on https://github.com/terraform-aws-modules/terraform-aws-eks/issues/979.

@max-rocket-internet are you or someone else working on it? Would this custom launch template something that the user would input as a variable to the main module which in turn would pass it back to the node_groups module? If yes, I can make this change and open an PR for that. Shouldn't be that hard to do it....

[amazingandyyy]

attaching extra sg to worker under node_group is mostly needed.

[amazingandyyy]

meantime, another request is that, due to the fact the node goups doesn't support extra sg, but worker_create_security_group is true and it always create a sg named eks_worker_sg, but it never gets used by node group, that resource is redundant for who only use node groups.

[barryib]

There is already the primary SG and it's attached to the managed node groups. Why can we just create a security group rule in that SG ?

Plus, we now support launch template for MNG.

[lenny]

meantime, another request is that, due to the fact the node goups doesn't support extra sg, but worker_create_security_group is true and it always create a sg named eks_worker_sg, but it never gets used by node group, that resource is redundant for who only use node groups.

To work around this, I wound up with the following:

data "aws_instances" "workers" {
  instance_tags = {
    "eks:nodegroup-name" = module.eks_cluster.node_groups.default.node_group_name
  }
}

data "aws_instance" "workers" {
  for_each = toset(data.aws_instances.workers.ids)
  instance_id = each.value
}

resource "aws_network_interface_sg_attachment" "sg_attachment" {
  for_each = { for instance in data.aws_instance.workers : instance.id => instance.network_interface_id }
  security_group_id    = module.eks_cluster.worker_security_group_id
  network_interface_id = each.value
}

Also added this to name the instances as an alternative to using a launch template.

resource "aws_ec2_tag" "worker" {
  count = length(data.aws_instances.workers.ids)
  resource_id = data.aws_instances.workers.ids[count.index]
  key         = "Name"
  value       = "${local.name_prefix}-worker${count.index}"
}