Terraform-aws-eks: Terraform Module forcing EKS Cluster recreation in case of instance_type change or ami change

[jmukhtar]

I have issues

I'm submitting a...

• [x] bug report
• [ ] feature request
• [x] support request
• [ ] kudos, thank you, warm fuzzy

What is the current behavior?

We are running an EKS Cluster that was created using the terraform-aws-eks module. We need to change the instance type from t3.medium to r4.large. We updated the instance_type field and run terraform apply, terraform is trying to recreate the cluster as well.

If this is a bug, how to reproduce? Please include a code sample if relevant.

Not sure if it's a bug or our understanding. We have a very simple configuration

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "5.1.0"

  cluster_name                    = "eks-devtest"
  subnets                         = [aws_subnet.eks-privateAzA.id, aws_subnet.eks-privateAzB.id]
  vpc_id                          = aws_vpc.eks-vpc.id
  config_output_path              = "/tmp/"
  cluster_endpoint_private_access = "true"
  write_aws_auth_config           = "false"
  write_kubeconfig                = "false"

  worker_groups = [
    {
      instance_type         = "t3.medium"
      asg_max_size          = 5
      autoscaling_enabled   = true
      protect_from_scale_in = true
    },
  ]

  workers_group_defaults = {
    key_name = "devKey"
  }

  tags = {
    environment = "DevEnv"
    technology  = "EKS"
  }
}

Once created, we changed the instance_type to r4.large and run terraform plan followed by terraform apply and got following error

  # module.eks.aws_eks_cluster.this must be replaced
+/- resource "aws_eks_cluster" "this" {
.
.
.
.
Plan: 3 to add, 2 to change, 2 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.eks.aws_eks_cluster.this: Creating...

Error: error creating EKS Cluster (eks-devtest): ResourceInUseException: Cluster already exists with name: eks-devtest
    status code: 409, request id: bb131dff-0156-456f-a47e-46bd45ce7506

What's the expected behavior?

Expected behaviour should be to update the worker nodes instance type

Are you able to fix this problem and submit a PR? Link here if you have already.

No

Environment details

Dev

• Affected module version:
• OS: Amazon Linux
• Terraform version: v0.12.6

Any other relevant info

[barryib]

Can you provide the terraform plan output for module.eks.aws_eks_cluster ? From the provider code, it only force replacement when one of these argument change : name, role_arn, security_group_ids or subnet_ids.

[jmukhtar]

module.eks.aws_eks_cluster.this must be replaced

+/- resource "aws_eks_cluster" "this" {
~ arn =
"arn:aws:eks:eu-west-1:AWS_ACCOUNT_ID:cluster/eks-devtest" -> (known after
apply)
~ certificate_authority = [
- {
- data =
"DATA"
},
] -> (known after apply)
~ created_at = "2019-05-21 09:11:26 +0000 UTC" ->
(known after apply)
- enabled_cluster_log_types = [] -> null
~ endpoint = "
https://LONG_BORING_ID.sk1.eu-west-1.eks.amazonaws.com" -> (known after
apply)
~ id = "eks-devtest" -> (known after apply)
~ identity = [
- {
- oidc = [
- {
- issuer = "
https://oidc.eks.eu-west-1.amazonaws.com/id/ANOTHER_LONG_ID"
},
]
},
] -> (known after apply)
name = "eks-devtest"
~ platform_version = "eks.4" -> (known after apply)
role_arn =
"arn:aws:iam::AWS_ACCOUNT:role/eks-devtest20190521091115568800000001"
~ status = "ACTIVE" -> (known after apply)
version = "1.13"

    timeouts {
        create = "15m"
        delete = "15m"
    }

It also looks like we are being forced to use a new AMI

image_id = "ami-0199284372364b02a" ->
"ami-00ea6211202297fe8" # forces replacement

Any help as to how to get around this would be really helpful

[jmukhtar]

Thanks for your tip @barryib it was the subnet that was causing the cluster recreation.

On a side note, it might be a good idea to know how to remove a subnet from a running EKS cluster via terraform

[dpiddockcmp]

You cannot currently change the security groups or subnets associated with a cluster. This is an AWS limitation.

From the API docs:

Important
At this time, you can not update the subnets or security group IDs for an existing cluster.

[bry5an]

It also looks like we are being forced to use a new AMI

image_id = "ami-0199284372364b02a" ->
"ami-00ea6211202297fe8" # forces replacement

I'm having the same issue with the force replacement. I just manually set the created image_id to avoid the change. Ideally some sort of lifecycle ignore_changes setup would be cool

```
worker_groups = [
{
name = "${var.node_group_1}"
instance_type = "${var.node_group_1_instance_type}"
additional_userdata = "echo foo bar"
asg_desired_capacity = 2
additional_security_group_ids = [aws_security_group.worker_group_mgmt_one.id]
ami_id = "ami-07be7092831897fd6"
},
]

[dpiddockcmp]

Are you able to provide your full module definition and the diff output?

The AMI will get updated occasionally. This should only cause the launch configuration or template to be recreated and the ASG to be updated. It shouldn't be triggering an eks_cluster replacement.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had recent activity since being marked as stale.