Terraform-provider-aws: aws_eks_cluster and create_before_destroy

[tristanpemble]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.7
+ provider.aws v1.22.0

Affected Resource(s)

• aws_eks_cluster

Description

Amazon requires EKS clusters to have a unique name. This means create_before_destroy cannot be used on an EKS cluster, because you will get a ResourceInUseException: Cluster already exists with name whenever there are changes to the cluster.

This is unfortunate, since in order to configure an ASG for worker nodes, you need to reference aws_eks_cluster.endpoint in the user data of the launch configuration. In order to use launch configurations effectively, you have to use create_before_destroy = true on them.

And since the launch configuration depends on the EKS cluster, that forces the EKS cluster to also create_before_destroy. If you have changes to both launch configuration and cluster, then terraform apply will fail without a clear fix.

I'm not sure what solution there is or if there is any, but I wanted to open this to have a discussion about the issue.

[SpamapS]

Yep, I've run into this too. The only way I can find to change an EKS that uses ASG's with terraform is to rip the whole thing down with destroy.

[nhuray]

I've run into this too...I finally remove manually the cluster then refresh and apply.

[jamesdobson]

Now that the userdata no longer needs a reference to the cluster endpoint URL, do you still get this problem?

See the launch configuration in the new CloudFormation template: https://amazon-eks.s3-us-west-2.amazonaws.com/cloudformation/2018-08-30/amazon-eks-nodegroup.yaml

For reference, the bootstrap script that is now baked in to the AMIs is here: https://github.com/awslabs/amazon-eks-ami/blob/master/files/bootstrap.sh

[aaron-trout]

Oh man, thanks so much for this! I never would have figured it out! 😆

Yeah, this is pretty annoying! What is more annoying though is the massive inconsistency in AWS where some resources have name as a first class citizen and some do not, and for some it must be unique... and some not. WTF Amazon!?

[tdmalone]

Thinking out loud here... would a possible workaround be interpolating a random UUID in to the cluster name? See https://www.terraform.io/docs/providers/random/r/uuid.html

[th31nitiate]

Oh man, thanks so much for this! I never would have figured it out! 😆

Yeah, this is pretty annoying! What is more annoying though is the massive inconsistency in AWS where some resources have named as a first-class citizen and some do not, and for some, it must be unique... and some not. WTF Amazon!?

Those operations and developer silos are the main cause of this. This could also be an onboarding problem but I was going to blame terraform but turns out is an Amazon thing. I was going to suggest to just use cloud formation but that is not an option in this scenario. I can see where data issues start to happen, how.

Thinking out loud here... would a possible workaround be interpolating a random UUID into the cluster name? See https://www.terraform.io/docs/providers/random/r/uuid.html

You can do, add a prefix to the cluster name. The only problem is that may create a new cluster for each randomly generated URL.

The plugin.terraform-provider-aws_v2.11.0_x4: X-Amzn-Errortype: ResourceInUseException issue is the worst one to be fair.

[jamesongithub]

Seems like aws_eks_cluster and aws_eks_node_group should add support for name_prefix.

[bflad]

There are now instructions about how to add naming generation to resources in the Contributing Guide.