Terraform-provider-aws: Breaking change for aws_dx_gateway_association vpn_gateway_id parameter

[jdpaton]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

0.10.8

Affected Resource(s)

• aws_dx_gateway_association

Terraform Configuration Files

resource "aws_dx_gateway_association" "dx" {
  dx_gateway_id  = "${aws_dx_gateway.dx_gw.id}"
  vpn_gateway_id = "${aws_vpn_gateway._.id}"

  count = "${var.create_dx ? 1 : 0}"
}

With todays release of aws provider 2.12 we updated the parameter via the deprecation warning:

resource "aws_dx_gateway_association" "dx" {
  dx_gateway_id  = "${aws_dx_gateway.dx_gw.id}"
  aws_dx_gateway_association = "${aws_vpn_gateway._.id}"

  count = "${var.create_dx ? 1 : 0}"
}

Expected Behavior

Should have continued to work for parameter vpn_gateway_id without any breaking change related to this noted in the changelog.

Actual Behavior

```Error: Error refreshing state: 1 error(s) occurred:

• module.xxx_vpc.aws_dx_gateway_association.dx: 1 error(s) occurred:

• module.xxx_vpc.aws_dx_gateway_association.dx: aws_dx_gateway_association.dx: error reading Direct Connect gateway association (ga-7fc7cfe1-9bf8-47db-856a-160f5afa9551vgw-0723e965940fbede5): DirectConnectClientException: Association ID has an invalid format.
  status code: 400, request id: 3b6c2456-7e5a-11e9-9383-ab3c9ca45af6

error: Output does not contain proper preface. Use --trace to view backtrace
ERROR: Job failed: exit code 1
```

Steps to Reproduce

1. Upgrade aws provider to 2.12
2. See vpn_gateway_id will now fail to work in aws_dx_gateway_association resource.

Important Factoids

I don't think there is anything unusual about the association we have setup.

References

https://github.com/terraform-providers/terraform-provider-aws/blob/master/CHANGELOG.md#2120-may-24-2019

[jdpaton]

We've pinned the provider back to 2.11 for now and it's working again.

[ewbankkit]

@jdpaton Sorry for the confusion here. We are following the published best practices around attribute removal and we have just got to _3. Release a MINOR version with the deprecation._ so we have not yet provided automatic migration from vpn_gateway_id to associated_gateway_id and you should continue to use vpn_gateway_id for existing resources and use associated_gateway_id for new resources.

That said, I don't think that's the cause of the error message. I think I see what the problem is here - The new dx_gateway_association_id will be empty for existing resources and that is what we are now using during the read phase. I'll investigate...

[ewbankkit]

@jdpaton Do you know which version of the AWS provider your aws_dx_gateway_association resource was created with?

[jdpaton]

@ewbankkit Managed to find it from our CI job history:

• provider.aws: version = "~> 1.51"

[ewbankkit]

@jdpaton Thanks.
I can reproduce this with v2.7.0 as the dx_gateway_association_id attribute was introduced with https://github.com/terraform-providers/terraform-provider-aws/pull/8222 in v2.8.0.
I think we'll need to add a check for the existence of dx_gateway_association_id during the read phrase and if it's not available fall back to the old VGW ID/DX GW ID refresh method.

[matthiasr]

I'm working around this by doing a two-step provider upgrade:

1. upgrade AWS provider to ~> 2.11.0
2. terraform plan -out tf.plan -refresh=true
3. terraform apply tf.plan (even if it does not have any changes, this will push the refreshed state)
4. upgrade AWS provider to ~> 2.13
5. plan again -> works now

This is a roundabout way of doing the proper migration in #8776, but possible today.

[ewbankkit]

@matthiasr Thanks for documenting the workaround; Any provider version between 2.8.0 and 2.12.0 inclusive should work for the intermediate terraform apply.

[JagdeepChahal]

@ewbankkit - We are on Terraform version 11.14 and upgraded to AWS provider version to 2.18.
Your changes #8970 (2.16) were suppose to fix this but we are still getting the 'Association ID has an invalid format' error.
We are hit by (#9106) as well so we are not able to go for the above workaround also. Kindly look in to it, this is big blocker for us so we will really appreciate any urgent help!

error -
2019-07-11T18:27:00.179Z [DEBUG] plugin.terraform-provider-aws_v2.18.0_x4: 2019/07/11 18:27:00 [DEBUG] [aws-sdk-go] {"__type":"DirectConnectClientException","message":"Association ID has an invalid format."}
2019-07-11T18:27:00.179Z [DEBUG] plugin.terraform-provider-aws_v2.18.0_x4: 2019/07/11 18:27:00 [DEBUG] [aws-sdk-go] DEBUG: Validate Response directconnect/DescribeDirectConnectGatewayAssociations failed, attempt 0/25, error DirectConnectClientException: Association ID has an invalid format.
2019-07-11T18:27:00.179Z [DEBUG] plugin.terraform-provider-aws_v2.18.0_x4: status code: 400, request id: 8315b767-a409-11e9-9816-f1d876812434
2019/07/11 18:27:00 [ERROR] root.tf_mod_dx: eval: *terraform.EvalRefresh, err: aws_dx_gateway_association.dx_connect: error reading Direct Connect gateway association (ga-dd695e31-7713-4bbd-aa74-6cf82b330894vgw-0886ff71ad6933949): DirectConnectClientException: Association ID has an invalid format.
status code: 400, request id: 8315b767-a409-11e9-9816-f1d876812434
2019/07/11 18:27:00 [ERROR] root.tf_mod_dx: eval: *terraform.EvalSequence, err: aws_dx_gateway_association.dx_connect: error reading Direct Connect gateway association (ga-dd695e31-7713-4bbd-aa74-6cf82b330894vgw-0886ff71ad6933949): DirectConnectClientException: Association ID has an invalid format.
status code: 400, request id: 8315b767-a409-11e9-9816-f1d876812434
2019/07/11 18:27:00 [TRACE] [walkRefresh] Exiting eval tree: module.tf_mod_dx.aws_dx_gateway_association.dx_connect
2019/07/11 18:27:00 [TRACE] dag/walk: upstream errored, not walking "module.tf_mod_dx.output.dx_connect_id"

[JagdeepChahal]

I will create new ticket for this.

[ewbankkit]

~@JagdeepChahal Do you know what version of the AWS Provider was used to create the DX GW association initially (looking towards the top of the Terraform state file, if you have it available, should have that information)?
If the initial version was < 2.8.0 then try following the steps in the @matthiasr workaround - https://github.com/terraform-providers/terraform-provider-aws/issues/8773#issuecomment-498995330,~

[ewbankkit]

@JagdeepChahal Just noticed that you can't apply the workaround in your case.

[bflad]

The fix for this has been merged and will release in version 2.34.0 of the Terraform AWS Provider, on Thursday. Thanks to @ewbankkit for the fix implementation.

[bflad]

This has been released in version 2.34.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!