Terraform-provider-aws: aws_ec2_transit_gateway_vpc_attachment does not set transit_gateway_default_route_table_association or transit_gateway_default_route_table_propagation correctly

[BrandonsAccount]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Issue reproduced on 11.11 and 11.13.

I can confirm that this was not a bug in aws provider version 2.0.0. The issue exists in version 2.7.0.

Affected Resource(s)

• aws_ec2_transit_gateway_vpc_attachment

Terraform Configuration Files

resource "aws_ec2_transit_gateway_vpc_attachment" "this" {
  subnet_ids                                      = ["${aws_subnet.private.*.id}"]
  transit_gateway_id                              = "${var.transit_gateway_id}"
  vpc_id                                          = "${aws_vpc.this.id}"
  transit_gateway_default_route_table_association = false
  transit_gateway_default_route_table_propagation = false
  tags = "${merge(var.tags, map("Name", format("%s-attachment", var.name)))}"
}

This is the plan output

  + aws_ec2_transit_gateway_vpc_attachment.this
      id:                                              <computed>
      dns_support:                                     "enable"
      ipv6_support:                                    "disable"
      subnet_ids.#:                                    "4"
      subnet_ids.104XXXXXX:                           "subnet-02b3XXXXXXXXXXX"
      subnet_ids.178XXXXXX:                           "subnet-0cb3XXXXXXXXXXX"
      subnet_ids.207XXXXXX:                           "subnet-0daXXXXXXXXXXX"
      subnet_ids.264148865:                            "subnet-001XXXXXXXXXXXX"
      tags.%:                                          "1"
      tags.Name:                                       "vpc-attachment"
      transit_gateway_default_route_table_association: "false"
      transit_gateway_default_route_table_propagation: "false"
      transit_gateway_id:                              "tgw-XXXXXXXXXXXXXXXXXX"
      vpc_id:                                          "vpc-0c8XXXXXXXXXXXXXXXXXX"
      vpc_owner_id:                                    <computed>

Expected Behavior

Running terraform state show aws_ec2_transit_gateway_vpc_attachment.this should show this...

transit_gateway_default_route_table_association = false
transit_gateway_default_route_table_propagation = false

It also should not detect drift immediately after applying these changes.

Actual Behavior

Running terraform state show aws_ec2_transit_gateway_vpc_attachment.this shows this...

transit_gateway_default_route_table_association = true
transit_gateway_default_route_table_propagation = true

When running terraform apply a second time, the following drift is reported (but the apply runs successfully). It looks like this...

  ~ aws_ec2_transit_gateway_vpc_attachment.this
      transit_gateway_default_route_table_association: "true" => "false"
      transit_gateway_default_route_table_propagation: "true" => "false"

The terraform state shows the correct values after the 2nd apply...

transit_gateway_default_route_table_association = false
transit_gateway_default_route_table_propagation = false

Every terraform apply moving forward continues to detect drift though...

  ~ aws_ec2_transit_gateway_vpc_attachment.this
      transit_gateway_default_route_table_association: "true" => "false"
      transit_gateway_default_route_table_propagation: "true" => "false"

Steps to Reproduce

There's a fair amount of setup involved so I don't know that I can cover it all...

1. Create an organization within your aws account
2. Create an aws account for your transit gateway
3. Create a transit gateway with default route table association and propagation disabled
4. Create two route tables for the tgw. One to associate with and one to propagate to
5. Use RAM to share the tgw to all accounts in your org
6. Create an aws account and create a VPC in it
7. In this account use terraform to attach the VPC to the transit gateway that should be shared with it

Important Factoids

• The transit gateway attachment creates successfully and associates successfully with the tgw. It even propagates successfully

[furtber]

I have basically a "me-too" here, but on 0.12-beta2

+ terraform --version -no-color
Terraform v0.12.0-beta2
+ provider.aws v2.7.0

and I'm sure it was the same with this combo

+ terraform --version -no-color
Terraform v0.12.0-beta1
+ provider.aws v2.6.0

[jeffmccollum]

We are having this issue as well. One thing to note, attachments for VPCs in the same account as the transit gateway do not exhibit this behavior. This only occurs with attachments that are created in another account using RAM.

[ewbankkit]

Related:

• https://github.com/terraform-providers/terraform-provider-aws/issues/8600

[ewbankkit]

@BrandonsAccount @piersf @jeffmccollum @furtber
The transit_gateway_default_route_table_association and transit_gateway_default_route_table_propagation attributes don't do anything for cross-account VPC attachments as hinted at in the documentation:

This cannot be configured or perform drift detection with Resource Access Manager shared EC2 Transit Gateways.

but I agree we should do something to try and stop this drift being detected in case the attributes are set mistakenly, especially since a second terraform apply for me gave an error:

aws_ec2_transit_gateway_vpc_attachment.test: Modifying... (ID: tgw-attach-0000000000000000)
  transit_gateway_default_route_table_association: "true" => "false"
  transit_gateway_default_route_table_propagation: "true" => "false"

Error: Error applying plan:

1 error(s) occurred:

* aws_ec2_transit_gateway_vpc_attachment.test: 1 error(s) occurred:

* aws_ec2_transit_gateway_vpc_attachment.test: error updating EC2 Transit Gateway Attachment (tgw-attach-0000000000000000) Route Table (tgw-rtb-0000000000000000) association: error determining EC2 Transit Gateway Attachment Route Table (tgw-rtb-0000000000000000) association (tgw-attach-0000000000000000): InvalidRouteTableID.NotFound: Transit Gateway Route Table tgw-rtb-0000000000000000 was deleted or does not exist.
    status code: 400, request id: 2993f92d-230f-46f0-a595-21729fe95789

For cross-account VPC attachments maybe the correct resource to specify the default route table attributes on is the vpc_attachment_accepter resource? See https://github.com/terraform-providers/terraform-provider-aws/pull/8679#issuecomment-493797516.

[michelzanini]

This is my Transit Gateway:

module "transit_gateway" {
  source  = "terraform-aws-modules/transit-gateway/aws"
  version = "~> 1.0"
  name    = "transit-gateway"

  enable_auto_accept_shared_attachments  = true
  enable_default_route_table_association = false
  enable_default_route_table_propagation = false

  ram_principals = var.ram_principals
}

As you see enable_default_route_table_association and enable_default_route_table_propagation are false as that's what I need.

Then my VPC attachment initially was this:

resource "aws_ec2_transit_gateway_vpc_attachment" "vpc_attachment" {
  transit_gateway_id = var.transit_gateway_id
  vpc_id             = var.vpc_id
  subnet_ids         = var.subnet_ids

  transit_gateway_default_route_table_association = false
  transit_gateway_default_route_table_propagation = false
}

But, by using the above I run into this issue.
It works ok on the same account as the transit gateway, but it ALWAYS detect drift on state on accounts that I shared the gateway with:

transit_gateway_default_route_table_association: "true" => "false"
transit_gateway_default_route_table_propagation: "true" => "false"

To "escape" from this drift issue I set it to true. Which then "resolves" the issue for the accounts which I have shared the TGW with.
But, then when running this same code (which is a shared module for me) on the same account as the TGW it fails with the error found on issue https://github.com/terraform-providers/terraform-provider-aws/issues/13512.

So in the end, either I have the issue here https://github.com/terraform-providers/terraform-provider-aws/issues/8383 when using false or I have the issue there https://github.com/terraform-providers/terraform-provider-aws/issues/13512 when using true.

So in the end what I have done was this:

data "aws_caller_identity" "current" {}

data "aws_ec2_transit_gateway" "transit_gateway" {
  id = var.transit_gateway_id
}

locals {
  aws_account_id           = data.aws_caller_identity.current.account_id
  transit_gateway_owner_id = data.aws_ec2_transit_gateway.transit_gateway.owner_id

  different_account_as_transit_gateway = local.aws_account_id != local.transit_gateway_owner_id
}

resource "aws_ec2_transit_gateway_vpc_attachment" "vpc_attachment" {
  for_each = var.vpc_subnets_to_attach

  transit_gateway_id = var.transit_gateway_id
  vpc_id             = var.vpc_id
  subnet_ids         = var.subnet_ids

  transit_gateway_default_route_table_association = local.different_account_as_transit_gateway
  transit_gateway_default_route_table_propagation = local.different_account_as_transit_gateway
}

This detects if the account is the same or not as the TGW and sets the values to "escape" both issues.

But what I really need is this drift issue to be fixed so I can come back to set transit_gateway_default_route_table_association and transit_gateway_default_route_table_propagation to false as they should be in my use case.

[michelzanini]

Any chance we can fix this annoying problem?

[michelzanini]

Another solution as workaround:

  transit_gateway_default_route_table_association = false
  transit_gateway_default_route_table_propagation = false

  lifecycle {
    ignore_changes = [
      transit_gateway_default_route_table_association,
      transit_gateway_default_route_table_propagation
    ]
  }