Terraform: Can't setup a VPC peering with VPC in other account

[FransUrbo]

If I use auto_accept = false, then the peering connection will be created (but not setup correctly) and I go and accept the peering in the remote account.

Next run TF runs, it will fail to modify it:

aws_vpc_peering_connection.test-access: Modifying...
  requester.1102046665.allow_classic_link_to_remote_vpc: "" => "false"
  requester.1102046665.allow_remote_vpc_dns_resolution:  "" => "true"
  requester.1102046665.allow_vpc_to_remote_classic_link: "" => "false"
  requester.3244290774.allow_classic_link_to_remote_vpc: "false" => "false"
  requester.3244290774.allow_remote_vpc_dns_resolution:  "false" => "false"
  requester.3244290774.allow_vpc_to_remote_classic_link: "false" => "false"
[...]
* aws_vpc_peering_connection.test-access: Error modifying VPC Peering Connection options: OperationNotPermitted: User LOCAL_ACCOUNT_ID does not have permission to modify the accepter side peering options
        status code: 400, request id: 348f0ae0-21ca-4caf-ba30-fdeb07e8a109

Terraform Version

0.8.5

Affected Resource(s)

• aws_vpc_peering_connection

Terraform Configuration Files

resource "aws_vpc_peering_connection" "test-access" {
  vpc_id                            = "${aws_vpc.test.id}"

  peer_vpc_id                       = "${data.terraform_remote_state.core.access_vpc_id}"
  peer_owner_id                     = "${data.terraform_remote_state.core.account_id}"

  auto_accept                       = "false"

  accepter {
    allow_remote_vpc_dns_resolution = true
  }

  requester {
    allow_remote_vpc_dns_resolution = true
  }

  tags {
    Name                            = "test-access"
    environment                     = "test"
    location                        = "test"
    service                         = "network"
  }
}

Expected Behavior

TF should have created the peering connections with all the correct variables, attributes and other settings (but left it non-accepted), so when the user (me) goes into the remote account and accepts them, they're "all ready to go".

Actual Behavior

The peering connection was only partially created, requiring a modification of it after it had been accepted manually. Which fails because of lack of access rights.

[FransUrbo]

Also, the remote name of the peering connection was unset, even though I had tags set.

[FransUrbo]

And looking at the plan closer, TF tries to modify the local connection, even though it's not necessary

  requester.3244290774.allow_classic_link_to_remote_vpc: "false" => "false"
  requester.3244290774.allow_remote_vpc_dns_resolution:  "false" => "false"
  requester.3244290774.allow_vpc_to_remote_classic_link: "false" => "false"

[ewbankkit]

@FransUrbo You can use the aws_vpc_peering_connection_accepter resource on the accepter's side of a cross-account VPC Peering Connection.

[FransUrbo]

Not sure how to use this:

provider "aws" {
  region  = "eu-west-1"
  profile = "core"
  alias   = "core"
}

provider "aws" {
  region  = "eu-west-1"
  profile = "test"
  # Can't seem to be possible to have an "alias = test" here - TF asks me about region etc...
  # Not that it's necessary/needed, but still...
}

# === TEST -> ACCESS
resource "aws_vpc_peering_connection" "test-access" {
  vpc_id                            = "${aws_vpc.test.id}"

  peer_vpc_id                       = "${data.terraform_remote_state.core.access_vpc_id}"
  peer_owner_id                     = "${data.terraform_remote_state.core.account_id}"

  auto_accept                       = "false"

  accepter {
    allow_remote_vpc_dns_resolution = true
  }

  requester {
    allow_remote_vpc_dns_resolution = true
  }

  tags {
    Name                            = "test-access"
    environment                     = "test"
    location                        = "test"
    service                         = "network"
  }
}

resource "aws_vpc_peering_connection_accepter" "test-access" {
  provider                          = "aws.core"
  vpc_peering_connection_id         = "${aws_vpc_peering_connection.test-access.id}"

  auto_accept                       = true

  tags {
    Side                            = "Accepter"
    Name                            = "test-access"
    environment                     = "test"
    location                        = "test"
    service                         = "network"
  }
}

resource "aws_route" "test-access" {
  route_table_id                    = "${data.terraform_remote_state.core.access_route_table}"
  destination_cidr_block            = "${aws_vpc.test.cidr_block}"
  vpc_peering_connection_id         = "${aws_vpc_peering_connection.test-access.id}"
}

resource "aws_route" "access-test" {
  route_table_id                    = "${aws_route_table.test.id}"
  destination_cidr_block            = "${data.terraform_remote_state.core.access_vpc_cidr}"
  vpc_peering_connection_id         = "${aws_vpc_peering_connection.test-access.id}"
}

Gives me:

  * aws_vpc_peering_connection_accepter.test-main: Provider doesn't support resource: aws_vpc_peering_connection_accepter
  * aws_vpc_peering_connection_accepter.test-support: Provider doesn't support resource: aws_vpc_peering_connection_accepter
  * aws_vpc_peering_connection_accepter.test-management: Provider doesn't support resource: aws_vpc_peering_connection_accepter
  * aws_vpc_peering_connection_accepter.test-access: Provider doesn't support resource: aws_vpc_peering_connection_accepter
  * aws_vpc_peering_connection_accepter.test-jenkins: Provider doesn't support resource: aws_vpc_peering_connection_accepter

[FransUrbo]

Might have been a version thing. Trying it with v0.8.8 (instead of v0.8.5) seems to have worked, but I still get:

aws_vpc_peering_connection.test-access: Modifying...
  requester.1102046665.allow_classic_link_to_remote_vpc: "" => "false"
  requester.1102046665.allow_remote_vpc_dns_resolution:  "" => "true"
  requester.1102046665.allow_vpc_to_remote_classic_link: "" => "false"
  requester.3244290774.allow_classic_link_to_remote_vpc: "false" => "false"
  requester.3244290774.allow_remote_vpc_dns_resolution:  "false" => "false"
  requester.3244290774.allow_vpc_to_remote_classic_link: "false" => "false"
[...]
* aws_vpc_peering_connection.test-access: Unable to modify peering options. The VPC Peering Connection "pcx-aabbccdd" is not active. Please set `auto_accept` attribute to `true`, or activate VPC Peering Connection manually.

And I get that even if I go into the AWS Console and accept the pending requests.

Also, the remote (accepters) peering connection name isn't changed.. Nor apparently the allow_* options.

[FransUrbo]

Well, waiting ten, fifteen minutes (!!) seems to have solved that part at least (document this?).

But now I get:

* aws_vpc_peering_connection.test-access: Error modifying VPC Peering Connection options: OperationNotPermitted: User LOCAL_ACCOUNT_ID does not have permission to modify the accepter side peering options

But it still seem to use the local account ID, not the one I specified in the provider! So it almost looks like it doesn't use the provider option (correctly?).

[FransUrbo]

Ah, it was the

  accepter {
    allow_remote_vpc_dns_resolution = true
  }

  requester {
    allow_remote_vpc_dns_resolution = true
  }

parts! Once I removed those, it work.

I'm sensing documentation update(s) here!

[FransUrbo]

@paddyforan Replace bug with documentation perhaps?

[antvak]

I'd like to know how I can use this if I set my credentials with the environment variables? I run a job in Jenkins that runs Terraform and it sets the credentials in environment variables and I'd like to keep it that way. How can I use two sets of variables so that I can run the accepter-bits?

[ewbankkit]

@antvak Only one of the AWS providers (provider "aws" {} blocks) can use the environment variables. The other will have to use explicit values.

[FernandoMiguel]

thanks for the tip @FransUrbo
but setting dns resolution to false doesnt fix the issue

module.vpc_peer_tools_prodapp.aws_vpc_peering_connection.peer: Creating...
  accept_status:                                         "" => "<computed>"
  accepter.#:                                            "" => "1"
  accepter.XXX.allow_classic_link_to_remote_vpc:  "" => "false"
  accepter.XXX.allow_remote_vpc_dns_resolution:   "" => "false"
  accepter.XXX.allow_vpc_to_remote_classic_link:  "" => "false"
  auto_accept:                                           "" => "false"
  peer_owner_id:                                         "" => "XXX"
  peer_vpc_id:                                           "" => "vpc-XXX"
  requester.#:                                           "" => "1"
  requester.XXX.allow_classic_link_to_remote_vpc: "" => "false"
  requester.XXX.allow_remote_vpc_dns_resolution:  "" => "false"
  requester.XXX.allow_vpc_to_remote_classic_link: "" => "false"

  vpc_id:                                                "" => "vpc-XXX"
Error applying plan:

1 error(s) occurred:

* module.vpc_peer_tools_prodapp.aws_vpc_peering_connection.peer: 1 error(s) occurred:

* aws_vpc_peering_connection.peer: Unable to modify peering options. The VPC Peering Connection "pcx-XXX" is not active. Please set `auto_accept` attribute to `true`, or activate VPC Peering Connection manually.

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.