Terraform: Route Tables keep getting refreshed

[mcraig88]

Every time I run Terrafrom Plan/Apply, my route tables are getting refreshed/updated. Is there a way to stop this?

Note that I am only using aws_route_table resources,

Terraform Version

v0.7.6

Affected Resource(s)

aws_route_table

Terraform Configuration Files

#### Core Infrastructure Template - VPC

#### VPC

resource "aws_vpc" "main" {
cidr_block = "${var.cidr_block}"
enable_dns_hostnames = true
enable_dns_support = true
instance_tenancy = "default"

tags {
    "Name" = "${var.team} ${var.zone}"
    "Environment" = "${var.zone}"
    "data_classification" ="${var.data_classification}"
    "team" ="${var.team}"
}

}

#### GATEWAYS

Create necessary gateways for connectivity

resource "aws_internet_gateway" "igw" {
depends_on = ["aws_vpc.main"]
vpc_id = "${aws_vpc.main.id}"

tags {
    Name = "${var.zone} IGW"
}

}

resource "aws_eip" "nat" {
depends_on = ["aws_vpc.main"]
vpc = true
count = "${length(var.availability_zones)}"
}

resource "aws_nat_gateway" "ngw" {
depends_on = ["aws_internet_gateway.igw"]
count = "${length(var.availability_zones)}"
allocation_id = "${element(aws_eip.nat._.id, count.index)}"
subnet_id = "${element(aws_subnet.public._.id, count.index)}"
}

#### SUBNETS

resource "aws_subnet" "public" {
depends_on = ["aws_vpc.main"]
vpc_id = "${aws_vpc.main.id}"
count = "${length(var.dmz_subnet_cidr)}"
cidr_block = "${element(var.dmz_subnet_cidr, count.index)}"
availability_zone = "${element(var.availability_zones, count.index)}"
map_public_ip_on_launch = false

tags {
    "Name" = "${var.zone}-DMZ"
}

}

resource "aws_subnet" "private" {
depends_on = ["aws_vpc.main"]
vpc_id = "${aws_vpc.main.id}"
count = "${length(var.application_subnet_cidr)}"
cidr_block = "${element(var.application_subnet_cidr, count.index)}"
availability_zone = "${element(var.availability_zones, count.index)}"
map_public_ip_on_launch = false

tags {
    "Name" = "${var.zone}-Application"
}

}

resource "aws_subnet" "idx" {
depends_on = ["aws_vpc.main"]
vpc_id = "${aws_vpc.main.id}"
count = "${length(var.admin_subnet_cidr)}"
cidr_block = "${element(var.admin_subnet_cidr, count.index)}"
availability_zone = "${element(var.availability_zones, count.index)}"
map_public_ip_on_launch = false

tags {
    "Name" = "${var.zone}-IDX"
}

}

#### ROUTE TABLES

resource "aws_route_table" "public" {
vpc_id = "${aws_vpc.main.id}"
route {
cidr_block = "0.0.0.0/0"
gateway_id = "${aws_internet_gateway.igw.id}"
}
/* route {
cidr_block = "${var.ecp_shared_vpc_cidr}"
gateway_id = "${var.ecp_shared_vpc_id}"
}
*/ tags {
Name = "${var.zone} Public"
}
}

resource "aws_route_table" "private-cust" {
count = "${length(var.application_subnet_cidr)}"
vpc_id = "${aws_vpc.main.id}"
route {
cidr_block = "0.0.0.0/0"
gateway_id = "${element(aws_nat_gateway.ngw._.id, count.index)}"
}
/_ route {
cidr_block = "${var.ecp_shared_vpc_cidr}"
gateway_id = "${var.ecp_shared_vpc_id}"
}
_/ ## propagating_vgws = ["${var.direct_connect_vgw}"]
tags {
Name = "${var.zone} Private ${count.index}"
}
}
resource "aws_route_table" "idx-cust" {
count = "${length(var.admin_subnet_cidr)}"
vpc_id = "${aws_vpc.main.id}"
route {
cidr_block = "0.0.0.0/0"
gateway_id = "${element(aws_nat_gateway.ngw._.id, count.index)}"
}
/* route {
cidr_block = "${var.ecp_shared_vpc_cidr}"
gateway_id = "${var.ecp_shared_vpc_id}"
}
*/ ##propagating_vgws = ["${var.direct_connect_vgw}"]
tags {
Name = "${var.zone} IDX ${count.index}"
}
}

#### ROUTE TABLE ASSOCIATIONS

resource "aws_route_table_association" "public" {
count = "${length(var.dmz_subnet_cidr)}"
subnet_id = "${element(aws_subnet.public.*.id, count.index)}"
route_table_id = "${aws_route_table.public.id}"
}

resource "aws_route_table_association" "private-cust-rta" {
count = "${length(var.application_subnet_cidr)}"
subnet_id = "${element(aws_subnet.private._.id, count.index)}"
route_table_id = "${element(aws_route_table.private-cust._.id, count.index)}"
}
resource "aws_route_table_association" "idx-cust-rta" {
count = "${length(var.admin_subnet_cidr)}"
subnet_id = "${element(aws_subnet.idx._.id, count.index)}"
route_table_id = "${element(aws_route_table.idx-cust._.id, count.index)}"
}

#### OUTPUTS

output "zone" {
value = "${var.zone}"
}

output "cidr_block" {
value = "${var.cidr_block}"
}

output "vpc_id" {
value = "${aws_vpc.main.id}"
}

output "ngw_id" {
value = "${join(",", aws_eip.nat.*.id)}"
}

output "public_subnet_id" {
value = "${join(",", aws_subnet.public.*.id)}"
}

output "idx_subnet_id" {
value = "${join(",", aws_subnet.idx.*.id)}"
}

output "private_subnet_id" {
value = "${join(",", aws_subnet.private.*.id)}"
}

output "dmz_subnet_cidr" {
value = "${var.dmz_subnet_cidr}"
}

output "admin_subnet_cidr" {
value = "${var.admin_subnet_cidr}"
}

output "application_subnet_cidr" {
value = "${var.application_subnet_cidr}"
}
/*
output "public_route_id" {
value = "${aws_route_table.public.id}"
}

output "private_route_id" {
value = "${aws_route_table.private.id}"
}
*/
output "igw_gateway_id"{
value= "${aws_internet_gateway.igw.id}"
}

Debug Output

Path: .\plan.out

~ aws_route_table.idx-cust.0
route.1288548767.cidr_block: "0.0.0.0/0" => ""
route.1288548767.gateway_id: "" => ""
route.1288548767.instance_id: "" => ""
route.1288548767.nat_gateway_id: "nat-022878db1dc9b6e17" => ""
route.1288548767.network_interface_id: "" => ""
route.1288548767.vpc_peering_connection_id: "" => ""
route.2875932809.cidr_block: "" => "0.0.0.0/0"
route.2875932809.gateway_id: "" => "nat-022878db1dc9b6e17"
route.2875932809.instance_id: "" => ""
route.2875932809.nat_gateway_id: "" => ""
route.2875932809.network_interface_id: "" => ""
route.2875932809.vpc_peering_connection_id: "" => ""

~ aws_route_table.idx-cust.1
route.2115746285.cidr_block: "0.0.0.0/0" => ""
route.2115746285.gateway_id: "" => ""
route.2115746285.instance_id: "" => ""
route.2115746285.nat_gateway_id: "nat-09d41043d0cb8e07e" => ""
route.2115746285.network_interface_id: "" => ""
route.2115746285.vpc_peering_connection_id: "" => ""
route.3099189529.cidr_block: "" => "0.0.0.0/0"
route.3099189529.gateway_id: "" => "nat-09d41043d0cb8e07e"
route.3099189529.instance_id: "" => ""
route.3099189529.nat_gateway_id: "" => ""
route.3099189529.network_interface_id: "" => ""
route.3099189529.vpc_peering_connection_id: "" => ""

~ aws_route_table.idx-cust.2
route.3542914727.cidr_block: "0.0.0.0/0" => ""
route.3542914727.gateway_id: "" => ""
route.3542914727.instance_id: "" => ""
route.3542914727.nat_gateway_id: "nat-093cbf5da68e3c359" => ""
route.3542914727.network_interface_id: "" => ""
route.3542914727.vpc_peering_connection_id: "" => ""
route.436131576.cidr_block: "" => "0.0.0.0/0"
route.436131576.gateway_id: "" => "nat-093cbf5da68e3c359"
route.436131576.instance_id: "" => ""
route.436131576.nat_gateway_id: "" => ""
route.436131576.network_interface_id: "" => ""
route.436131576.vpc_peering_connection_id: "" => ""

~ aws_route_table.private-cust.0
route.1288548767.cidr_block: "0.0.0.0/0" => ""
route.1288548767.gateway_id: "" => ""
route.1288548767.instance_id: "" => ""
route.1288548767.nat_gateway_id: "nat-022878db1dc9b6e17" => ""
route.1288548767.network_interface_id: "" => ""
route.1288548767.vpc_peering_connection_id: "" => ""
route.2875932809.cidr_block: "" => "0.0.0.0/0"
route.2875932809.gateway_id: "" => "nat-022878db1dc9b6e17"
route.2875932809.instance_id: "" => ""
route.2875932809.nat_gateway_id: "" => ""
route.2875932809.network_interface_id: "" => ""
route.2875932809.vpc_peering_connection_id: "" => ""

~ aws_route_table.private-cust.1
route.2115746285.cidr_block: "0.0.0.0/0" => ""
route.2115746285.gateway_id: "" => ""
route.2115746285.instance_id: "" => ""
route.2115746285.nat_gateway_id: "nat-09d41043d0cb8e07e" => ""
route.2115746285.network_interface_id: "" => ""
route.2115746285.vpc_peering_connection_id: "" => ""
route.3099189529.cidr_block: "" => "0.0.0.0/0"
route.3099189529.gateway_id: "" => "nat-09d41043d0cb8e07e"
route.3099189529.instance_id: "" => ""
route.3099189529.nat_gateway_id: "" => ""
route.3099189529.network_interface_id: "" => ""
route.3099189529.vpc_peering_connection_id: "" => ""

~ aws_route_table.private-cust.2
route.3542914727.cidr_block: "0.0.0.0/0" => ""
route.3542914727.gateway_id: "" => ""
route.3542914727.instance_id: "" => ""
route.3542914727.nat_gateway_id: "nat-093cbf5da68e3c359" => ""
route.3542914727.network_interface_id: "" => ""
route.3542914727.vpc_peering_connection_id: "" => ""
route.436131576.cidr_block: "" => "0.0.0.0/0"
route.436131576.gateway_id: "" => "nat-093cbf5da68e3c359"
route.436131576.instance_id: "" => ""
route.436131576.nat_gateway_id: "" => ""
route.436131576.network_interface_id: "" => ""
route.436131576.vpc_peering_connection_id: "" => ""

~ aws_route_table.private-cust.3
route.1288548767.cidr_block: "0.0.0.0/0" => ""
route.1288548767.gateway_id: "" => ""
route.1288548767.instance_id: "" => ""
route.1288548767.nat_gateway_id: "nat-022878db1dc9b6e17" => ""
route.1288548767.network_interface_id: "" => ""
route.1288548767.vpc_peering_connection_id: "" => ""
route.2875932809.cidr_block: "" => "0.0.0.0/0"
route.2875932809.gateway_id: "" => "nat-022878db1dc9b6e17"
route.2875932809.instance_id: "" => ""
route.2875932809.nat_gateway_id: "" => ""
route.2875932809.network_interface_id: "" => ""
route.2875932809.vpc_peering_connection_id: "" => ""

~ aws_route_table.private-cust.4
route.2115746285.cidr_block: "0.0.0.0/0" => ""
route.2115746285.gateway_id: "" => ""
route.2115746285.instance_id: "" => ""
route.2115746285.nat_gateway_id: "nat-09d41043d0cb8e07e" => ""
route.2115746285.network_interface_id: "" => ""
route.2115746285.vpc_peering_connection_id: "" => ""
route.3099189529.cidr_block: "" => "0.0.0.0/0"
route.3099189529.gateway_id: "" => "nat-09d41043d0cb8e07e"
route.3099189529.instance_id: "" => ""
route.3099189529.nat_gateway_id: "" => ""
route.3099189529.network_interface_id: "" => ""
route.3099189529.vpc_peering_connection_id: "" => ""

~ aws_route_table.private-cust.5
route.3542914727.cidr_block: "0.0.0.0/0" => ""
route.3542914727.gateway_id: "" => ""
route.3542914727.instance_id: "" => ""
route.3542914727.nat_gateway_id: "nat-093cbf5da68e3c359" => ""
route.3542914727.network_interface_id: "" => ""
route.3542914727.vpc_peering_connection_id: "" => ""
route.436131576.cidr_block: "" => "0.0.0.0/0"
route.436131576.gateway_id: "" => "nat-093cbf5da68e3c359"
route.436131576.instance_id: "" => ""
route.436131576.nat_gateway_id: "" => ""
route.436131576.network_interface_id: "" => ""
route.436131576.vpc_peering_connection_id: "" => ""

~ aws_route_table.private-cust.6
route.1288548767.cidr_block: "0.0.0.0/0" => ""
route.1288548767.gateway_id: "" => ""
route.1288548767.instance_id: "" => ""
route.1288548767.nat_gateway_id: "nat-022878db1dc9b6e17" => ""
route.1288548767.network_interface_id: "" => ""
route.1288548767.vpc_peering_connection_id: "" => ""
route.2875932809.cidr_block: "" => "0.0.0.0/0"
route.2875932809.gateway_id: "" => "nat-022878db1dc9b6e17"
route.2875932809.instance_id: "" => ""
route.2875932809.nat_gateway_id: "" => ""
route.2875932809.network_interface_id: "" => ""
route.2875932809.vpc_peering_connection_id: "" => ""

~ aws_route_table.private-cust.7
route.2115746285.cidr_block: "0.0.0.0/0" => ""
route.2115746285.gateway_id: "" => ""
route.2115746285.instance_id: "" => ""
route.2115746285.nat_gateway_id: "nat-09d41043d0cb8e07e" => ""
route.2115746285.network_interface_id: "" => ""
route.2115746285.vpc_peering_connection_id: "" => ""
route.3099189529.cidr_block: "" => "0.0.0.0/0"
route.3099189529.gateway_id: "" => "nat-09d41043d0cb8e07e"
route.3099189529.instance_id: "" => ""
route.3099189529.nat_gateway_id: "" => ""
route.3099189529.network_interface_id: "" => ""
route.3099189529.vpc_peering_connection_id: "" => ""

~ aws_route_table.private-cust.8
route.3542914727.cidr_block: "0.0.0.0/0" => ""
route.3542914727.gateway_id: "" => ""
route.3542914727.instance_id: "" => ""
route.3542914727.nat_gateway_id: "nat-093cbf5da68e3c359" => ""
route.3542914727.network_interface_id: "" => ""
route.3542914727.vpc_peering_connection_id: "" => ""
route.436131576.cidr_block: "" => "0.0.0.0/0"
route.436131576.gateway_id: "" => "nat-093cbf5da68e3c359"
route.436131576.instance_id: "" => ""
route.436131576.nat_gateway_id: "" => ""
route.436131576.network_interface_id: "" => ""
route.436131576.vpc_peering_connection_id: "" => ""

Expected Behavior

When underlying resources have not changed, I would expect no 'changes' to be detected and applied to resources.

Actual Behavior

With no change to underlying aws_route_table mappings, the route table continuously get modified with each execution.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

Important Factoids

References

Other possibly related issues:
GH-7303
GH-6832

[catsby]

Hello @mcraig88 – #6832 is certainly the same symptoms and it looks like that same issue I mention there, specifically:

for your aws_route_table, looks like you're specifying a Nat ID in the Gateway ID attribute

In the configuration you've shared it seems that way here as well:

route {
cidr_block = "0.0.0.0/0"
gateway_id = "${element(aws_nat_gateway.ngw..id, count.index)}"
}

You can see this in your diffs, although I admit it can be overwhelming and hard to spot right away:

~ aws_route_table.private-cust.7
route.2115746285.nat_gateway_id: "nat-09d41043d0cb8e07e" => ""
route.3099189529.gateway_id: "" => "nat-09d41043d0cb8e07e"

you should change gateway_id there to be nat_gateway_id in your configuration. I recommend reviewing the rest of your configuration for other similar issues.

Let me know if you have any other issues

[mcraig88]

Sorry I figured this out.

For the DMZ route table, connecting to the IGW, I would use gateway_id.
In the case of a route table connecting to a NGW, I would use
nat_gateway_id.

Whew!

Thank you very much, HUGE help!

On Fri, Oct 21, 2016 at 2:57 PM, Mike Craig [email protected] wrote:

That seems to have fixed this issue, is this also true of route tables
connecting to an IGW?
I see the same symptoms in my diff, for my DMZ route table.

Should I be able to put my igw id in the nat_gateway_id field as well for
that route table?

On Fri, Oct 21, 2016 at 2:23 PM, Clint [email protected] wrote:

Hello @mcraig88 https://github.com/mcraig88 – #6832
https://github.com/hashicorp/terraform/issues/6832 is certainly the
same symptoms and it looks like that same issue I mention there,
specifically:

for your aws_route_table, looks like you're specifying a Nat ID in the Gateway ID attribute

In the configuration you've shared it seems that way here as well:

route {
cidr_block = "0.0.0.0/0"
gateway_id = "${element(aws_nat_gateway.ngw..id, count.index)}"
}

You can see this in your diffs, although I admit it can be overwhelming
and hard to spot right away:

~ aws_route_table.private-cust.7
route.2115746285.nat_gateway_id: "nat-09d41043d0cb8e07e" => ""
route.3099189529.gateway_id: "" => "nat-09d41043d0cb8e07e"

you should change gateway_id there to be nat_gateway_id. I recommend
reviewing your configuration for other similar issues.

Let me know if you have any other issues

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/terraform/issues/9507#issuecomment-255468711,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIFybQ-bc6Q0nVT_AhHBjq6thzWrkXpfks5q2S1lgaJpZM4KdX-a
.

[mcraig88]

That seems to have fixed this issue, is this also true of route tables
connecting to an IGW?
I see the same symptoms in my diff, for my DMZ route table.

Should I be able to put my igw id in the nat_gateway_id field as well for
that route table?

On Fri, Oct 21, 2016 at 2:23 PM, Clint [email protected] wrote:

Hello @mcraig88 https://github.com/mcraig88 – #6832
https://github.com/hashicorp/terraform/issues/6832 is certainly the
same symptoms and it looks like that same issue I mention there,
specifically:

for your aws_route_table, looks like you're specifying a Nat ID in the Gateway ID attribute

In the configuration you've shared it seems that way here as well:

route {
cidr_block = "0.0.0.0/0"
gateway_id = "${element(aws_nat_gateway.ngw..id, count.index)}"
}

You can see this in your diffs, although I admit it can be overwhelming
and hard to spot right away:

~ aws_route_table.private-cust.7
route.2115746285.nat_gateway_id: "nat-09d41043d0cb8e07e" => ""
route.3099189529.gateway_id: "" => "nat-09d41043d0cb8e07e"

you should change gateway_id there to be nat_gateway_id. I recommend
reviewing your configuration for other similar issues.

Let me know if you have any other issues

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/terraform/issues/9507#issuecomment-255468711,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIFybQ-bc6Q0nVT_AhHBjq6thzWrkXpfks5q2S1lgaJpZM4KdX-a
.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.