Terraform: taint for resource with count > 1 affects all resources referring to it with element()

Created on 10 May 2017  ·  5Comments  ·  Source: hashicorp/terraform

When you have a secondary resource with a count > 1 that references another resource, with the same count, via the element function it treats all of the secondary resources as tainted if a single item in the primary resource is tainted.

Terraform Version

Terraform v0.9.2

Affected Resource(s)

Please list the resources as a list, for example:

  • openstack_compute_instance_v2
  • openstack_compute_floatingip_associate_v2
  • openstack_compute_floatingip_v2

Terraform Configuration Files

variable "num-nodes" {
  default = "3"
}

variable "use_floating_ip" {
  default = "1"
}

resource "openstack_compute_instance_v2" "cmgeneric" {
  count = "${var.num-nodes}"
  name  = "tf-bug-example-${format("%02d", count.index+1)}"

  key_pair = "rtb-tm-sjc-1b"
  flavor_name = "m1.medium"
  image_name = "ubuntu-16.04-x86_64"
  security_groups = ["default"]

  network =  {
    name = "rtb-own-network"
  }

}

resource "openstack_compute_floatingip_v2" "fip" {
  count = "${var.use_floating_ip > 0 ? var.num-nodes : 0}"
  pool = "public"
}

resource "openstack_compute_floatingip_associate_v2" "fip" {
  count       = "${var.use_floating_ip > 0 ? var.num-nodes : 0}"

  floating_ip = "${element(openstack_compute_floatingip_v2.fip.*.address, count.index)}"
  instance_id = "${element(openstack_compute_instance_v2.cmgeneric.*.id, count.index)}"
}

terraform apply output:

openstack_compute_floatingip_v2.fip.1: Creating...
  address:     "" => "<computed>"
  fixed_ip:    "" => "<computed>"
  instance_id: "" => "<computed>"
  pool:        "" => "public"
  region:      "" => "RegionOne"
openstack_compute_floatingip_v2.fip.0: Creating...
  address:     "" => "<computed>"
  fixed_ip:    "" => "<computed>"
  instance_id: "" => "<computed>"
  pool:        "" => "public"
  region:      "" => "RegionOne"
openstack_compute_floatingip_v2.fip.2: Creating...
  address:     "" => "<computed>"
  fixed_ip:    "" => "<computed>"
  instance_id: "" => "<computed>"
  pool:        "" => "public"
  region:      "" => "RegionOne"
openstack_compute_instance_v2.cmgeneric.0: Creating...
  access_ip_v4:               "" => "<computed>"
  access_ip_v6:               "" => "<computed>"
  all_metadata.%:             "" => "<computed>"
  availability_zone:          "" => "<computed>"
  flavor_id:                  "" => "<computed>"
  flavor_name:                "" => "m1.medium"
  force_delete:               "" => "false"
  image_id:                   "" => "<computed>"
  image_name:                 "" => "ubuntu-16.04-x86_64"
  key_pair:                   "" => "rtb-tm-sjc-1b"
  name:                       "" => "tf-bug-example-01"
  network.#:                  "" => "1"
  network.0.access_network:   "" => "false"
  network.0.fixed_ip_v4:      "" => "<computed>"
  network.0.fixed_ip_v6:      "" => "<computed>"
  network.0.floating_ip:      "" => "<computed>"
  network.0.mac:              "" => "<computed>"
  network.0.name:             "" => "rtb-own-network"
  network.0.port:             "" => "<computed>"
  network.0.uuid:             "" => "<computed>"
  region:                     "" => "RegionOne"
  security_groups.#:          "" => "1"
  security_groups.3814588639: "" => "default"
  stop_before_destroy:        "" => "false"
openstack_compute_instance_v2.cmgeneric.2: Creating...
  access_ip_v4:               "" => "<computed>"
  access_ip_v6:               "" => "<computed>"
  all_metadata.%:             "" => "<computed>"
  availability_zone:          "" => "<computed>"
  flavor_id:                  "" => "<computed>"
  flavor_name:                "" => "m1.medium"
  force_delete:               "" => "false"
  image_id:                   "" => "<computed>"
  image_name:                 "" => "ubuntu-16.04-x86_64"
  key_pair:                   "" => "rtb-tm-sjc-1b"
  name:                       "" => "tf-bug-example-03"
  network.#:                  "" => "1"
  network.0.access_network:   "" => "false"
  network.0.fixed_ip_v4:      "" => "<computed>"
  network.0.fixed_ip_v6:      "" => "<computed>"
  network.0.floating_ip:      "" => "<computed>"
  network.0.mac:              "" => "<computed>"
  network.0.name:             "" => "rtb-own-network"
  network.0.port:             "" => "<computed>"
  network.0.uuid:             "" => "<computed>"
  region:                     "" => "RegionOne"
  security_groups.#:          "" => "1"
  security_groups.3814588639: "" => "default"
  stop_before_destroy:        "" => "false"
openstack_compute_instance_v2.cmgeneric.1: Creating...
  access_ip_v4:               "" => "<computed>"
  access_ip_v6:               "" => "<computed>"
  all_metadata.%:             "" => "<computed>"
  availability_zone:          "" => "<computed>"
  flavor_id:                  "" => "<computed>"
  flavor_name:                "" => "m1.medium"
  force_delete:               "" => "false"
  image_id:                   "" => "<computed>"
  image_name:                 "" => "ubuntu-16.04-x86_64"
  key_pair:                   "" => "rtb-tm-sjc-1b"
  name:                       "" => "tf-bug-example-02"
  network.#:                  "" => "1"
  network.0.access_network:   "" => "false"
  network.0.fixed_ip_v4:      "" => "<computed>"
  network.0.fixed_ip_v6:      "" => "<computed>"
  network.0.floating_ip:      "" => "<computed>"
  network.0.mac:              "" => "<computed>"
  network.0.name:             "" => "rtb-own-network"
  network.0.port:             "" => "<computed>"
  network.0.uuid:             "" => "<computed>"
  region:                     "" => "RegionOne"
  security_groups.#:          "" => "1"
  security_groups.3814588639: "" => "default"
  stop_before_destroy:        "" => "false"
openstack_compute_floatingip_v2.fip.0: Creation complete (ID: 1a4a4416-...e8067db4)
openstack_compute_floatingip_v2.fip.2: Creation complete (ID: 093edfa8-...2c021000)
openstack_compute_floatingip_v2.fip.1: Creation complete (ID: 58be6ca3-...9785839e)
openstack_compute_instance_v2.cmgeneric.0: Still creating... (10s elapsed)
openstack_compute_instance_v2.cmgeneric.2: Still creating... (10s elapsed)
openstack_compute_instance_v2.cmgeneric.1: Still creating... (10s elapsed)
openstack_compute_instance_v2.cmgeneric.0: Still creating... (20s elapsed)
openstack_compute_instance_v2.cmgeneric.2: Still creating... (20s elapsed)
openstack_compute_instance_v2.cmgeneric.1: Still creating... (20s elapsed)
openstack_compute_instance_v2.cmgeneric.0: Creation complete (ID: 1ad39e8d-...42f4928d)
openstack_compute_instance_v2.cmgeneric.1: Creation complete (ID: a0c00a5f-...57ed48c2)
openstack_compute_instance_v2.cmgeneric.2: Creation complete (ID: a17c6ea8-...0e71ec0c)
openstack_compute_floatingip_associate_v2.fip.1: Creating...
  floating_ip: "" => "REDACTED"
  instance_id: "" => "a0c00a5f-e75f-47a1-a5ee-93e957ed48c2"
  region:      "" => "RegionOne"
openstack_compute_floatingip_associate_v2.fip.2: Creating...
  floating_ip: "" => "REDACTED"
  instance_id: "" => "a17c6ea8-fcbb-4c10-ae4e-308f0e71ec0c"
  region:      "" => "RegionOne"
openstack_compute_floatingip_associate_v2.fip.0: Creating...
  floating_ip: "" => "REDACTED"
  instance_id: "" => "1ad39e8d-83c8-4588-8f0c-3df542f4928d"
  region:      "" => "RegionOne"
openstack_compute_floatingip_associate_v2.fip.0: Creation complete (ID: REDACTED...2f4928d/)
openstack_compute_floatingip_associate_v2.fip.1: Creation complete (ID: REDACTED...7ed48c2/)
openstack_compute_floatingip_associate_v2.fip.2: Creation complete (ID: REDACTED...e71ec0c/)

Apply complete! Resources: 9 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path:

Expected Behavior

Running terraform taint on only 1 of the openstack_compute_instance_v2.cmgeneric resources should only recreate the single tainted openstack_compute_instance_v2 and the associated openstack_compute_floatingip_associate_v2 resource instead of all the openstack_compute_floatingip_associate_v2 resources.

Actual Behavior

It considers all openstack_compute_floatingip_associate_v2 resources need to be recreated.

Steps to Reproduce

First I taint an instance with terraform taint openstack_compute_instance_v2.cmgeneric.2

The resource openstack_compute_instance_v2.cmgeneric.2 in the module root has been marked as tainted!

Then when I run terraform apply it recreated all 3 openstack_compute_floatingip_associate_v2 resources.

openstack_compute_floatingip_v2.fip.2: Refreshing state... (ID: 093edfa8-...2c021000)
openstack_compute_floatingip_v2.fip.1: Refreshing state... (ID: 58be6ca3-...9785839e)
openstack_compute_instance_v2.cmgeneric.0: Refreshing state... (ID: 1ad39e8d-...42f4928d)
openstack_compute_instance_v2.cmgeneric.2: Refreshing state... (ID: a17c6ea8-...0e71ec0c)
openstack_compute_floatingip_v2.fip.0: Refreshing state... (ID: 1a4a4416-...e8067db4)
openstack_compute_instance_v2.cmgeneric.1: Refreshing state... (ID: a0c00a5f-...57ed48c2)
openstack_compute_floatingip_associate_v2.fip.0: Refreshing state... (ID: REDACTED...2f4928d/)
openstack_compute_floatingip_associate_v2.fip.2: Refreshing state... (ID: REDACTED...e71ec0c/)
openstack_compute_floatingip_associate_v2.fip.1: Refreshing state... (ID: REDACTED...7ed48c2/)
openstack_compute_floatingip_associate_v2.fip.1: Destroying... (ID: REDACTED...7ed48c2/)
openstack_compute_floatingip_associate_v2.fip.0: Destroying... (ID: REDACTED...2f4928d/)
openstack_compute_floatingip_associate_v2.fip.2: Destroying... (ID: REDACTED...e71ec0c/)
openstack_compute_floatingip_associate_v2.fip.2: Destruction complete
openstack_compute_floatingip_associate_v2.fip.0: Destruction complete
openstack_compute_floatingip_associate_v2.fip.1: Destruction complete
openstack_compute_instance_v2.cmgeneric.2: Destroying... (ID: a17c6ea8-...0e71ec0c)
openstack_compute_instance_v2.cmgeneric.2: Still destroying... (ID: a17c6ea8-...0e71ec0c, 10s elapsed)
openstack_compute_instance_v2.cmgeneric.2: Destruction complete
openstack_compute_instance_v2.cmgeneric.2: Creating...
  access_ip_v4:               "" => "<computed>"
  access_ip_v6:               "" => "<computed>"
  all_metadata.%:             "" => "<computed>"
  availability_zone:          "" => "<computed>"
  flavor_id:                  "" => "<computed>"
  flavor_name:                "" => "m1.medium"
  force_delete:               "" => "false"
  image_id:                   "" => "<computed>"
  image_name:                 "" => "ubuntu-16.04-x86_64"
  key_pair:                   "" => "rtb-tm-sjc-1b"
  name:                       "" => "tf-bug-example-03"
  network.#:                  "" => "1"
  network.0.access_network:   "" => "false"
  network.0.fixed_ip_v4:      "" => "<computed>"
  network.0.fixed_ip_v6:      "" => "<computed>"
  network.0.floating_ip:      "" => "<computed>"
  network.0.mac:              "" => "<computed>"
  network.0.name:             "" => "rtb-own-network"
  network.0.port:             "" => "<computed>"
  network.0.uuid:             "" => "<computed>"
  region:                     "" => "RegionOne"
  security_groups.#:          "" => "1"
  security_groups.3814588639: "" => "default"
  stop_before_destroy:        "" => "false"
openstack_compute_instance_v2.cmgeneric.2: Still creating... (10s elapsed)
openstack_compute_instance_v2.cmgeneric.2: Creation complete (ID: b5a888af-...4d4f9894)
openstack_compute_floatingip_associate_v2.fip.2: Creating...
  floating_ip: "" => "REDACTED"
  instance_id: "" => "b5a888af-d2d8-4eac-8063-d4c14d4f9894"
  region:      "" => "RegionOne"
openstack_compute_floatingip_associate_v2.fip.1: Creating...
  floating_ip: "" => "REDACTED"
  instance_id: "" => "a0c00a5f-e75f-47a1-a5ee-93e957ed48c2"
  region:      "" => "RegionOne"
openstack_compute_floatingip_associate_v2.fip.0: Creating...
  floating_ip: "" => "REDACTED"
  instance_id: "" => "1ad39e8d-83c8-4588-8f0c-3df542f4928d"
  region:      "" => "RegionOne"
openstack_compute_floatingip_associate_v2.fip.0: Creation complete (ID: REDACTED...2f4928d/)
openstack_compute_floatingip_associate_v2.fip.2: Creation complete (ID: REDACTED...d4f9894/)
openstack_compute_floatingip_associate_v2.fip.1: Creation complete (ID: REDACTED...7ed48c2/)

Apply complete! Resources: 4 added, 0 changed, 4 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path:
picture mtougeron

Most helpful comment

@mtougeron I think this is fixed by #14135 if you change your uses of element to use the 0.7-style index operator [ ... ] instead. That fix will be in the next Terraform release.

If you're able, it'd be interesting to try your config on a build of current Terraform master and see if you still see the problem after replacing your use of the element function. Conversely, if you can wait a few days then this fix should be out in a stable release and we could test it then!

picture apparentlymart on 10 May 2017
👍5

All 5 comments

This also happens for the null_resource. I added

resource "null_resource" "foo" {
  count = "${var.use_floating_ip > 0 ? var.num-nodes : 0}"
  triggers {
    instance_id = "${element(openstack_compute_instance_v2.cmgeneric.*.id, count.index)}"
  }
  provisioner "local-exec" {
    command = "echo \"hi\""
  }  
}

and the plan and apply shows triggers are all changed.

terraform taint openstack_compute_instance_v2.cmgeneric.2
The resource openstack_compute_instance_v2.cmgeneric.2 in the module root has been marked as tainted!

-/+ null_resource.foo.0
    triggers.%:           "1" => "<computed>" (forces new resource)
    triggers.instance_id: "1ad39e8d-83c8-4588-8f0c-3df542f4928d" => "" (forces new resource)

-/+ null_resource.foo.1
    triggers.%:           "1" => "<computed>" (forces new resource)
    triggers.instance_id: "a0c00a5f-e75f-47a1-a5ee-93e957ed48c2" => "" (forces new resource)

-/+ null_resource.foo.2
    triggers.%:           "1" => "<computed>" (forces new resource)
    triggers.instance_id: "7b7def55-1ea4-44b1-ac2e-16ae0d97fb74" => "" (forces new resource)
mtougeron picture mtougeron on 10 May 2017

@mtougeron I think this is fixed by #14135 if you change your uses of element to use the 0.7-style index operator [ ... ] instead. That fix will be in the next Terraform release.

If you're able, it'd be interesting to try your config on a build of current Terraform master and see if you still see the problem after replacing your use of the element function. Conversely, if you can wait a few days then this fix should be out in a stable release and we could test it then!

apparentlymart picture apparentlymart on 10 May 2017
👍5

@apparentlymart yes! This works as I would expect it to. You rock as usual by fixing my bugs before I'm even able to report them. :)

resource "openstack_compute_floatingip_associate_v2" "fip" {
  count       = "${var.use_floating_ip > 0 ? var.num-nodes : 0}"

  floating_ip = "${openstack_compute_floatingip_v2.fip.*.address[count.index]}"
  instance_id = "${openstack_compute_instance_v2.cmgeneric.*.id[count.index]}"
}
mtougeron picture mtougeron on 10 May 2017
👍1

@mtougeron I'm facing the same issue when using lookup(list,key)

I'm having a variable structure similar to:

foo = [
  {
    image = "bar",
    size  = "123",
  },
  {
    image = "baz",
    size  = "123",
  }
]

When ever I add another map to that variable, the lookup(var.foo[count.index],"image") function inside a resource forces it to update even though nothing changed for that particular index.

Any workaround for this?

f0o picture f0o on 4 Apr 2018

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashibot[bot] picture hashibot[bot] on 3 Apr 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

jrnt30 picture jrnt30  ·  3Comments
darron picture darron  ·  3Comments
cpoole picture cpoole  ·  3Comments
franklinwise picture franklinwise  ·  3Comments
larstobi picture larstobi  ·  3Comments