Terraform: Terraform destroy destroys more resources than expected

[michalmedvecky]

Terraform Version

0.9.11

I cannot test with 0.10.0 because there is a bug preventing me from creating openstack instances.

Actually I don't believe this is provider-specific, but a problem in dependency calculation.

Problem description

Trying to destroy only one port out ouf count destroys all (unrelated) instances using the same array (is this the correct term?).

Terraform Configuration Files

resource "openstack_networking_network_v2" "net-backend" {
  name = "${var.backend_network}"
  admin_state_up = "true"
}

resource "openstack_networking_subnet_v2" "net-backend-subnet" {
  name = "${var.backend_network}"
  network_id = "${openstack_networking_network_v2.net-backend.id}"
  cidr = "${var.backend_network_cidr}"
  dns_nameservers = [ "100.127.70.70" ]
  ip_version = 4
  depends_on = ["openstack_networking_network_v2.net-backend"]
}

resource "openstack_networking_port_v2" "test_port_backend" {
  name = "test${count.index+1}"
  count = "3"
  admin_state_up = "true"
  network_id = "${openstack_networking_network_v2.net-backend.id}"
}

resource "openstack_compute_instance_v2" "test" {
  name = "test${count.index+1}"
  count = "3"
  image_id = "66ae409c-f2ea-4a8c-a25c-3859d79f9f33"
  flavor_name = "m1.small"
  key_pair = "misko"
  availability_zone = "mz"

  network {
    port = "${openstack_networking_port_v2.test_port_backend.*.id[count.index]}"  }

}

variable "backend_network" {
  description = "local network name"
}

variable "backend_network_cidr" {
  description = "local network cidr"
}

Expected Behavior

destroy only one port, and the instance that is referring to it

Actual Behavior

All my test instances are destroyed:

$ terraform-0.9.11 destroy -var-file=sandbox.tfvars -target openstack_networking_port_v2.test_port_backend[2]
Do you really want to destroy?
  Terraform will delete the following infrastructure:
    openstack_networking_port_v2.test_port_backend[2]
  There is no undo. Only 'yes' will be accepted to confirm

  Enter a value: yes

openstack_networking_network_v2.net-backend: Refreshing state... (ID: d3ebd619-3f2d-45cb-8e28-86da7cdf4cc5)
openstack_networking_port_v2.test_port_backend.2: Refreshing state... (ID: 267f8d1c-1d71-4c12-856a-e19f031bdc15)
openstack_compute_instance_v2.test.1: Destroying... (ID: 6e967dd3-647d-45ad-8724-c1530bafd827)
openstack_compute_instance_v2.test.2: Destroying... (ID: 8c82978f-fb0b-4d3f-bcfb-8e5020f71622)
openstack_compute_instance_v2.test.0: Destroying... (ID: e2e09783-e31a-4dc0-b67d-d4d9a6e60dd0)

Steps to Reproduce

1. terraform apply (create all the resources)
2. terraform destroy -var-file=sandbox.tfvars -target openstack_networking_port_v2.test_port_backend[2]
3. yes

[apparentlymart]

Hi @michalmedvecky! Sorry this doesn't work as expected.

This is a consequence of how Terraform currently understands openstack_networking_port_v2.test_port_backend.*.id, and is a known limitation. The problem is that this splat reference creates a dependency on _all_ of the port instances, rather than just the one indicated by count.index.

This is one of the issues on our list for fixing in a forthcoming round of configuration language improvements. Although the design for that is still in progress, and so this might change before actually being released, the sketch right now is to introduce the following alternative syntax which will then allow Terraform to better understand the intent here:

# DRAFT: not yet implemented, may change before release
port = "${openstack_networking_port_v2.test_port_backend[count.index].id}"

Note that in general we do not recommend using -target except in exceptional circumstances. In this situation I'd recommend instead using terraform taint to mark specific resources as needing replacement:

terraform taint openstack_networking_port_v2.test_port_backend.2
terraform taint openstack_compute_instance_v2.test.2

On the next plan Terraform will then include a destroy action for these two resources alone, leaving other resources unaffected. (Taint does not consider dependencies.)

(Once we merge #12289 -- which likely won't be until the next major release, due to it being a breaking change -- this syntax will change to openstack_networking_port_v2.test_port_backend[2] for consistency with -target. Just noting this here in case someone finds this comment in future, after that's been released, and wonders why the above commands don't work anymore.)

[jzampieron]

Is there a workaround for this limitation?

[AdrienneCohea]

Yep, what @apparentlymart said about "this splat reference creates a dependency on all of the port instances" is still true in 0.11.7. Consider the following configuration:

resource "digitalocean_droplet" "control" {
  count              = "${var.counts["control"]}"
  image              = "ubuntu-18-04-x64"
  size               = "${var.control_size}"
  region             = "sfo1"
  name               = "control${count.index + 1}"
  ssh_keys           = ["${digitalocean_ssh_key.ssh.id}"]
  private_networking = true
  tags               = ["hashicorp", "control"]
}

resource "digitalocean_record" "control_fqdns" {
  count  = "${var.counts["control"]}"
  domain = "${var.domain}"
  type   = "A"
  ttl    = 60
  name   = "${element(digitalocean_droplet.control.*.name, count.index)}"
  value  = "${element(digitalocean_droplet.control.*.ipv4_address, count.index)}"
}

If I create this and then issue a terraform taint digitalocean_droplet.control.1 that will result in the following plan (destroys and recreates all DNS records rather than the one for digitalocean_droplet.control.1).

-/+ digitalocean_droplet.control[1] (tainted) (new resource required)
      id:                   "103245727" => <computed> (forces new resource)
      disk:                 "60" => <computed>
      image:                "ubuntu-18-04-x64" => "ubuntu-18-04-x64"
      ipv4_address:         "107.170.249.159" => <computed>
      ipv4_address_private: "10.134.26.84" => <computed>
      ipv6_address:         "" => <computed>
      ipv6_address_private: "" => <computed>
      locked:               "false" => <computed>
      name:                 "control2" => "control2"
      price_hourly:         "0.02232" => <computed>
      price_monthly:        "15" => <computed>
      private_networking:   "true" => "true"
      region:               "sfo1" => "sfo1"
      resize_disk:          "true" => "true"
      size:                 "s-2vcpu-2gb" => "s-2vcpu-2gb"
      ssh_keys.#:           "1" => "1"
      ssh_keys.0:           "22084294" => "22084294"
      status:               "active" => <computed>
      tags.#:               "2" => "2"
      tags.0:               "hashicorp" => "hashicorp"
      tags.1:               "control" => "control"
      vcpus:                "2" => <computed>

-/+ digitalocean_record.control_fqdns[0] (new resource required)
      id:                   "50930957" => <computed> (forces new resource)
      domain:               "adriennecohea.ninja" => "adriennecohea.ninja"
      fqdn:                 "control1.adriennecohea.ninja" => <computed>
      name:                 "control1" => "control1"
      port:                 "0" => <computed>
      priority:             "0" => <computed>
      ttl:                  "60" => "60"
      type:                 "A" => "A"
      value:                "107.170.249.168" => "${element(digitalocean_droplet.control.*.ipv4_address, count.index)}" (forces new resource)
      weight:               "0" => <computed>

-/+ digitalocean_record.control_fqdns[1] (new resource required)
      id:                   "50930956" => <computed> (forces new resource)
      domain:               "adriennecohea.ninja" => "adriennecohea.ninja"
      fqdn:                 "control2.adriennecohea.ninja" => <computed>
      name:                 "control2" => "control2"
      port:                 "0" => <computed>
      priority:             "0" => <computed>
      ttl:                  "60" => "60"
      type:                 "A" => "A"
      value:                "107.170.249.159" => "${element(digitalocean_droplet.control.*.ipv4_address, count.index)}" (forces new resource)
      weight:               "0" => <computed>

-/+ digitalocean_record.control_fqdns[2] (new resource required)
      id:                   "50930958" => <computed> (forces new resource)
      domain:               "adriennecohea.ninja" => "adriennecohea.ninja"
      fqdn:                 "control3.adriennecohea.ninja" => <computed>
      name:                 "control3" => "control3"
      port:                 "0" => <computed>
      priority:             "0" => <computed>
      ttl:                  "60" => "60"
      type:                 "A" => "A"
      value:                "107.170.247.53" => "${element(digitalocean_droplet.control.*.ipv4_address, count.index)}" (forces new resource)
      weight:               "0" => <computed>


Plan: 4 to add, 0 to change, 4 to destroy.

It appears at least for now that tainting a resource that is in a collection will cause the taint to cascade to all elements of the collection rather than the single instance of the resource.

In this instance I am not terribly concerned because it's just DNS records, but if I had another role of compute resources depending on this, I wouldn't want them getting destroyed and recreated simultaneously.

[vaneyckt]

@apparentlymart do you know if this bug is supposed to be fixed in Terraform 0.12 or is still an ongoing issue?

[rayuduc]

Hello @AdrienneCohea,
Am also facing same issue, If I destroy any VM with Index value its deleting all DNS records. And same case with disk If I destroy one disk destroying all VM's,
splat reference creates a dependency on all instances, hence deleting all instances
rrdatas = ["${google_compute_instance.node.*.network_interface.0.network_ip[count.index]}"

terraform taint is recreating but we want to completely destroy resource
google provider version- google-beta and google latest
terraform version: 0.12.10

implicit dependency on google_compute_instance.
terraform destroy -target=“module.testing-vm.google_compute_instance.node[1]”
terraform destroy -target="module.qarc-gcp-bastion.google_compute_disk.node[0]"
google provider version- google-beta and google latest
terraform version: 0.12.10

module/google_vm/google-compute.tf
count = (var.add_external_ip == true ? var.amount : 0)
resource “google_compute_instance” “node” {
count = var.amount

name = format("%s%02d", var.name, count.index + 1)
machine_type = var.machine_type
project = var.test_project
allow_stopping_for_update = true
zone = element(var.zones, count.index%length(var.zones))
tags = var.tags

DNS records explicit definition
depends_on = [google_dns_record_set.dns_record]
boot_disk {
initialize_params {
image = var.boot_disk_image
size = var.boot_disk_size
}
}
network_interface {
subnetwork = var.subnetwork
}

metadata = {
env = var.res_env
}
}

resource “google_dns_record_set” “dns_record” {
count = var.amount
name = “{format("%s%02d", var.name, count.index + 1)}.{var.dns_zone}”
managed_zone = “{var.dns_name}" type = "{var.dns_record_type}”
ttl = “${var.dns_record_ttl}”

compute implicit definition
rrdatas = ["${google_compute_instance.node.*.network_interface.0.network_ip[count.index]}"]
}

================= above module usage
module “dev-test” {
source = “./module/google_vm”
name = "dev-test
amount = 2
machine_type = var.app_machine
project = var.test-project
subnetwork = google_compute_subnetwork.test1_sub_network.self_link
zones = var.vm_zone
dns_record = true
dns_name = var.dns_name
dns_zone = var.dns_zone
dns_record_ttl = var.dns_record_ttl
dns_record_type = var.dns_record_type
tags = [
“env”,
]
label = “dev-test”
}