Terraform: destroy provisioner fails to evaluate orphaned for_each instance

[autero1]

Terraform Version

Terraform v0.12.7

Terraform Configuration Files

resource "null_resource" "default" {
  for_each = {delete-me = "first", delete-me-as-well = "second"}

  provisioner "local-exec" {
    when = create
    command = <<-EOF
      echo "Creating ${each.key}=${each.value}"
    EOF
  }

  provisioner "local-exec" {
    when = destroy
    command = <<-EOF
      echo "Deleting ${each.key}=${each.value}"
    EOF
  }
}

Crash Output

https://gist.github.com/autero1/06743847a4b1deefcbd1b42e506574b8

Expected Behavior

terraform removes single null_resource

Actual Behavior

terraform crashes

Steps to Reproduce

Using the provided configuration:

1. terraform init
2. terraform apply
3. Remove single entry from the for_each map, for example delete-me-as-well = "second"
4. terraform apply

--> crash

Additional Context

If you just apply and destroy without changing the for_each map, terraform destroys the resources correctly. If you add entries, terraform adds those correctly, but on every re-apply, terraform crashes if you've removed entries.

[jbardin]

The crash will now be prevented by #22729, but the root cause still needs some work.
The configuration for the destroy provisioner cannot evaluate the for_each value when it's no longer in the config. The error that was masked by the crash output look like:

Reference to "each" in context without for_each: The "each" object can be used
only in "resource" blocks, and only when the "for_each" argument is set.

[autero1]

I ran the same terraformconfig with 0.12.8 and it still crashed. Any update on this?

[pselle]

@autero1 This has not been released, but will go out in 0.12.9.

[jameseck]

Is there an issue for the root cause of this? Having terraform display an error message is better than crashing but still means my code doesn't work.
In my case, I'm taking a map of values and using those with for_each to create libvirt domains. I have ansible provisioners for create and destroy (create is attached to null_resource since I need to exec it before the instance is created, destroy is embedded in the libvirt domain). Now that I've tried to destroy a domain, terraform errors every time and it's not at all clear how to recover from this.

For context, the ansible provisioners are creating and removing dns and static dhcp entries for the domains on a different host. The create needs to exec before the domain is up or it doesn't get a dhcp lease (or the wrong one). The destroy removes the entries for that domain.

[autero1]

Tested with terraform 0.12.9 and doesn't at least crash anymore. Still getting the Error: Reference to "each" in context without for_each: The "each" object can be used only in "resource" blocks, and only when the "for_each" argument is set, though.

[mharm001]

Still getting the error on terraform 0.12.10:
Error: Reference to "each" in context without for_each: The "each" object can be used only in "resource" blocks, and only when the "for_each" argument is set
Is there an update when a fix will be released?

[pselle]

@jameseck I believe this would be the correct issue to report on, as far as the root cause ... the issue is that when destroy provisioners are run, the relevant for_each value is no longer present in the config (by virtue of the fact that is was removed).

When destroy provisioners run for resources managed with count, we don't run into this issue, because the relevant value (index) is available as the instance key. For resources managed with for_each, we only have the key value of for_each available as/in the instance key, so more work/thinking is necessary to expose each.value (pondering this currently).

Now, it would be a marginally small change to expose each.key to destroy provisioners, but that feels like a half-solution. However, @jameseck in particular, since you provide a use case, I wonder if that would alleviate some pressure for you.

And to the other comments, yes, no crash, and confirming that for_each managed resources do not currently support destroy provisioners at present, for the explanation above.

My earlier comment about the release should have been more specific about the release removing the crash, but the issue is open because the problem remains.

[yorinasub17]

it would be a marginally small change to expose each.key to destroy provisioners

For our use case (I work with @autero1), I think we can make this work. It would certainly be better than not being able to use this workflow at all. We are using local-exec to provision some TLS certificates without it leaking to the TF state. We use a map to encode the parameters for the TLS certificate generation which we use as the each.value.

This fix is not ideal, but I think we can update our routine so that we can store the parameters somewhere under what we use for each.key, and then read that back on destroy.

[pselle]

@yorinasub17 I really appreciate this feedback, and now that you mention this, I can definitely see how people will be able to work around this (~for example, for_each references some variable map, and then you would be able to reference that map with the key in the destroy provisioner~ edit: this doesn't work, of course, because the value is gone there as well. Will improve the error at least, but still needs work).

Really appreciate the input that this would indeed be helpful (and I'll get to work on it).

[pselle]

I've confirmed that this works already -- I can make the error message better, but you can indeed use each.key in a destroy provisioner without issue, but not each.value, for those following this issue who might benefit from that knowledge.

[jbardin]

This appears to be working correctly now.
Since the original issue is fixed, any related or similar problem should be raised in a new issue.

Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.