Terraform-provider-aws: Rename WAF rule fails with `WAFReferencedItemException`

[hashibot[bot]]

_This issue was originally opened by @gasi as hashicorp/terraform#21069. It was migrated here as a result of the provider split. The original body of the issue is below._

---

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

> terraform --version
Terraform v0.11.13
+ provider.aws v2.6.0
+ provider.template v2.1.1

Affected Resource(s)

• aws_wafregional_byte_match_set

Expected Behavior

• Terraform successfully renames WAF rule.

Actual Behavior

• Terraform could not rename WAF rule as destroying it before creation fails due this error:

Error: Error applying plan:

1 error(s) occurred:

* module.vines_waf.aws_wafregional_byte_match_set.foo (destroy): 1 error(s) occurred:

* aws_wafregional_byte_match_set.foo: Error deleting ByteMatchSet: WAFReferencedItemException: This entity is still referenced by other entities.
    status code: 400, request id: <redacted>

Steps to Reproduce

1. Create WAF resources:

resource "aws_wafregional_rule" "foo_parent" {
  name        = "${var.environment}foo_parent"
  metric_name = "${var.environment}foo_parent"

  # predicate whose `name` we change
  predicate {
    data_id = "${aws_wafregional_byte_match_set.foo.id}"
    negated = false
    type = "ByteMatch"
  }
}

resource "aws_wafregional_byte_match_set" "foo" {
  name = "${var.environment}-foo"

  byte_match_tuples {
    text_transformation = "LOWERCASE"
    target_string = "bar"
    positional_constraint = "CONTAINS"
    field_to_match {
      type = "URI"
    }
  }
}

1. Run terraform apply

2. Change name of foo to bar:

  name = "${var.environment}-bar"

1. Run terraform apply again

Important Factoids

References

• hashicorp/terraform#0000

[mxlan]

I've encountered this same issue,

It appears that the resource or provider is not respecting the WAF relationships and doesn't attempt to perform the necessary actions in the correct order.

It outputs the plan to change the rule / or acl, and destroy and recreate the condition / rule.

Normally if you do that you need to create the new rule / condition first, then modify the rule / acl, and finally delete the old condition / rule (it seems to have reversed these last two steps). It doesn't seem like that terraform sticks to that order of operations.

Are there any workarounds? using a "depends_on" block doesn't appear to work.
Ah, okay I'll need to use create_before_destroy. Though this will help if a resource is renamed but will not help resolve the case in which you remove a resource and create a new resource reference, the new resource will be created but deletion of the old resource will be attempted before the rule/acl modification happens. I've had to manually modify the rule/acl to remove the reference before running the apply.

Provider versions: 1.6 & 2.7, haven't tried all the others in between but I'm assuming they all experience the same issue.

[jsmigovsky]

To destroy a rule I had to manually go into the console and remove the association between the ACL and the Rule. Not an ideal course of action, but running terraform apply is no longer chocking on this association.

[rdelcampog]

I'm experimenting the same issue:

  # aws_wafregional_ipset.blacklist will be destroyed
  - resource "aws_wafregional_ipset" "blacklist" {
      - arn  = "xxxxxxxxxx" -> null
      - id   = "e15c39cd-fc2f-4510-aa02-a4a3fdf9e1a0" -> null
      - name = "xxxxxxxxxx" -> null

      - ip_set_descriptor {
          - type  = "IPV4" -> null
          - value = "xxxxxxxxxx" -> null
        }
    }

  # aws_wafregional_rule.blacklist will be destroyed
  - resource "aws_wafregional_rule" "blacklist" {
      - arn         = "xxxxxxxxxx" -> null
      - id          = "xxxxxxxxxx" -> null
      - metric_name = "genericdetectblacklistedips" -> null
      - name        = "xxxxxxxxxx" -> null
      - tags        = {} -> null

      - predicate {
          - data_id = "e15c39cd-fc2f-4510-aa02-a4a3fdf9e1a0" -> null
          - negated = false -> null
          - type    = "IPMatch" -> null
        }
    }

  # aws_wafregional_web_acl.wafregional_acl will be updated in-place
  ~ resource "aws_wafregional_web_acl" "wafregional_acl" {
     ...
      - rule {
          - priority = 1 -> null
          - rule_id  = "9e9f157b-ec2c-4be7-a1fe-1d877c8505a4" -> null
          - type     = "REGULAR" -> null

          - action {
              - type = "DENY" -> null
            }
        }
    }

After running terraform apply:

Error: Error deleting ByteMatchSet: WAFReferencedItemException: This entity is still referenced by other entities.
    status code: 400, request id: c4fe0a4c-efc1-48a1-a9ea-c39889106e19

Terraform is not removing the WAF ACL rule after trying to remove the rule.

[voycey]

Any updates on this - this is unusable right now

[blang]

For me this involved the association between aws_wafregional_rule and aws_wafregional_web_acl, where the rule is destroyed before the web_acl is changed.
I solved this temporarily by forcing the web_acl to be recreated when rules change.
E.g.:

resource "random_pet" "acl_name" {
  separator = "-"
  length    = 2

  keepers = { # Create string from your rules }
}

resource "aws_wafregional_web_acl" "default" {
  name        = format("%s-%s", var.name, random_pet.acl_name.id)
  ...
}

[headlessme]

This is causing us some pain – did anyone find a workaround that doesn't involve any manual steps?