Terraform-provider-aws: Bug with foreach in origin group member block in resource aws_cloudfront_distribution

[jmgreg31]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.1
+ provider.aws v2.13.0

Affected Resource(s)

• aws_cloudfront_distribution

Terraform Configuration Files

It appears there may be a bug with how Terraform version 12 interprets member nested blocks within the origin group nested block using the new for each syntax. If I have a dynamic member defined with more than 1 member, I get an error message stating at least two members are required.

This is a partial config - the assumption is the origin_id has already been defined in an origin block

main.tf

 variable origin_group_member {
   type = any
 }

variable dynamic_origin_group {
  type = any
}

  dynamic "origin_group"  {
    for_each = [for i in "${var.dynamic_origin_group}": {
      id = i.origin_id
      status_codes = i.status_codes
    }]

      content {
        origin_id   = origin_group.value.id
        failover_criteria {
          status_codes = origin_group.value.status_codes
        }        

        dynamic "member" {
          for_each = [for j in "${var.origin_group_member}": {
            id = j.origin_id
          }]

            content {
              origin_id = member.value.id
            }
        }
     }
  }

terraform.tfvars

dynamic_origin_group = [
    {
    origin_id = "OriginGroup-1-S3-TEST"
    status_codes = [403, 404, 500, 502, 503, 504]
    }
]

origin_group_member = [
    {
    origin_id = "TEST-1"
    },
    {
    origin_id = "TEST-2"
    }
]

Debug Output

Error: Insufficient member blocks

  on  line 0:
  (source code not available)

At least 2 "member" blocks are required.

Confirmation

If I add a blank member, I not only see it in the plan, but I also see that my for each block is picking up both defined members

  dynamic "origin_group"  {
    for_each = [for i in "${var.dynamic_origin_group}": {
      id = i.origin_id
      status_codes = i.status_codes
    }]

      content {
        origin_id   = origin_group.value.id
        failover_criteria {
          status_codes = origin_group.value.status_codes
        }        
        member {
          origin_id = ""
        }
        dynamic "member" {
          for_each = [for j in "${var.origin_group_member}": {
            id = j.origin_id
          }]

            content {
              origin_id = member.value.id
            }
        }
     }
  }

plan output:

+ origin_group {
          + origin_id = "OriginGroup-1-S3-TEST"

          + failover_criteria {
              + status_codes = [
                  + 403,
                  + 404,
                  + 500,
                  + 502,
                  + 503,
                  + 504,
                ]
            }

          + member {}
          + member {
              + origin_id = "TEST-1"
            }
          + member {
              + origin_id = "TEST-2"
            }
        }

Expected Behavior

The nested member block should pick up multiple members defined within the for each

Actual Behavior

Error

Steps to Reproduce

1. terraform plan

[bflad]

Hi @jmgreg31 👋 Thanks for reporting this and sorry for the trouble here.

It turns out this should be related to this upstream Terraform Provider SDK issue (https://github.com/hashicorp/terraform/issues/21436) whose fix was just released yesterday in Terraform 0.12.1. Since this deals with the Provider SDK itself and not the Terraform runtime, it requires a dependency update within this codebase as well (https://github.com/terraform-providers/terraform-provider-aws/pull/8860), which was just merged this morning.

If you're willing/able, you should be able to verify that the current master branch of this repository does work as intended in your situation by following the README instructions to custom build and install the provider. Otherwise, this will be released with version 2.14.0 of the Terraform AWS Provider, likely tomorrow. 👍

[jmgreg31]

@bflad Thanks for the quick response. Unfortunately, I am seeing the same issue in the latest aws provider release

Terraform v0.12.1
+ provider.aws v2.14.0

[owenfarrell]

I'm still seeing this issue on Terraform 0.12.2 with the 2.15.0 AWS provider when trying to dynamically create content within a aws_route53_resolver_endpoint.

Error: Insufficient ip_address blocks

At least 2 "ip_address" blocks are required

[bflad]

Hi @owenfarrell can you please share a self-contained reproduction configuration? Since it is affecting a different resource than the original report, we may encourage a separate GitHub issue to be filed, but for now being able to reproduce the issue is most important so we can further triage this. Thanks.

[jmgreg31]

@bflad This is still an issue, but I see it has been moved to milestone 2.16.

However, another related issue I am running across is, how do you handle proper associations of dynamic blocks within a dynamic block? For example; If I have multiple origin group blocks and multiple member blocks, both of which are defined as being dynamic, how do I ensure the correct member blocks get associated to the correct origin group blocks? The logic above would essentially add every dynamic member to every origin group defined.

For reference, I came across this issue with the lambda_function_association and was able to test due to no minimum restrictions.

I would like define a dynamic lambda_function_association block within the cache behaviors to account for multiple configurations or none at all. Is it possible to associate the lambda_function_association with the specific ordered_cache_behavior? As written, this will add every association to every cache behavior.

dynamic "ordered_cache_behavior" {
    for_each = [for i in "${var.dynamic_ordered_cache_behavior}" : {
      path_pattern           = i.path_pattern
      allowed_methods        = i.allowed_methods
      cached_methods         = i.cached_methods
      target_origin_id       = i.target_origin_id
      compress               = i.compress
      query_string           = i.query_string
      cookies_forward        = i.cookies_forward
      headers                = i.headers
      viewer_protocol_policy = i.viewer_protocol_policy
      min_ttl                = i.min_ttl
      default_ttl            = i.default_ttl
      max_ttl                = i.max_ttl
    }]
    content {
      path_pattern     = ordered_cache_behavior.value.path_pattern
      allowed_methods  = ordered_cache_behavior.value.allowed_methods
      cached_methods   = ordered_cache_behavior.value.cached_methods
      target_origin_id = ordered_cache_behavior.value.target_origin_id
      compress         = ordered_cache_behavior.value.compress

      forwarded_values {
        query_string = ordered_cache_behavior.value.query_string
        cookies {
          forward = ordered_cache_behavior.value.cookies_forward
        }
        headers = ordered_cache_behavior.value.headers
      }

      dynamic "lambda_function_association" {
        for_each = [for j in "${var.dynamic_lambda_function_association_ordered}": {
          event_type = j.event_type
          lambda_arn = j.lambda_arn
          include_body = j.include_body
        }]
        content {
          event_type = lambda_function_association.value.event_type
          lambda_arn = lambda_function_association.value.lambda_arn
          include_body = lambda_function_association.value.include_body
        }
      }

      viewer_protocol_policy = ordered_cache_behavior.value.viewer_protocol_policy
      min_ttl                = ordered_cache_behavior.value.min_ttl
      default_ttl            = ordered_cache_behavior.value.default_ttl
      max_ttl                = ordered_cache_behavior.value.max_ttl
    }
  }

Please let me know if this should be opened as a separate issue. Thanks!

[owenfarrell]

@bflad - Here's a quick and dirty example of how I got the same style error message. In the below scenario, my VPC has 2 subnets. But Terraform doesn't seem to be deferring validation as described in Terraform 21436

data "aws_subnet_ids" "example" {
  vpc_id = var.vpc_id
}

data "aws_subnet" "example" {
  count = length(data.aws_subnet_ids.example.ids)
  id    = data.aws_subnet_ids.example.ids[count.index]
}

resource "aws_security_group" "route53_resolver" {
  vpc_id = var.vpc_id
}

resource "aws_route53_resolver_endpoint" "route53_resolver_endpoint" {
  direction          = "OUTBOUND"
  security_group_ids = [aws_security_group.route53_resolver.id]

  dynamic "ip_address" {
    for_each = aws_subnet.example.*.id
    iterator = element
    content {
      subnet_id = element.value
    }
  }
}

[tkellen]

You can work around this thusly, for now:

  # Remove these static entries when this is resolved:
  # https://github.com/terraform-providers/terraform-provider-aws/issues/8873
  ip_address {
    subnet_id = aws_subnet.example[0].id
  }
  ip_address {
    subnet_id = aws_subnet.example[1].id
  }
  dynamic "ip_address" {
    for_each = slice(aws_subnet.example[*].id, 1, length(aws_subnet.example[*].id))
    content {
      subnet_id = ip_address.value
    }
  }

This takes the assumption there will always be two, since it is required, and dynamically adds any more that happen to be present.

[owenfarrell]

@tkellen Can we agree that that’s a workaround and not the expected long-term solution?

[vojtmen]

@tkellen Can we agree that that’s a workaround and not the expected long-term solution?

That is even not a workaround .. dynamic block was introduced to handle situations where you don't know the exact number of arguments.

I raised the issue in 0.12.0 and escalated it to Terraform Enterprise support. How come that this is still not fixed? I root specifically for aws_route53_resolver_endpoint, but I have seen this error already in multiple resources scattered among multiple services.

[bflad]

Hi folks 👋 I tried this out on recent versions of Terraform CLI and the Terraform AWS Provider, and was either able to have Terraform produce a more valid error or work as expected with configuration fixes.

Example configuration:

terraform {
  required_providers {
    aws = "2.40.0"
  }
  required_version = "0.12.16"
}

provider "aws" {
  region = "us-east-2"
}

data "aws_vpc" "test" {
  default = true
}

data "aws_subnet_ids" "test" {
  vpc_id = data.aws_vpc.test.id
}

data "aws_subnet" "test" {
  count = length(data.aws_subnet_ids.test.ids)
  id = data.aws_subnet_ids.test.ids[count.index]
}

resource "aws_security_group" "test" {
  vpc_id = data.aws_vpc.test.id
}

resource "aws_route53_resolver_endpoint" "test" {
  direction          = "OUTBOUND"
  security_group_ids = [aws_security_group.test.id]

  dynamic "ip_address" {
    for_each = data.aws_subnet.test[*].id

    content {
      subnet_id = ip_address.value
    }
  }
}

Plan:

terraform apply
data.aws_vpc.test: Refreshing state...
data.aws_subnet_ids.test: Refreshing state...

Error: Invalid index

  on main.tf line 22, in data "aws_subnet" "test":
  22:   id = data.aws_subnet_ids.test.ids[count.index]
    |----------------
    | count.index is 0
    | data.aws_subnet_ids.test.ids is set of string with 3 elements

This value does not have any indices.


Error: Invalid index

  on main.tf line 22, in data "aws_subnet" "test":
  22:   id = data.aws_subnet_ids.test.ids[count.index]
    |----------------
    | count.index is 1
    | data.aws_subnet_ids.test.ids is set of string with 3 elements

This value does not have any indices.


Error: Invalid index

  on main.tf line 22, in data "aws_subnet" "test":
  22:   id = data.aws_subnet_ids.test.ids[count.index]
    |----------------
    | count.index is 2
    | data.aws_subnet_ids.test.ids is set of string with 3 elements

This value does not have any indices.

These errors are correct in this situation since in Terraform 0.12 sets cannot be indexed (as compared to lists, which can be indexed). Changing the for_each line to use the aws_subnet_ids data source directly:

terraform {
  required_providers {
    aws = "2.40.0"
  }
  required_version = "0.12.16"
}

provider "aws" {
  region = "us-east-2"
}

data "aws_vpc" "test" {
  default = true
}

data "aws_subnet_ids" "test" {
  vpc_id = data.aws_vpc.test.id
}

resource "aws_security_group" "test" {
  vpc_id = data.aws_vpc.test.id
}

resource "aws_route53_resolver_endpoint" "test" {
  direction          = "OUTBOUND"
  security_group_ids = [aws_security_group.test.id]

  dynamic "ip_address" {
    for_each = data.aws_subnet_ids.test.ids

    content {
      subnet_id = ip_address.value
    }
  }
}

Plans as expected:

...
  # aws_route53_resolver_endpoint.test will be created
  + resource "aws_route53_resolver_endpoint" "test" {
      + arn                = (known after apply)
      + direction          = "OUTBOUND"
      + host_vpc_id        = (known after apply)
      + id                 = (known after apply)
      + security_group_ids = (known after apply)

      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + subnet_id = "subnet-2c271266"
        }
      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + subnet_id = "subnet-2d8d9855"
        }
      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + subnet_id = "subnet-7635c81f"
        }
    }
...

Its likely some combination of Terraform CLI and/or Terraform AWS Provider upgrade fixed this along the way. If you are still having trouble on recent versions, please open a new bug report with a fully replicating configuration and we'll take a further look. Thanks.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!