Terraform: Terraform 0.13 complains that all list elements must have the same type, while 0.12 does not.

[marcosdiez]

The issue I am describing fails on Terraform v0.13.3 and works on Terraform v.0.12.29

I have a terraform module that adds routes to aws transit gateway (it's not provider related, though).

These routes come either from other modules or can be added manually.

This is how it looks like:

        target_subnets = [
          module.some_aws_vpc.route_target,
          module.another_aws_vpc.route_target,
          {
            "name" = "my_custom_route"
            "value" = {
              "cidr_block" = "172.22.64.0/21"
              "tg_attachment_id" = var.some_custom_tg_attachment_id
            }
          }
        ]

route_target, on the other hand, is defined as such:

    output "route_target" {
      value = {
        name = var.name
        value = {
          cidr_block       = var.default_route_to_transit_gateway ? local.routable_cidr_block : "0.0.0.0/0"
          tg_attachment_id = aws_ec2_transit_gateway_vpc_attachment.attachment.id
        }
      }
    }

In other words, it's the same structure. Terraform 12 is fine with it, Terraform 13 gives me the following error message:

The given value is not suitable for child module variable "target_subnets"
defined at ../modules/transit_gateway_routes/transit_gateway_routes.tf:1,1-26:
all list elements must have the same type.

I consider this a regression. If this can't be simply fixed, can we have a "cast" command ?

Thanks you!

[alisdair]

Thanks for reporting this @marcosdiez. I can't reproduce the issue without more information.

Can you provide a small configuration that I can use to reproduce the error message?

[marcosdiez]

Hi.

Sorry for the delay.
I was able to do a minimalist version that reproduces the issue:

it needs 3 files:

• main.tf
• m1/m1.tf
• m2/m2.tf

To test, one must type:
rm -f terraform.tfstate;terraform12 init && terraform12 apply && terraform13 init && terraform13 apply

with terraform 0.12.x it will work as expected.
with terraform 0.13.3, it will fail.

===========================BEGIN OF main.tf ======================

module "m1" {
  source = "./m1"

  name             = "m1_name"
  cidr_block       = "10.0.1.0/24"
  tg_attachment_id = "tg-12345"
}

output "m1" {
    value = module.m1.route_target
}

module "m2" {
    source = "./m2"

    target_subnets = [
        module.m1.route_target,
        {
            name = "manual_entry"
            value = {
                cidr_block       = "10.0.2.0/24"
                tg_attachment_id = "tg-999999"
            }
        }
    ]
}

output "m2" {
    value = module.m2.target_subnets
}

===========================END OF main.tf ======================

===========================BEGIN OF m1/m1.tf ======================

variable "name" {
    type = string
}
variable "cidr_block" {
    type = string
}
variable "tg_attachment_id" {
    type = string
}

output "route_target" {
    value = {
        name = var.name
        value = {
            cidr_block = var.cidr_block
            tg_attachment_id = var.tg_attachment_id
        }
    }
}

===========================END OF m1/m1.tf ======================

===========================BEGIN OF m2/m2.tf ======================

variable "target_subnets" {
  type    = list
  default = []
}

output "target_subnets" {
    value = var.target_subnets
}

===========================END OF m2/m2.tf ======================

[alisdair]

Thank you, this is super helpful! I'm able to reproduce with these steps, and also confirm that the issue is present in Terraform 0.14.0-dev.

[alisdair]

An initial debugging session indicates that Terraform is trying to convert the target_subnets argument to m2 from a tuple to a list, and the tuple element types are dynamic and the expected object({…}) type. The output from module m1 is an unknown value of dynamic type at the time this is happening, which is surprising.

[alisdair]

While investigating this a bit further, I found a workaround that might be useful to you. If you specify the full type of the input variable to m2, Terraform's type checking works correctly:

variable "target_subnets" {
  type    = list(object({
    name = string
    value = object({
      cidr_block = string
      tg_attachment_id = string
    })
  }))
  default = []
}

output "target_subnets" {
    value = var.target_subnets
}

Hopefully this applies to your real configuration, too!

I currently still think this is a bug, and I'm going to keep looking into it for now.

[mdiez-modus]

Hey, this is a nice trick. Although I also think this is a bug, it does solve all my problems with the additive of making my modules have strict typing.

Dumb question, are these "complex types" in the terraform documentation ? I can't remember seeing them.

[alisdair]

Yes, there's documentation on structural types on this page. The example for defining a full object type isn't very prominent, though, so perhaps we could improve that.

[marcosdiez]

Thank you for showing me yet another trick!

[alisdair]

After some debugging, I have more of an idea of what's happening here.

When the type of the module input is just list, this is equivalent to list(any), i.e. a list where the elements are of any single type. Terraform validates that the elements are all unifiable to a single type, and eventually reaches this block of code in the cty library.

As noted in the comments there, "this is a special case where the caller wants us to find a suitable single type that all elements can convert to, if possible." The problem comes when one of the elements is unknown/dynamic, as is the case with the output from module m1. This results in hitting this block of code:

// If the list element type after unification is still the dynamic
// type, the only way this can result in a valid list is if all values
// are of dynamic type
if listEty == cty.DynamicPseudoType {
    for _, tupleEty := range tupleEtys {
        if !tupleEty.Equals(cty.DynamicPseudoType) {
            return nil
        }
    }
}

Because the other element in the list is of a known type, object({…}), this fails and returns nil. The result is the error that is in the original report. As previously noted, adding a concrete list element type avoids this branch altogether, and since we can defer conversion of unknown/dynamic values until later, no error occurs.

The above block of code was added (by me 🥴) in response to a panic caused by using null as a tuple element. It's clearly not working in this situation, but I'm not sure how to suggest moving forward.

I think this is enough digging to warrant the "explained" label on this ticket, although whoever picks it up will still have to figure out a way forward that doesn't break existing behaviour in cty or Terraform itself.

[ocervell]

I'm running into this too, but there is no way out in my situation ...

I have a YAML config that I'm deserializing with Terraform describing my exporters: it is a list of objects that can contain string and list elements.

config:
  ...
  exporters:
  - class: Bigquery
    project_id: ${PROJECT_ID}
    dataset_id: ${BIGQUERY_DATASET_ID}
    table_id: ${BIGQUERY_TABLE_ID}

  - class: Stackdriver
    project_id: ${STACKDRIVER_HOST_PROJECT_ID}
    metrics:
      - error_budget_burn_rate
      - sli_measurement
      - slo_target

  - class: Datadog
    api_key: ${DATADOG_API_KEY}
    app_key: ${DATADOG_APP_KEY}

  - class: Dynatrace
    api_token: ${DYNATRACE_API_TOKEN}
    api_url: ${DYNATRACE_API_URL}

If I specify:

variable "config" {
  description = "SLO Configuration"
  type = object({
    ...
    backend   = any
    exporters = any
}

This results in the error above:

The given value is not suitable for child module variable "exporters" defined
at
../../../../../terraform/modules/terraform-google-slo/modules/slo-pipeline/variables.tf:21,1-21:
all list elements must have the same type.

I've tried specifying exporters = list(any), exporters = list(object({})), exporters = list, exporters = any .... but not of them work.

Question: in the example above, how would you define the exporters variable so that Terraform does not fail ?

Some workarounds that could be added to Terraform to allow such inputs:

• any should not behave as if there is only 1 type in the object pass. E.g: list(any) should accept arbitrary list objects

• ability to pass a list of maps with any type using list(object(any))

• ability to define optional arguments and default values for any sub fields when using complex types

• ability to ignore variable type checking (possible ?)

[alisdair]

Question: in the example above, how would you define the exporters variable so that Terraform does not fail ?

This is not currently possible. All objects must have the same type, and at the moment that means they must have valid values for each of the attributes.

For now, the workaround is to adjust your data such that each element has all of the attribute names present, with suitable empty values (e.g. null or []) where appropriate.

I have better news for the future, though! In 0.14.0, we are releasing an experimental optional object argument feature, which we hope to stabilize by 0.15.0. If you have time to try that out and give feedback, that would be much appreciated.