Terraform: core: Allow the use heterogeneous maps in module variables and outputs

[joestump]

Terraform Version

0.9.4

Affected Resource(s)

Modules

Terraform Configuration Files

# foo.tf
module "test" {
  source = "./foo"
  my_var = {
    name = "hello"
    servers = ["foo", "bar"]
  }

  # This works, but seems verbose and doesn't read as nice IMO
  my_var_name = "foo"
  my_var_servers = ["foo", "bar"]
}

# foo/variables.tf
variable "my_var" {
  type = "map"
}

# foo/main.tf
data "template_file" "init" {
  template = "$${name}"

  vars {
    name = "${var.my_var["name"]}"
  }
}

Debug Output

$ terraform plan
1 error(s) occurred:

* module.test.data.template_file.init: 1 error(s) occurred:

* module.test.data.template_file.init: At column 13, line 1: map "var.my_var" does not have homogenous value types. found TypeString,  in:

${var.my_var["name"]}

Expected Behavior

I can pass a heterogeneous map to a module. This is especially acute when you'd like to pass in a subset of a different resource's information to a module. For instance, I can't pass zone_id and name_servers under a dns map to a module.

The alternative is to basically prefix module variable names (e.g. dns_zone_id and dns_name_servers, which seems somewhat verbose and less readable than using maps.

Actual Behavior

I cannot.

[lig]

@apparentlymart any news on this one?

[apparentlymart]

Sorry for the long silence here! Apparently I missed this one when posting updates. (I've re-labelled it to make it easier to spot this in future.)

We're currently working on integrating a new configuration language and interpreter to Terraform for inclusion in a future major release. One of the big changes in that interpreter is the ability to represent mixed-typed collections in the type system, which will enable the sort of usage described in this issue. This work is the current focus of the Terraform team at HashiCorp.

[joestump]

We're currently working on integrating a new configuration language and interpreter to Terraform for inclusion in a future major release.

Can you elaborate on this? Or point us to something that might? I'm probably reading too much into this but "new configuration language" sounds like a painful migration path might lay ahead of my team and I.

[apparentlymart]

Oh whoops.. sorry I missed a word in that sentence. I meant to say "a new configuration language _parser_ and interpreter". In other words, the language is the same but we have a more robust implementation of it.

To be fully transparent, there will be some things that will need to be updated once this new parser is implemented because the old parser had ambiguities and edge-cases that would've blocked these new capabilities and fixes from being implemented. However, we do intend to remain compatible with most existing configurations, and to not intend to create a painful migration path.

[shellshock1953]

Hello,
@apparentlymart can you say, how long it will take to release new parser?

[facundovictor]

This issue and #2114 are limiting ways to express group of values within a kind of data structure format. It would be a really awesome enhancement!

[vfab]

This feature would really have come in handy today, and greatly simplified some of my more complex modules. Please keep working on this! :)

[bfleming-ciena]

I also just found this issue after trying to pass values to a module from a map that contained lists and strings. Is this an ok work-around till that is supported? Kind of ugly, but I want to keep my config separate and use workspaces.

locals {

  env = {
    defaults = {
      location = "westus"
    }

    defaults_lists= {
      subnet_names = ["subnet1", "subnet2"]
    }
###################################################################
    dev = {
      location = "eastus"
    }

    dev_lists= {
      subnet_names = ["subnet1", "subnet2"]
    }

###################################################################
    prod = {
    }

    prod_lists= {
      subnet_names = ["subnet1", "subnet2"]
    }
  }

  workspace = "${merge(local.env["defaults"], local.env[terraform.workspace])}"
  workspace_lists= "${merge(local.env["defaults_lists"], local.env[format("%v_lists",terraform.workspace)])}"
}

[outworlder]

Just found out to my surprise that this is a limitation. It is a dangerous one, because:

• You can have heterogeneous maps as module outputs. This "works" just fine, if you then do a terraform output -json you can use them as inputs for other automation tools. That's great.

To show that this actually works, excerpt from my current terraform output -json

    "network": {
        "sensitive": false,
        "type": "map",
        "value": {
            "CIDR": "10.0.0.0/16",
            "subnets": {
                "k8s": "10.0.100.0/24",
                "public": "10.0.120.0/24",
                "storage": "10.0.110.0/24"
            }
        }
    },

• Since this appears to be working, you check it in, other team mates start using it in their tools, everything is peachy.

• Fast forward a few months, someone has the bright idea of using the Terraform remote state to retrieve data for other modules to use. You then get the dreaded

map "data.terraform_remote_state.base.network" does not have homogenous types. found TypeString and then TypeMap

And now you cannot do anything about it as you have a bunch of remote state files already created and being consumed.

Terrraform as an abstraction over cloud providers is great. But the configuration language is awful. Can't wait for this new parser, hopefully some restrictions will be lifted.

[stszap]

It seems that this issue may be fixed in 0.12 https://www.hashicorp.com/blog/terraform-0-12-rich-value-types

[f0o]

Any estimate when 0.12 is available? (Or any RC of it)

[apparentlymart]

Hi all! Sorry for the long silence here.

I just copied the examples from the opening comment in this issue to a local working directory and tried this with Terraform v0.12.0-alpha1.

I adjusted the variable "my_var" declaration to use the new object type syntax instead of a map, which is the heterogeneous equivalent of a map in the new v0.12 type system:

variable "my_var" {
  type = object({
    name    = string
    servers = list(string)
  })
}

Map and object are separate concepts in v0.12 so that we can retain the map behaviors from prior versions where they are significant. Variable types allow us to opt-in to the heterogeneous behavior while also defining exactly what is expected so that Terraform can produce good error messages in the case of a mismatch.

(Other than this, the only other minor change I made was to correct the vars argument in template_file to use the argument syntax vars = { ... } rather than the nested block syntax vars { ... }, since the new parser distinguishes these to allow map expressions to have arbitrary expressions as keys.)

The slightly-updated configuration example works in v0.12-alpha1:

$ terraform apply
module.test.data.template_file.init: Refreshing state...

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

The rendered result of the template is, as expected, the string "hello".

With that said, it seems like the use-case in this issue is now supported in the master branch and will be included in the eventual v0.12.0 final release, and so I'm going to close this out. Thanks for opening this issue, and thanks for the patience while we did the groundwork to make this possible.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.