Terraform: Feature request: Map & List Comprehensions

[NotBobTheBuilder]

Maps and Lists are great way to store data in terraform configurations, but can get tricky to manage, particularly when they need to interface with JSON objects that providers like AWS can require.

As an example, a map is the perfect structure for storing environment variables:

module "whatever" {
  environment = {
    DB_URL = "${module.db.conn_string}"
    BAR = "foo"
  }
}

However, the AWS provider requires them in a list format like so:

[
  {
    "name": "DB_URL",
    "value": "${module.db.conn_string}"
  },
  {
    "name": "BAR",
    "value": "foo"
  }
]

A comprehension structure, found in Python, Haskell and others, works like this:

[i * 2 for i in range(3)] # -> [0, 2, 4]
{str(i): i*2 for in range(4)} # -> {"0": 0, "1": 2, "2": 4}

They can also be used with iteration over other objects:

{val: key for (key, val) in {"a": "b"}} # -> {"b": "a"}

Using this pattern, the problem presented above could be tackled thusly:

[{"name": name, "value": value} for (name, value) in environment]

While this does require a syntax change, and is more complicated to implement than simply a function which explodes a map into a list of objects with name & value, it is more general purpose and would likely help in other cases of converting to JSON or passing lists or maps into modules.

[crossing]

It's much easier if the DSL is in a real programming language than keep re-inventing the wheel.

[dnk8n]

I agree with this feature request. For example, in aws, if I wanted to add an equivalent number of ip addresses to route53 as I have instances... how could I currently do it?

Currently I have to hardcode like this:

resource "aws_route53_record" "example" {
  zone_id = "${data.aws_route53_zone.internal_dns.zone_id}"
  name = "example"
  type = "A"
  ttl = "300"
  records = [
    "${aws_instance.example.0.private_ip}",
    "${aws_instance.example.1.private_ip}",
    "${aws_instance.example.2.private_ip}"
  ]
}

[apparentlymart]

Hi all! Sorry we missed this one when following up on features of the improved config language parser/interpreter we're currently working on.

Once the improved implementations are implemented, we will have for expressions, which are list and map comprehensions.

The example presented in the original comment on this issue could then be expressed like this:

  environment = [for name, value in var.environment: {
    name  = name
    value = value
  }]

The capabilities of for expressions are equivalent to Python's list comprehensions, but with an inverted syntax to allow for future implementation of autocomplete-type features in text editors and IDEs, so that (in more complex cases) the name and value variables are defined before they are used.

Integrating this new language implementation is the current focus for the Terraform Core team at HashiCorp. This change covers a lot of different issues here on GitHub, so I can't promise I'll always remember to update all of them as we make progress but keep an eye out for a "preview" version of this coming soon, to gather any feedback before we include it in a "real" release.

[cemo]

@apparentlymart thanks for information. Is there an ETA?

[dnk8n]

@apparentlymart Is this funtionality available anywhere yet?

[apparentlymart]

Hi all!

I just tried the following configuration in Terraform v0.12.0-alpha1:

variable "example" {
  type = map(string)
  default = {
    DB_URL = "jdbc:etc/etc"
    BAR    = "foo"
  }
}

output "example" {
  value = [
    for k, v in var.example : {
      name  = k
      value = v
    }
  ]
}

After applying this, I got the list value expected:

$ terraform output
example = [
  {
    "name" = "BAR"
    "value" = "foo"
  },
  {
    "name" = "DB_URL"
    "value" = "jdbc:etc/etc"
  },
]

The syntax is inverted relative to e.g. Python (for keyword first, then outputs) because the Terraform language has strong type information for most values and so in future text editor integrations may be able to support autocomplete features within this construct relying on the fact that the iterator variables (k and v in the above example) are declared _before_ they are used in source code order.

The v0.12.0-alpha1 release can be used to experiment further with this, and the feature will also be included in the eventual v0.12.0 final release. Thanks for sharing this feature request!

[NotBobTheBuilder]

Wow, this is great! Thank you for implementing this and sharing the update!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.