Terraform: [Proposal] interpolation: transforming map to list of maps

[trung]

I hit number of use cases which require transformation of a map to a list of maps.
For example, i define a common_tags to be used in resources creation

locals {
  common_tags = {
    ThisKey = "this value"
    AnotherKey = "another value"
  }
}

I can use it in aws_security_group with tags = "${local.common_tags}" but i'm unable to reuse this in aws_autoscaling_group due to different tag structure. I end up:

resource "aws_autoscaling_group" "this" {
   tag {
      key = "ThisKey"
      value = "${lookup(local.common_tags, "ThisKey")}"
      propagated_at_launch = true
   }
   tag {
      key = "AnotherKey"
      value = "${lookup(local.common_tags, "AnotherKey")}"
      propagated_at_launch = true
   }
}

I'm thinking of an interpolation function that can transform common_tags so i can reuse it.
Something like: transform(input_map, key_name, value_name, [additional_map]) so the above example can be done:

resource "aws_autoscaling_group" "this" {
   tags = "${transform(local.common_tags, "key", "value", map("propagated_at_launch", "true"))}"
}

[ColinHebert]

FYI this is how I do it:

data "null_data_source" "tags" {
  count = "${length(keys(var.tags))}"

  inputs = {
    key                 = "${select(keys(var.tags), count.index)}"
    value               = "${select(values(var.tags), count.index)}"
    propagate_at_launch = true
  }
}

resource "aws_autoscaling_group" "asg" {
  ...
  tags = "${data.null_data_source.tags.*.outputs}"
}

[trung]

@ColinHebert thanks for the solution. It would work nicely.
There seems having typos:

• element(...) instead of select(...)
• tags = ["..."] instead of tags = "..."

There's a gotcha, it's rather a general issue https://github.com/hashicorp/terraform/issues/16712, that makes the solution less elegant in my scenarios.

[apparentlymart]

Hi @trung!

I saw your PR #16985 first, so sorry for replying with this over there rather than here. I'll repeat my comment here for posterity:

---

In the new version of the configuration language we are planning to add a language feature that can support this use-case without a function. It will look like this:

  # NOT YET IMPLEMENTED: details may change before release

  example = [for k,v in var.any_map: {
    key   = k
    value = v
  }]

This is similar to list comprehensions in languages like Python, as a flexible way to create one collection value from another by mapping over each of the source collection elements.

We're in the process of integrating the configuration language changes that include this feature now, and we plan to have an opt-in experimental version of it soon. There's more detail on this feature in its specification (which is still subject to change as we gather feedback).

---

Seeing your use-case here now I see that the feature I described won't _entirely_ address your problem, since treating the tag block type from aws_autoscaling_group as an attribute is not actually supported, and only works today in some cases due to some implementation coincidences.

7034 is discussing the idea of supporting dynamic generation of nested configuration blocks based on interpolation expressions, where our current sketch is the special dynamic block type that allows Terraform to properly validate the block structure even though the specific values are not known until later:

# NOT YET IMPLEMENTED: details may change before release

resource "aws_autoscaling_group" "example" {
  # ...

  dynamic "tag" {
    foreach = "${local.common_tags}"
    content {
      key   = "${dynamic.foreach.name}"
      value = "${dynamic.foreach.value}"

      propagate_at_launch = true
    }
  }
}

[trung]

@apparentlymart thanks for the insights.
I really like the dynamic approach which solves other common scenarios in #7034.
I also like the simplicity of the current language more so I would rather implement extra logic in the Go code (like aws_autoscaling_group does with tag and tags along with interpolation functions) than extend the underlying semantics.

Just my 2 cents.

[cemo]

@apparentlymart Is there an active work on new version of the configuration language? Since whenever I check the repo, I see no progress for a long time. Honestly this is one of the most important thing since the beginning of this project.

[apparentlymart]

@cemo we are currently planning the next stage of work and prototyping. Our plan is to release an experimental opt-in version in the near future, but we are working to find the best way to integrate that so both approaches can coexist in a minor release. Alongside that we are planning the more substantial changes that will lead to the new language being fully integrated, which involves changes to essentially every subsystem of Terraform and thus requires care.

Actual code development should pick up again soon once we've completed the longer-horizon design work. This is the primary focus of the Terraform Core developers at HashiCorp for the foreseeable future.

[apparentlymart]

Hi all,

As discussed over in #8439, the features I mentioned earlier are now merged in master and will be included in the forthcoming v0.12.0 final release. Therefore I'm going to close this out. Thanks for this feature request, and thanks for your patience while we laid the groundwork to implement it.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.