Terraform: Flattening a list in 0.7.3

[jszwedko]

Prior to Terraform 0.7.0, lists of lists appeared to automatically be flattened when used as an input to another module (see configuration below for what this looked like). In Terraform > 0.7.0 the apply complains that the provided value "must be a single value, not a list". Is there a way to flatten a list of lists in Terraform 0.7?

Terraform Version

Terraform v0.7.3

Affected Resource(s)

N/A - interpolation

Terraform Configuration Files

provider "aws" {
  region = "us-east-1"
}

resource "aws_network_interface" "foo" {
  count = "3"
  subnet_id = "SUBNET ID"
}

resource "aws_route53_record" "aws_dns" {
  name = "test.com"

  zone_id = "ZONE ID"
  type = "CNAME"
  ttl = 30
  records = ["${aws_network_interface.foo.*.private_ips}"]
}

Debug Output

https://gist.github.com/d810e513f6cac425acedf6f7fa34e2ca

Expected Behavior

The aws_route53_record is created using all of the private_ips of the network interfaces.

Actual Behavior

Error running plan: 3 error(s) occurred:
2016/09/06 23:57:50 [DEBUG] plugin: waiting for all plugin processes to complete...

* aws_route53_record.aws_dns: records.0 must be a single value, not a list
* aws_route53_record.aws_dns: records.1 must be a single value, not a list
* aws_route53_record.aws_dns: records.2 must be a single value, not a list

Steps to Reproduce

1. terraform plan
2. terraform apply
   

   Important Factoids

This configuration worked as-is in Terraform < 0.7.0

References

Related:

https://github.com/hashicorp/terraform/issues/8695
https://github.com/hashicorp/terraform/issues/7906

[jszwedko]

It appears that formatlist automatically does flattening so this worked for our use case. I'll leave this open because I'm not sure if there are other use cases that would still benefit from being able to flatten a list of lists.

i.e.

["${formatlist("%s", aws_network_interface.syslog.*.private_ips)}"]

[jszwedko]

Scratch that, I accidentally tried with 0.6.16 rather than 0.7.3 as I was previously testing with so the issue remains as stated.

[brentley]

This is hitting me also.

[2rs2ts]

I'm not going to pretend that this is an elegant workaround, but it suffices at least for this example:

  records = ["${split(",",
    replace(
      replace(
        replace(
          format("%s", aws_network_interface.foo.*.private_ips), "/[^\\s\\d\\.]/", ""
          ), "/(\\d)\\s+/", "$1,"
        ), "/\\s+/", ""
      )
    )}"]

Basically, stringify the data structure, take out everything but the IP addresses and add commas to delimit them, then split by commas to get back to a list.

[2rs2ts]

Actually, it seems that my suggestion requires the network interfaces to already exist from a previous plan/apply... might not work for everyone.

[2rs2ts]

Looks like you can use a template file to work around this:

data "template_file" "ips" {
  template = "${format("%s", aws_network_interface.foo.*.private_ips)}"
}

resource "aws_route53_record" "aws_dns" {
  ...

  records = ["${split(",",
    replace(
      replace(
        replace(
          data.template_file.ips.rendered, "/[^\\s\\d\\.]/", ""
          ), "/(\\d)\\s+/", "$1,"
        ), "/\\s+/", ""
      )
    )}"]
}

This tricks terraform into seeing the value as being computed.

[purejamie]

Hello there - also having this problem with inputs into ELB subnet variables - I want to pass a list generated by another module in, but only get "should be string, got type list". IS there a way to flatten lists or allow their input into the ELB resource?

Thanks,
Jamie.

[2rs2ts]

My workaround broke in terraform 0.8. The IP addresses no longer appear in the string representation of the list. In 0.7, it looks like [{Variable (TypeList): [{Variable (TypeString): 10.41.17.25}]} {Variable (TypeList): [{Variable (TypeString): 10.41.22.236}]}], but in 0.8 it looks like [{Variable (TypeList): [{Variable (TypeString): }]} {Variable (TypeList): [{Variable (TypeString): }]}]. Of course, the underlying issue is also not solved, so unless I can find another workaround, I think it's safe to say that we're not going to be able to upgrade to 0.8 until this issue is resolved.

[2rs2ts]

Ah, actually, my workaround broke because of https://github.com/hashicorp/terraform/issues/10908 - in terraform 0.8.1, my workaround is still successful.

[apparentlymart]

Sorry that I'm diving into this issue late and probably missing context here, but I just wanted to call out something from the original issue report above:

  records = ["${aws_network_interface.foo.*.private_ips}"]

In 0.6 and prior, this weird syntax (a single-element array containing an interpolation) was necessary to trick the system into seeing this as a list. When first-class list support was added in 0.7, the recommended way to write this changed to this:

  records = "${aws_network_interface.foo.*.private_ips}"

With first-class list support, the expression evaluator can see that this expression is a list without needing the artificial list mapping. I believe the intent was that the old behavior would still work, but it looks like there was some sort of regression there in 0.7. I would suggest removing the brackets to see if it makes it work better on 0.7, though given the rest of the discussion here it does look like there's something else at play here in addition to the change in list representation.

[2rs2ts]

@apparentlymart We tried that, of course 😉

Error loading Terraform: module syslog.root: 1 error(s) occurred:

* resource 'aws_route53_record.syslog' config: use of the splat ('*') operator must be wrapped in a list declaration

[jamesbibby]

This thread is pretty old, but would join(",", my.splatted.*.list_of_ips) not do the job?

[burdandrei]

still an issue https://github.com/burdandrei/multiregion-terraform/blob/master/cdn/module.tf#L156

[mootpt]

https://github.com/hashicorp/terraform/pull/15278

[lblackstone]

Now that #15278 has merged, this should be resolved in 0.10.3

[apparentlymart]

Hi all,

As @lblackstone pointed out, Terraform v0.10.3 added a flatten function for dealing with this problem.

The forthcoming v0.12.0 release will also finally remove support for the old-style redundant list bracket syntax I mentioned in an earlier comment, which has been just a compatibility hack for several versions now. That syntax will instead return a list of lists. The configuration upgrade tool that will be included with the v0.12.0 release should be able to automatically fix this in most cases.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.