Terraform: Passing List as Template Variables

[geek876]

I want to render a template as below (effectively create an AWS Policy)

{
  "Version": "2012-10-17",
  "Statement": [
    {
       ...
       ...
      },
      "Action": "ec2:*",
      "Resource": "xxxx*"
    },
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "ec2:*",
      "Resource": "xxx/*",
      "Condition": {
        "IpAddress": {
          "aws:SourceIp": [
            "x.x.x.x/24",
            ...
            ...
            "x.x.x.x/16"
          ]
        }
      }
    }
  ]
}

My Template file _test.tpl_ is

{
  "Version": "2012-10-17",
  "Statement": [
    {
       ...
       ...
      },
      "Action": "ec2:*",
      "Resource": "xxxx*"
    },
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "ec2:*",
      "Resource": "xxx/*",
      "Condition": {
        "IpAddress": {
          "aws:SourceIp": [${whitelist_ips}]
        }
      }
    }
  ]
}

How do I pass ${whitelist_ips} as list to template when I render it ?

I tried the usual way as below but it gives me template parse error

data "template_file" "x" {
  template = "${file("test.tpl")}"

  vars {
    whitelist_ips = [ "${split(",", var.allow_list}" ]
  }
}

My use case is that I don't want to hard-code the IPs within access policy and want to provide a way to pass it as a variable like

variable "allow_list" { default = "x.x.x.x/20,y.y.y/10" }

[llarsson]

Try pushing the allow_list into your template as-is, and use a combo of split and formatlist (link) in the template to get your results.

I didn't know that the interpolation functions are available even within templates, but they are (I could use one to include another file that way... however, a file included that way is included verbatim, so it can't have interpolation functions in it).

[jen20]

Hi @geek876! I haven't read this fully but to give a quick drive-by answer that might get you further quicker than there is an answer: you might want to look at the aws_iam_policy_document data source which can be used to produce all kinds of policies without requiring template rendering. Despite the name, it can also be used for S3, SNS and SQS policies (and likely others too).

[geek876]

Thanks @llarsson and @jen20
I tried few things but none of them seem to work. In the end I scrapped the whole template thing and now trying to write the policy in-line but it is failing too. I may have same issue @jen20 if I use aws_iam_policy_document I believe.

Now I have my module as

variable "ip_whitelist" {}
...
...
resource "aws_elasticsearch_domain" "x" {
...
...
  access_policies = <<EOF
{
"Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "xxx"
      },
      "Action": "es:*",
      "Resource": "$xxxx"
    },
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "xxx",
      "Condition": {
        "IpAddress": {
          "aws:SourceIp": "${var.ip_whitelist}"
        }
      }
    }
  ]
}
EOF

Then I call this module

module "xxx" {
   source = "<path>"
   ...
   ...
  ip_whitelist = [ "${split(",", var.ip_whitelist)}" ]

}

I have

variable "ip_whitelist" { default = "10.0.0.1,10.0.0.2" }

This complained that variable ip_whitelist in module x should be type string, got list`

Then in module, I changed

variable "ip_whitelist" { type = "list" }

But then started to get error
At column 1, line 1: output of an HIL expression must be a string, or a single list (argument 8 is TypeList) in:

[mrwacky42]

Try also jsonencode. eg:

data "template_file" "foo" {
    template = "${file("foo_policy.json.tftemplate")}"
    vars {
        whitelist_ips = "${jsonencode(split(",", var.allow_list))}"    
    }
}

And a template that looks like so...

[SNIP]
      "Condition": {
        "IpAddress": {
          "aws:SourceIp": ${whitelist_ips}
        }
      }
[SNIP]

[geek876]

Thanks @mrwacky42. The jsonencode works well.

[gambol99]

I'm assuming this is the official stance of terraform ... i.e. map and lists are not going to be supported? ..

[jen20]

aws_iam_policy_document does not have these issues - you can interpolate variables at will since the policy document is written in HCL.

[gambol99]

still ... it would be nice though to support more than what the DSL supports ... A primary example being userdata for cloudinit, terraform is a pain when it comes to conditionals and customization

[apparentlymart]

As of right now, template_file is limited by the fact that the Terraform resource schema requires the type of everything to be concretely specified, so we can't have an attribute of type "map of anything".

This is very likely to change eventually, but there's a lot of internal work to do before we can get there to teach Terraform core how to deal with things that are dynamically-typed. For the time being, workarounds (such as those discussed above) are required for handling lists and maps.

The most _general_ workaround is to use the external data source to run an external program to produce the string you need. This data source is subject to the same limitation as template_file, since it all goes through the same core code, but at least with an external program it becomes possible to decode any temporary forms used to "smuggle" lists and maps out to the external program and reproduce the original list or map data to use in the external program's logic.

I'm sorry things aren't smoother in this area. I can only ask for patience as we design and implement the necessary changes to enable this to be improved, which is something we have already started to investigate.

[jen20]

@gambol99 https://www.terraform.io/docs/providers/template/d/cloudinit_config.html should work in the meantime, also.

[kevinchabreck]

I was able to get this to work with an in-line policy by wrapping my terraform list variable with jsonencode:

variable "whitelist" {
  default = [
    "10.0.0.1/32",
    "10.0.0.2/32"
  ]
}

resource "aws_s3_bucket_policy" "b" {
  bucket = "${aws_s3_bucket.b.id}"
  policy =<<POLICY
{
  "Version": "2012-10-17",
  "Id": "${aws_s3_bucket.b.id}-bucket-access-policy",
  "Statement": [
    {
      "Sid": "IPAllow",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::${aws_s3_bucket.b.id}/*",
      "Condition": {
         "IpAddress": {"aws:SourceIp": ${jsonencode(var.whitelist)}}
      }
    }
  ]
}
POLICY
}

You can also get extra explicit and define your whitelist as a map where the keys are the IP(s) and the values are their canonical descriptions (this is nice for creating well-named aws_security_group_rule resources).

variable "whitelist" {
  default = {
    "10.0.0.1/32" = "office"
    "10.0.0.2/32" = "home"
  }
}

resource "aws_s3_bucket_policy" "b" {
  bucket = "${aws_s3_bucket.b.id}"
  policy =<<POLICY
{
  "Version": "2012-10-17",
  "Id": "${aws_s3_bucket.b.id}-bucket-access-policy",
  "Statement": [
    {
      "Sid": "IPAllow",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::${aws_s3_bucket.b.id}/*",
      "Condition": {
         "IpAddress": {"aws:SourceIp": ${jsonencode(keys(var.whitelist))}}
      }
    }
  ]
}
POLICY
}

[osterman]

In addition to the suggestions above, I recommend to use iam_policy_document to generate the JSON using native HCL types. It provides a json output and you can chain multiple documents together by passing source_json.

[hashibot[bot]]

This issue has been automatically migrated to terraform-providers/terraform-provider-template#40 because it looks like an issue with that provider. If you believe this is _not_ an issue with the provider, please reply to terraform-providers/terraform-provider-template#40.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.