Terraform: nested variables, or 'Templatable' terraform module?

Created on 4 Mar 2015  ยท  19Comments  ยท  Source: hashicorp/terraform

Sorry if this is a dupe. I looked through the issues list and didn't find anything I was looking for.

What I am looking for is a way to pass a value to a module that is automatically interpolated back into the module. The use case here is a vpc module that sets up everything needed for a particular vpc - subnets, routing tables, et cetera. You do not reuse code, and you ensure all vpcs are configured the same. So you might have:

module "my_vpc_module" "vpc_name" {
    source "./my_vpc_module"
}

and in the module you'd have configuration like:

resource "aws_vpc" "${var.vpc_name}" {
    cidr_block = "${var.cidr_block}"
    enable_dns_hostnames = "true"
}
resource "aws_internet_gateway" "${var.vpc_name}" {
    vpc_id = "${aws_vpc.${var.vpc_name}.id}"
    tags {
        Name = "${var.vpc_name}"
    }
}

that clearly throws a syntax error, but look at most of the aws_provider resources: if they interact with a vpc they all have a vpc_id var that could be populated at creation by passing the name of the module up to the rest - and then terraform has created a vpc and its components that all share similar identification. But it's missing that step.

config enhancement

Most helpful comment

The fact that modules namespace their resources is fantastic. However, I still think this functionality would be useful:

vpc_id = "${aws_vpc.${var.vpc_name}.id}"

For instance, if the VPC referred to with ${var.vpc_name} is created outside of the module, this would be a great way to refer to it from inside the module. Instead, I'm forced to create a var for every property of the outside VPC and pass the values into the module.

Or to put it in other terms, it would be nice if I can pass a reference to an external resource into a module to use internally. Similarly to how in an object oriented language you can pass a reference to an object. I know this isn't trivial to implement but it's something to consider for the future :)

All 19 comments

I'm not sure I apologize the use case, but names within modules are namespaced so you can just name the VPC anything you'd like, and even re-use the module and that'd be okay. In this case:

resource "aws_vpc" "vpc" {
    cidr_block = "${var.cidr_block}"
    enable_dns_hostnames = "true"
}
resource "aws_internet_gateway" "${var.vpc_name}" {
    vpc_id = "${aws_vpc.vpc.id}"
    tags {
        Name = "${var.vpc_name}"
    }
}

(Feel free to continue discussion, but closing since I don't think you need this behavior as I currently undertand)

Just curious. So what you're saying is it's namespaced within the module itself? So the namespace only matters in the module?

Correct yeah. The names of resources are unique to their modules. Example:

module "foo" {
    source = "./child"
}

module "bar" {
    source = "./child"
}

Won't collide.

I gotta gush, terraform is well-architected. I am used to fighting tooth-and-nail to get a supposedly convenient app to work the way I want.

This is beautiful :joy_cat:.

I've re-read the Modules documentation and the Configuration/Modules page. It isn't clear in the documentation that this is possible. The syntax (in hindsight) makes it obvious that the same module can be referenced multiple times, but that doesn't give us hints as to the namespacing of the resources within the module.

One question: can you reference the NAME used in the module block from within the module's resources? For instance, given the above foo/bar example:

resource "aws_instance" "web" {
    name = "$var.NAME-web"
}

Obviously, I could alternatively require the name as a parameter.

@mpareja You can't reference the name, we don't want to leak that state into it. You can use a module variable though. We might change stances on this as time goes on and more use cases emerge, but for now we're happy with this apporach.

Using soft links actually works pretty well for most of my needs

for file in $(ls ../common); do ln -s "../common/${file}" "common_${file}"; done

The fact that modules namespace their resources is fantastic. However, I still think this functionality would be useful:

vpc_id = "${aws_vpc.${var.vpc_name}.id}"

For instance, if the VPC referred to with ${var.vpc_name} is created outside of the module, this would be a great way to refer to it from inside the module. Instead, I'm forced to create a var for every property of the outside VPC and pass the values into the module.

Or to put it in other terms, it would be nice if I can pass a reference to an external resource into a module to use internally. Similarly to how in an object oriented language you can pass a reference to an object. I know this isn't trivial to implement but it's something to consider for the future :)

I have a case for this.
I'm using some cloudformation temaplates in terraform and I would like to create DNS records based on the outputs:

resource "aws_route53_record" "public-stacks" {
  count   = "${length(var.public_agent_stacks)}"
  zone_id = "${var.dns_hosted_zone}"
  name    = "*.${substr(element(var.public_agent_stacks, count.index), 7, -1)}.${var.cluster_name}"
  type    = "CNAME"
  records = ["${aws_cloudformation_stack.public.${var.cluster_name}-${element(var.public_agent_stacks, count.index)}.outputs.PublicAgentDNSName}"]
  ttl     = 300
}

Is there any other way I could achieve this?

Hi all,

It will not be possible to interpolate into variable names within interpolation expressions; this would add far too much dynamism and make it hard or impossible to do the graph analysis Terraform requires to do its work.

However, as part of some early planning for improving the configuration language we have passing complex objects as values as one idea. It's still too early to say exactly what that will look like, but I just wanted to note that this use-case _is_ on our radar, it'll just take some time for us to figure out the right combination of features to holistically support many different use-cases. More news when we have it!

Awesome, thanks for the update.

BTW, I found the solution for my problem above, I was over complicating it:

resource "aws_route53_record" "public-stacks" {
  count   = "${length(var.public_agent_stacks)}"
  zone_id = "${var.dns_hosted_zone}"
  name    = "*.${substr(element(var.public_agent_stacks, count.index), 7, -1)}.${var.cluster_name}"
  type    = "CNAME"
  records = ["${element(var.public_agent_stacks, count.index)}.outputs.PublicAgentDNSName"]
  ttl     = 300
}

I have a use case for this feature.
I have the variables.tf where I would declare the template variable either in one map called common or in another map associated to the instance.
Those templates are stored in Consul, and the main.tf is generated against a Jinja template.
I can use coalesce, to use either the variable used in the instance, or use a failback value if the first doesn't exist.

Now, if variable nesting would work, I need something as following:

guest_id = "${data.vsphere_virtual_machine.${coalesce(var.{{ instance }}["template"], var.common["template"]).guest_id}"
scsi_type = "${data.vsphere_virtual_machine.${coalesce(var.{{ instance }}["template"], var.common["template"]).scsi_type}"
template_uuid = "${data.vsphere_virtual_machine.${coalesce(var.{{ instance }}["template"], var.common["template"]).id}"

p.s.: the double curly brackets come from Jinia --> {{ instance }}

Use case:

user_data = "${data.template_file.${var.os}.rendered}"

Another use case. AWS S3 buckets are global. I need to create buckets per region

resourse "aws_s3_bucket" "logs_${var.region} {
...
}

resource "aws_s3_bucket_policy" "logs_${var.region}" {
bucket = "${aws_s3_bucket.logs_${var.region}}"
}

I'm getting around this via locals:
locals {
logs_bucket_id = "aws_s3_bucket.logs_${var.region}.id"
}
resource "aws_s3_bucket_policy" "logs_${var.region}" {
bucket = "${logs_bucket_id}"
}

But this just complicates matters. A recursive eval of variables by terraform would make this so much more straight forward.

Arg, running into the other limitation that you can't dynamically specify resource names.

${lookup(var.map2, lookup(var.map1, var.key1))} can help you

I'm going to lock this issue because it has been closed for _30 days_ โณ. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

ketzacoatl picture ketzacoatl  ยท  3Comments

zeninfinity picture zeninfinity  ยท  3Comments

rjinski picture rjinski  ยท  3Comments

jrnt30 picture jrnt30  ยท  3Comments

rnowosielski picture rnowosielski  ยท  3Comments