Terraform: interpolation of objects (provisioners, resources etc)

[ronpanto]

Let's say I have two types of instances, that share some provisioners, I would have liked to be able to declare a provisioner as a variable before hand, and then just point to that provisioner. Right now I am copying the same provisioner declaration and If it needs fixing I have to fix it in several places.

[mitchellh]

I agree this is something we eventually want to support. I've tagged with "thinking" since we're still thinking on how to best support this.

[willejs]

@mitchellh has there been any progress on this?
Is there a roadmap for terraform?

[mitchellh]

@willejs Not immediately for 0.4. We're taking a look at a lot of the harder ones for 0.4 though so we'll cross through this one as well and I'll let lyou know in better detail.

[knuckolls]

One thought to throw out there is that I've considered that a resource_template might be a nice thing to have.

For example:

template "aws_instance" "my_aws_instance_template" {
    instance_type = "m1.small"
    provisioner "remote-exec" {
        inline = [
        "do_stuff.sh"
        ]
    }
  }
}

resource_template "my_aws_instance_template" "zookeeper1" {
    instance_type = "m1.large"
}

This would set up an aws_instance, apply the template, then override the template. That provides us a way to set defaults and enhance code reusability. I dunno if the terminology is right but that's the sort of thing I'm hoping to see someday. That or something better that solves the same problem.

[tvaughan]

I'd like to be able to have something like:

main.tf:

resource "aws_elb" "elasticsearch_elb" {
    name = "elasticsearch-elb-${var.environment}"
    subnets = ["${var.elasticsearch_elb_subnet}"]
}

staging.tfvars:

environment = "staging"
elasticsearch_elb_subnet = "subnet-38dc4e5d"

production.tfvars:

environment = "production"
elasticsearch_elb_subnet = "subnet-4bcd7d3c"

And be able to run terraform -var-file=production.tfvars to create the production environment, and terraform -var-file=staging.tfvars to create the staging environment. If I run this now I mutate one environment into the other. For my needs, it would be enough if terraform used a new attribute that could contain a variable to keep track of the resources it has built. I assume it uses the resource name (e.g. elasticsearch_elb above) now.

[inferiorhumanorgans]

@tvaughan Does the state argument accomplish what you're looking for?

https://terraform.io/docs/commands/plan.html

[tvaughan]

Does the state argument accomplish what you're looking for?

@inferiorhumanorgans Thank you for the suggestion. I think you mean do something like:

terraform -var-file=production.tfvars -state=production.tfstate
terraform -var-file=staging.tfvars -state=staging.tfstate

right? AFAIK This won't work. The resources in staging.tfstate and production.tfstate will still point to the same resources, and each of these two commands will try to mutate the same resources. Also, I think that one of the two state files will always be out-of-date.

[inferiorhumanorgans]

@tvaughan Correct -- my suggestion there would be to put a UID in the names of your resources and define it in the appropriate variables files. Maybe set a variable named tf_environment to either staging or production and then just use interpolation in the resource name.

[tvaughan]

my suggestion there would be to put a UID in the names of your resources

Tried this. See https://github.com/hashicorp/terraform/issues/353#issuecomment-94917367.

[inferiorhumanorgans]

So I've had a bit of trouble with the security group resource, but the interpolation seems to work fine for me.

https://gist.github.com/inferiorhumanorgans/892008bd3fc1b543fe6b

AWS_PROFILE=aws_profile_name terraform apply -var-file=./stage.tfvars -state=./stage.tfstate
AWS_PROFILE=aws_profile_name terraform apply -var-file=./prod.tfvars -state=./prod.tfstate

Take a look at the state files, the IDs for the security groups are different. The key piece is to ensure that your UID goes into the name attribute. This is with TF 0.5.x.

[tvaughan]

@inferiorhumanorgans Thanks! That did the trick!

[devinrsmith]

I like the template idea.

[andrewmilson]

@mitchellh any progress on this?

[shija03]

One possible work around for templating could be using ruby, and specifically, chef-solo. Though it may seem like an overkill - chef-solo - thanks to Ruby - provides a "template" directive. Using a chef recipe, one could create a template for a terraform file, and then replace everything one desires with variables - the chef-solo script would spit out in any form an shape 1 or more "*.tf" files .

I have tested as follows:

1- Install chef-solo

curl -L https://www.opscode.com/chef/install.sh | bash

2- Confirm chef-solo is installed

type chef-solo

3- Proceed to create a new cookbook and recipes

knife cookbook create terraform
vim /var/chef/cookbooks/terraform/templates/default/consul.erb
vim /var/chef/cookbooks/terraform/recipes/default.rb
vim /etc/chef/solo.rb

4- If all went well - now run

chef-solo -o terraform::default

terraform.zip

[howdoicomputer]

There is something specific I want to do and I'm not sure if this feature or a more robust 'overrides' is the answer.

I've been using kitchen-terraform and awspec to write tests for Terraform modules. It's neat.

However, I have a problem where resources will collide with eachother if tests are ran in parallel or if the infrastructure already exists. This has led me to do stuff like this:

resource "random_id" "testing" {
  byte_length = 8
}

data "template_file" "conditional_testing" {
  template = "$${hex_string}"

  vars {
    hex_string = "${var.testing == "true" ? random_id.testing.hex : ""}"
  }
}

resource "aws_security_group" "standard_lb" {
  name = "foobar${data.template_file.conditional_testing.rendered}"
}

Where I'm generating a randomized suffix per resource depending on whether or not a testing variable is passed into a module.

A resource template would help with this quite a bit.

Edit: @apparentlymart I see you penned the support for the random resources here

Is there a way to provide a random id as a conditional in a way that is more DRY than the approach that I'm taking?

[darenjacobs]

I have 3 virtual networks that I need to create vnet 0 = dmz, vnet 1 = prod, vnet 2 = nonprod.
I list them as follows, i.e. I have 3 blocks that look like this:
resource "azurerm_virtual_network" "vnet0" {
name = "${var.vnet0}"
location = "${var.vnet_location}"
resource_group_name = "${azurerm_resource_group.rg.name}"
address_space = ["${var.vnet0_address_space}"]
tags = "${merge(local.common_tags, var.vnet0_tags)}"
}

Is there any way that I can iterate over these such that 0 is replaced by an index?

[szandala]

Any update on this issue?
I have few instances, all with the same provisioner block, which is annoying (breaks DRY rule as hell)