Terraform: Write a rendered template on a local file

[sbourlon]

Hi there,

I am trying to write a rendered template containing a cloud-init user-data in a local file. Then, I can use this file to generate an ISO for VMware vSphere.

If I understand correctly the documentation, I should be able to write in a file with the resource/data source template_cloudinit_config (https://www.terraform.io/docs/providers/template/d/cloudinit_config.html) thanks to the filename key.

Thanks for your help!

Terraform Version

Tried with Terraform v0.6.16 and v0.7.0.

Affected Resource

• template_cloudinit_config
  

  Terraform Configuration Files

resource "template_cloudinit_config" "test" {
  gzip          = false
  base64_encode = false

  part {
    filename     = "user-data.txt"
    content      = "my userdata content"
  }
}

Expected Behavior

The file user-data.txt should have been created with the content my userdata content.

Actual Behavior

The file user-data.txt is not created.

Steps to Reproduce

1. terraform apply

[kwilczynski]

@sbourlon hello! I am not entirely sure if your use case is something I understood correctly, especially with writing the file locally (where I assumed that you meant the host onto which you run Terraform).

Consider the following example, it would give you some idea how to do it (uses 0.7.x data sources):

variable "hostname"    {}
variable "domain_name" {}

variable "filename" {
  default = "cloud-config.cfg"
}

data "template_file" "test" {
  template = <<EOF
#cloud-config
hostname: ${hostname}
fqdn: ${fqdn}
mounts:
  - [ ephemeral, null ]
output:
  all: '| tee -a /var/log/cloud-init-output.log'
EOF

  vars {
    hostname = "${var.hostname}"
    fqdn     = "${format("%s.%s", var.hostname, var.domain_name)}"
  }
}

data "template_cloudinit_config" "test" {
  gzip          = false
  base64_encode = false

  part {
    filename     = "${var.filename}"
    content_type = "text/cloud-config"
    content      = "${data.template_file.test.rendered}"
  }
}

resource "null_resource" "local" {
  triggers {
    template = "${data.template_file.test.rendered}"
  }

  provisioner "local-exec" {
    command = "echo \"${data.template_file.test.rendered}\" > ${var.filename}"
  }
}

When run terraform apply:

$ terraform apply
var.domain_name
  Enter a value: ellingson-mineral.com

var.hostname
  Enter a value: gibson-01

data.template_file.test: Refreshing state...
data.template_cloudinit_config.test: Refreshing state...
null_resource.local: Creating...
  triggers.%:        "" => "1"
  triggers.template: "" => "Content-Type: multipart/mixed; boundary=\"MIMEBOUNDARY\"\nMIME-Version: 1.0\r\n--MIMEBOUNDARY\r\nContent-Disposition: attachment; filename=\"cloud-config.cfg\"\r\nContent-Transfer-Encoding: 7bit\r\nContent-Type: text/cloud-config\r\nMime-Version: 1.0\r\n\r\n#cloud-config\nhostname: gibson-01\nfqdn: gibson-01.ellingson-mineral.com\nmounts:\n  - [ ephemeral, null ]\noutput:\n  all: '| tee -a /var/log/cloud-init-output.log'\n\r\n--MIMEBOUNDARY--\r\n"
null_resource.local: Provisioning with 'local-exec'...
null_resource.local (local-exec): Executing: /bin/sh -c "echo "Content-Type: multipart/mixed; boundary="MIMEBOUNDARY"
null_resource.local (local-exec): MIME-Version: 1.0
null_resource.local (local-exec): --MIMEBOUNDARY
null_resource.local (local-exec): Content-Disposition: attachment; filename="cloud-config.cfg"
null_resource.local (local-exec): Content-Transfer-Encoding: 7bit
null_resource.local (local-exec): Content-Type: text/cloud-config
null_resource.local (local-exec): Mime-Version: 1.0
null_resource.local (local-exec): #cloud-config
null_resource.local (local-exec): hostname: gibson-01
null_resource.local (local-exec): fqdn: gibson-01.ellingson-mineral.com
null_resource.local (local-exec): mounts:
null_resource.local (local-exec):   - [ ephemeral, null ]
null_resource.local (local-exec): output:
null_resource.local (local-exec):   all: '| tee -a /var/log/cloud-init-output.log'
null_resource.local (local-exec): --MIMEBOUNDARY--
null_resource.local (local-exec): " > cloud-config.cfg"
null_resource.local: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Result:

$ cat cloud-config.cfg
Content-Type: multipart/mixed; boundary=MIMEBOUNDARY
MIME-Version: 1.0
--MIMEBOUNDARY
Content-Disposition: attachment; filename=cloud-config.cfg
Content-Transfer-Encoding: 7bit
Content-Type: text/cloud-config
Mime-Version: 1.0

#cloud-config
hostname: gibson-01
fqdn: gibson-01.ellingson-mineral.com
mounts:
  - [ ephemeral, null ]
output:
  all: '| tee -a /var/log/cloud-init-output.log'

--MIMEBOUNDARY--

The template_cloudinit_config data source (or a resource in the 0.6 speak) would render a cloud-init template (which should be assembled correctly for cloud-init to consume) in memory, so that its rendered content can be used somewhere else. The filename attribute is used to assemble a multi-part payload which cloud-init would understand and in turn deploy multiple files into desired locations - neither it does not save the file onto the remote machine itself nor on the node where you run Terraform from.

If you need to create file on a remote machine, then have a look at the file provisioner which would definitely help.

Hope this helps.

[sbourlon]

Thank you @kwilczynski! It worked!

[sbourlon]

I am reopening this issue because the example given by @kwilczynski is using Shell to write to the file. Is data "template_cloudinit_config" supposed to write the content key value into the file given by the key filename?

resource "null_resource" "local" {
  triggers {
    template = "${data.template_file.test.rendered}"
  }

  provisioner "local-exec" {
    command = "echo \"${data.template_file.test.rendered}\" > ${var.filename}"
  }
}

[kwilczynski]

@sbourlon if you look at the bottom of my reply, you will get an idea what the filename is used for, also pay attention to the multi-part payload content, example:

Content-Disposition: attachment; filename=cloud-config.cfg

I think, we need to update documentation to mention that this data source would not save anything locally, and the filename is only valid within the part section, which in turn is used for the multi-part payload. At the moment this is not very clear, indeed.

For posterity: https://help.ubuntu.com/community/CloudInit

@sbourlon at the moment, there are no facilities other than local-exec to perform actions and save content locally on the machine from which the Terrafrom was run, this is why I used echo to output to content of the template to the file.

[sbourlon]

Thanks @kwilczynski for your patience and clarification :-)

I was indeed able to write my user-data locally thanks to your example and convert it into an ISO.

[kwilczynski]

@sbourlon any time! If you have any other question, feel free to ask!

Also, apologies for previous message being so terse in details.

[keymon]

I did this approach, because I am not sure if that echo is safe for special chars. I use a here document without substitutions:

variable "filename" {
  default = "container_env.json"
}

resource "null_resource" "test" {
  triggers {
      test = "${data.template_file.container_env.rendered}"
  }
  provisioner "local-exec" {
    command = "${format("cat <<\"EOF\" > \"%s\"\n%s\nEOF", var.filename, data.template_file.container_env.rendered)}"
  }
}

[kwilczynski]

@keymon good point, actually. Since it was just an example (with the echo) I didn't really thought much about escaling (shell-safe) the special characters.

[Remigius2011]

@keymon and @kwilczynski works like a charm! thanks. :+1:

In my case I had to use a template file in the module directory as follows (for reference - in case someone stumbles upon this thread):

data "template_file" "interfaces" {
  template = "${file("${path.module}/my_template.tpl")}"
  vars {
    my_var = "my_value"
  }
}

[ehaselwanter]

@sbourlon: do you have a complete example. especially on how you push out the iso to the vm ?

[flmmartins]

I bumped in this issue because I needed to generate a file and the pass it as an argument to a command I was going to execute locally.
After some researching, instead doing like this:

 provisioner "local-exec" {
    command = "${format("cat <<\"EOF\" > \"%s\"\n%s\nEOF", var.filename, data.template_file.container_env.rendered)}"
  }

I opted for:

resource "local_file" "saved-manifesto" {
  content = "${data.template_file.cluster-manifesto.rendered}"
  filename = "${local.cluster_manifesto_path}"
}

resource "null_resource" "run" {
  triggers {
    file = "${data.template_file.cluster-manifesto.rendered}"
  }

  provisioner "local-exec" {
    command = "cat ${local.cluster_manifesto_path}"
  }
}

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.