Terraform: Rendering the template_file resource into a file

[bluk]

Is there any easy way to save the rendered template_file resource into a file itself? I want to render a configuration file and save it to a remote machine eventually. I was hoping that I could use the file provisioner and just set something like a "contents" variable to the rendered config.

[worldspawn]

I too am wondering how to do this

[chiefy]

@bluk you can use the output command and redirect to file. I did this when trying to mock some templates locally for use w/ Vagrant.

[ketzacoatl]

@chiefy, would you be able to paste/gist an code as an example of what you do?

[stvdilln]

Until something more elegant shows up, you can use as below, but your template file needs to use correct quotes. It works for some files, not others.

provisioner "remote-exec" {
  inline= [
     "echo \"${template_file.etc-sysconfig-docker.rendered}\" > tmp.txt",
     "sudo cp tmp.txt /etc/sysconfig/docker"
    ]
}

[packplusplus]

@stvdilln FWIW I'm pretty sure heredocs will work around whatever quoting errors you're getting.

provisioner "remote-exec" {
  inline = [
    "cat <<FILEXXX > /tmp/somefile",
    "${template_file.my_tpl.rendered}",
    "FILEXXX"
  ]
}

Does a PR that adds that trick to the docs make sense?

[stvdilln]

The heredoc looks like a good improvement because you shouldn't have to worry about < && | and other interesting characters that would foul up a command line.

This is tagged as a bug against the template provider, and I'd rather see it fixed than promote these work arounds.

[jorgemarey]

Maybe something like this could be done:

provisioner "file" {
        content = "${template_file.config.rendered}"
        destination = "/etc/config.d/config.json"
}

One of content or source must be provided, but no both.

This would copy the content provided to the destination file. I think this could be achieved without a lot of change in code. Simple by creating a file in your machine tmp directory, copy the content there and use it as the source file.

[ketzacoatl]

++ to @jorgemarey's recommendation, great stuff!

[apparentlymart]

Terraform has been moving towards allowing inline strings instead of filenames in a bunch of places, so something like what @jorgemarey proposed seems reasonable.

Reminds me that I also once had a use-case for copying files in the "reverse" direction (remote to local) during provisioning, but I apparently worked around that and lost the use-case in the mean time. Maybe if we're altering the "file" provisioner it could gain that flexibility too, kinda like how packer's file provisioner recently got a direction property for this. (Though if it's non-trivial to implement I certainly don't mean to imply that it should block fixing the issue described by this ticket!)

[packplusplus]

:+1: to adding contents to the file provisioner would be great.
:+1: to direction for the file provisioner (being able to do that thru a bastion host :+1: :+1: )

There is a hack/workaround for the contents idea, so I'm not sure how soon that would happen.

[pmvilaca]

+1

[calvn]

:+1:

[anosulchik]

+1

[urog]

:+1:

[iMoses]

+1

[jonapich]

+1, this would really simplify passing complex information to custom local-exec provisioners.

[jonapich]

I have not found a workaround for my use case. I have a rather complex JSON to feed to a script. The easiest would be to render the template into a file and pass the filename to the script. Using the echo/redirect trick doesn't seem to work: the resulting file has all its double quotes escaped and the script cannot read it as JSON anymore. When the pound sign is included in windows, the echo doesn't work at all.

Here are several attempts that don't work. The closest one is the 10-a/10-b attempts, but I couldn't find a way to get rid of the escaped double-quotes in the resulting file.

test.py can be used with a value or with a filename.
These executions will work:

test.py testvalue "{\"somekey\": \"foo\"}"
test.py testfilename expected.json

import baker, json
@baker.command
def testvalue(value):
    d = json.loads(value)
    try:
        assert 'somekey' in d
        assert d['somekey'] == 'foo'
        print("OK SUCCESS!")
    except AssertionError:
        print("FAILED: " + value)
@baker.command
def testfilename(filename):
    with open(filename) as f:
        value = f.read()
    testvalue(value)
baker.run()

json.template:

{
    "somekey": "${somekey}"
}

test.tf:

resource "template_file" "json" {
  template = "${file("json.template")}"
  vars { somekey = "foo" }
}

resource "null_resource" "test1" {
  provisioner "local-exec" {
   command = "python ./test.py testvalue ${template_file.json.rendered}"
  }
}

resource "null_resource" "test2" {
  provisioner "local-exec" {
   command = "python ./test.py testvalue '${template_file.json.rendered}'"
  }
}

resource "null_resource" "test3" {
  provisioner "local-exec" {
   command = "python ./test.py testvalue \"${template_file.json.rendered}\""
  }
}

resource "null_resource" "test4" {
  provisioner "local-exec" {
   command = "python ./test.py testvalue ${replace(template_file.json.rendered, "\n", "")}"
  }
}

resource "null_resource" "test5" {
  provisioner "local-exec" {
   command = "python ./test.py testvalue '${replace(template_file.json.rendered, "\n", "")}'"
  }
}

resource "null_resource" "test6" {
  provisioner "local-exec" {
   command = "python ./test.py testvalue \"${replace(template_file.json.rendered, "\n", "")}\""
  }
}

# this one looks like it passes, but the python script is never called executed
resource "null_resource" "test7" {
  provisioner "local-exec" {
   command = <<EOF
echo ${template_file.json.rendered} > test.json
python ./test.py testfilename test.json
EOF
  }
}

resource "null_resource" "test8" {
  provisioner "local-exec" {
   command = <<EOF
"echo \"${template_file.json.rendered}\" > test.json"
"python ./test.py testfilename test.json"
EOF
  }
}

resource "null_resource" "test9" {
  provisioner "local-exec" {
   command = <<EOF
"echo ${replace(template_file.json.rendered, "\n", "")} > test.json"
"python ./test.py testfilename test.json"
EOF
  }
}

resource "null_resource" "test10-a" {
  provisioner "local-exec" {
   command = "echo ${replace(template_file.json.rendered, "\n", "")} > test.json"
  }
}

resource "null_resource" "test10-b" {
  depends_on = ["null_resource.test10-a"]
  provisioner "local-exec" {
   command = "python ./test.py testfilename test.json"
  }
}

expected.json file:

{
    "somekey": "foo"
}

[hansnqyr]

:100: for creating a file from rendered template

Using rendered with remote-exec or local-exec is impractical on Windows, as CMD returns on the first new line character (and it's not practical to make all files a single line long :cry: )

Using rendered with local-exec in a multi-OS environment is actually impossible afaict - there's no command I can construct that will output a multiline string to file on both Windows & *nix.

[leigh507]

+1 just ran into this. Would be incredibly useful!

[mmulji]

I had a need for this exact same functionality @jorgemarey , found terraform-provider-localfile.
Could we get this integrated into the central project @mitchellh ?

https://github.com/mcoffin/terraform-provider-localfile

[jwadolowski]

Had a similar issue, but the following seems to work quite well when sudo is required:

provisioner "remote-exec" {
    inline = [
      "sudo tee /etc/opscode/chef-server.rb <<EOF",
      "${template_file.chef-server-rb.rendered}",
      "EOF",
    ]
}

sudo combined with cat won't work because of redirection issue.

[kesor]

When a community module asks for a filename in a variable to be provided to it, it would be helpful to generate the file from a template and provide the generated file easily.

Example:

module "launch_config" {
  userdata_filename => "${template_file.userdata.rendered_filename}"
}

[jen20]

Fixed in #7561!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.