Terraform: How to insert spaces to make something yaml compliant?

[ctindel]

Using terraform 0.7.11 here. I have not yet figured out a good way to do this that is clean and doesn't involve some external script.

What I'm trying to do is use variables to insert scripts into the write_files section of my cloud-config. For example, I have a service.sh script:

#!/bin/bash

echo "this is a test"

So I want to have a template file like this:

data "template_file" "user-data" {
  template = "${file("${path.module}/user_data.yml")}"

  vars {
    service_script = "${file("${path.module}/service.sh")}"
  }
}

And then use that in my cloud-config like this:

write_files:
    - path: /root/service.sh
      content: |
${service_script}

But when I do that, the script content won't be indented and so the yaml will be invalid.

Is there a way to say "indent every line in this string by 8 characters" or somehow otherwise get terraform to output valid yaml here? I'm trying to avoid putting my script code right into the user_data.yml file because it has $ all over the place and I don't want to go around making invalid bash code with $$ just to work around terraform trying to do a substitution in the bash code.

Alternately, is there some way to denote to terraform inside my template that "this section should not try to have variables interpolated" ? Similar to wrapping an EOF in single quotes in bash a la:

cat - > /tmp/file.txt << 'EOF'
$this_string_will_not_be_interpolated
EOF

[apparentlymart]

Hi @ctindel,

There isn't a good way to do this right now.

One possible workaround would be to exploit the fact that JSON is (mostly) valid YAML:

data "template_file" "user-data" {
  template = "${file("${path.module}/user_data.yml")}"

  vars {
    service_script_json = "${jsonencode(file("${path.module}/service.sh"))}"
  }
}

write_files:
    - path: /root/service.sh
      content: ${service_script_json}

The result will not be as human readable as having it inlined with indentation as you wanted, but I think it should still parse the same to a YAML parser.

[ctindel]

Thanks @apparentlymart.

Well, I guess another question would be, is there a best practice for passing files along from terraform directories into the cloud-config user data? I'm not married to the way i'm trying to do it, but it seems like having to escape every single dollar sign is just a recipe for disaster because there's no way to test that code on its own outside of rendering a template first.

I can't believe I'm the first person to want to insert files into cloud-config yaml but I couldn't find a single example of someone trying to do it online.

[apparentlymart]

Hi @ctindel,

I'm not sure I completely understand the general problem you're referring to here, putting aside the specific situation you described here with your bash script in YAML.

I think what you're asking about is whether Terraform has or could have a "raw string" sort of construct, where interpolations aren't processed? In which case, I can say that such a thing does not exist now, though whether it could be made to exist is of course a different question... it's architecturally a bit tricky due to how Terraform's configuration language is really two separate languages (HCL for the overall configuration structure and HIL for the interpolations), but of course that doesn't mean it can't happen.

In the short term I would say that what you are doing here by reading your literal data from separate files using file(...) is the best practice. This avoids them passing through Terraform's interpolation layer, and as a bonus also makes them easier to edit since your text editor can use its native mode for the file type (bash scripts, in this case).

[ctindel]

@apparentlymart at the highest level of abstraction the general problem I'm referring to is "I have files that I want to create on a server when it boots by having the cloud-config yaml write it out via the write_files parameter". A lot of times these are just config files and so it works fine but when you're trying to do a bash script or something with a lot of $ it becomes problematic because terraform tries to do interpolation.

For now I'm just indenting all my source files by 8 spaces to work around this problem because at least that way they still run on their own even if they look funny in code.

I'm thinking that it might be nice to have a directive for rendering files where indentation matters (like yaml, python scripts, etc) to say "render this file and indent everything to start at the current location of the interpolated variable". That is, if I put:

write_files:
    - path: /root/service.sh
      content: |
        ${service_script_json, KEEP_INDENTATION_PARAMETER=true}

Then what will be rendered is:

write_files:
    - path: /root/service.sh
      content: |
        #!/bin/bash

        echo "this is a test"

I have no idea what that parameter passing would actually look like in terraform-land so I'll just put a placeholder there.

I'm suddenly reminded of an episode of Silicon Valley, because terraform would probably have to notice where tabs or spaces were used and match accordingly. :)

I guess this is my feature request. I'm not sure how to do that officially, so terraform devs can do with this what they will. :)

[paultyng]

Instead of dealing with indentation in the cloud config, you can base64 encode the content via terraform interpolation and then the indentation doesn't matter: https://coreos.com/os/docs/latest/cloud-config.html#writefiles

write-files:
  - path: /etc/kubernetes/ssl/ca.pem
    encoding: base64
    content: ${base64encode(ca_pem)}

(I know this doesn't address the request about the indentation / raw feature, but how I solved a similar problem)

[ctindel]

@paultyng Yes, that is precisely what I was looking for, thank you it works great!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.