Terraform: cloud-init failures for aws autoscaling when user data compressed and encoded

[dtaylor-sfdc]

cloud-init fails for me every time under the following circumstances: an aws_launch_configuration using template_cloudinit_config user data with base64_encode = true and gzip = true (i.e., the defaults).

In /var/log/cloud-init.log this is the only WARNING message (there are no ERROR messages):

__init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata: 'H4sIAAAJbogA/7xWbY/bxBN/…

I thought it might be an encoding/marshalling problem. However, curl -L http://169.254.169.254/latest/user-data/ | base64 --decode | gunzip | munpack -t unpacks the user data fine when I have base64_encode = true and gzip = true.

If I explicitly set base64_encode = false and gzip = false (without changing anything else) cloud-init works.

We also use the defaults of base64_encode = true and gzip = true with the aws_instance resource and don't have any problems with cloud-init.

Terraform Version

0.6.16

Affected Resource(s)

• aws_launch_configuration
• template_cloudinit_config
  

  Terraform Configuration Files

I've hacked down our Terraform configuration files to give a rough skeleton of the problem:

resource "template_cloudinit_config" "as_cloudinit_config" {
  part {
    content_type = "text/cloud-config"

    content = <<EOF
hostname: ${format(var.as_fqdn_format, var.role, var.revision, var.vpc, var.domain)}
fqdn: ${format(var.as_fqdn_format, var.role, var.revision, var.vpc, var.domain)}
EOF
  }

  part {
    content_type = "text/cloud-config"
    content      = "${file("${path.module}/files/cloud-init/common.txt")}"
  }

  part {
    content_type = "text/x-shellscript"
    content      = "${template_file.puppet_install.rendered}"
  }

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_launch_configuration" "as_configuration" {
  name_prefix                 = "${format("%v-%v-%v-", var.role, var.revision, var.vpc)}"
  image_id                    = "${var.ami}"
  instance_type               = "${var.instance_type}"
  iam_instance_profile        = "${var.instance_profile}"
  security_groups             = ["${compact(split(",", "${aws_security_group.as_role_security_group.id},${var.vpc_security_group_id}"))}"]
  user_data = "${template_cloudinit_config.as_cloudinit_config.rendered}"

  lifecycle {
    create_before_destroy = true
  }
}

[svperfecta]

This was happening to me too. Thanks for the bug, helped me get my setting fixed so that at least things are working!

[chiefy]

@genexp small world - I just ran into this as well!

[calvn]

We are also encountering this issue, though no triggered with aws_launch_configuration. We are passing in the cloud-init configuration (through rendered template_file) via user_data property under aws_instance, and the cloud-init works fine on the first boot. However, on subsequent reboots the user data somehow gets decoded even though it doesn't need to which then turns the valid user data to gibberish.

Edit: Might be worth mentioning that we are using template_file to directly pass in the cloud-init configuration, and not using template_cloudinit_config, so it shouldn't be an encoding issue since there is no property for that on template_file. Might be a different case.

[carlossg]

This happens in openstack too, setting base64_encode = true causes the same error

Unhandled non-multipart (text/x-not-multipart) userdata

What I noticed is that in EC2 instances created with base64 = true the contents of /var/lib/cloud/instance/user-data.txt and http://169.254.169.254/latest/user-data are NOT base64 decoded
So it seems as if EC2 translates them before creating the instance, it is not done by cloud-init. Maybe AWS launch configurations and openstack instances don't support base64 encoded user-data.

[miry]

I have same issue for digitalocean droplet. Switched gzip and base64 to false

[edify42]

Still happening on v0.7.11

[JimiHFord]

In case other people make the same mistake I did, you get a very similar warning if you forget to start your userdata script with

#!/usr/bin/env bash

documentation

That is, of course, only if it is a script and not something like cloud config yaml. I'm not sure what happens in that case but I imagine you would get a similar (unhelpful) warning like this.

[tamsky]

This is caused by #cloud-config being missing from the first line of parts with type text/cloud-config .

Reported as "bug":

resource "template_cloudinit_config" "as_cloudinit_config" {
...
    content = <<EOF
hostname: ${format(var.as_fqdn_format, var.role, var.revision, var.vpc, var.domain)}
fqdn: ${format(var.as_fqdn_format, var.role, var.revision, var.vpc, var.domain)}
EOF
  }

Correct:

resource "template_cloudinit_config" "as_cloudinit_config" {
...
    content = <<EOF
#cloud-config
hostname: ${format(var.as_fqdn_format, var.role, var.revision, var.vpc, var.domain)}
fqdn: ${format(var.as_fqdn_format, var.role, var.revision, var.vpc, var.domain)}
EOF
  }

[ariesyous]

Can we not reference the cloud-init file as a separate file in the terraform directory? Or do we have to use the content argument? @tamsky

[ffledgling]

@tamsky adding the missing #cloud-config did indeed fix it for me. Cheers!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.