Terraform: Troubles with template_cloudinit_config and Terraform 0.12

[ycreachcadec]

Terraform Version

v0.12.7

Terraform Configuration Files

provider "aws" {}

resource "aws_ebs_volume" "volume-a" {
  encrypted         = "true"
  type              = "gp2"
  size              = 10
  availability_zone = "eu-west-1a"

  tags = {
    FOO        = "change-mee"
  }
}

data "template_cloudinit_config" "cloudinit" {
  gzip          = false
  base64_encode = false

  part {
    filename     = "cloud-config"
    content_type = "text/cloud-config"
    content = templatefile("./cloud-config",
      {
        FOO = "bar"
      }
    )
  }

  part {
    content_type = "text/x-shellscript"
    content = templatefile("./user_data.sh",
      {
        FOO  = aws_ebs_volume.volume-a.id
        BAR = "baz"
      }
    )
  }
}

resource "aws_instance" "foo" {
  ami           = "ami-0e58c687b1228df2f"
  instance_type = "t2.micro"
  user_data = data.template_cloudinit_config.cloudinit.rendered
  subnet_id = "subnet-9e9e79f8"
}

Expected Behavior

When changing a tag value of the EBS resource, Terraform should only change the EBS, not altering any other resource.

Actual Behavior

When updating a tag of the EBS, Terraform resets the template_cloudinit_config and wants to recreate the EC2 whereas the cloudinit template does not refer to these tags.

user_data = "f108a43b6021c7b27c255decbed14cbabf40b43a" -> (known after apply)

Steps to Reproduce

• Create 2 files matching the template_cloudinit_config
• Make sure terraform is up to date
• Modify a tag of the EBS
• On next plan Terraform wants to recreate the EC2

Additional Context

The problem appears only in Terraform 0.12.
Works great with Terraform 0.11.14

[teamterraform]

Hi @artichaulo! Sorry for this strange behavior and thanks for reporting it.

In order to investigate this further, we will need to see the full plan output and will also _probably_ need the full trace log that our issue template requested.

Could you please add a follow up comment containing the plan output and a link to the trace log? Then we'll see if we can explain what happened here. Thanks!

[ycreachcadec]

Hello Terraform team,

Here is the debug & plan output

Debug Output

https://gist.github.com/artichaulo/e2dee5136454f5724a03613e1470084a

plan output

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

aws_ebs_volume.volume-a: Refreshing state... [id=vol-0ee2c76f1719d817e]
data.template_cloudinit_config.cloudinit: Refreshing state...
aws_instance.foo: Refreshing state... [id=i-09287e6395d4a70c7]

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place
-/+ destroy and then create replacement
 <= read (data resources)

Terraform will perform the following actions:

  # data.template_cloudinit_config.cloudinit will be read during apply
  # (config refers to values not yet known)
 <= data "template_cloudinit_config" "cloudinit"  {
      + base64_encode = false
      + gzip          = false
      + id            = (known after apply)
      + rendered      = (known after apply)

      + part {
          + content      = "#cloud-config\npreserve_hostname: false\nhostname: bar \nrepo_update: true\nrepo_upgrade: all\n\npackages:\n  - lvm2\n  - htop\n  - vim\n  - ntp\n  - curl\n  - zip\n  - python-dev\n  - python-pip\n  - rsync\n  - nfs-common\n  - unzip\n  - ca-certificates\n  - wget\n  - jq\n\n# Configure bashrc, vim and other stuff\nruncmd:\n  - echo \"alias ll='ls -ltrh'\" >> /home/ubuntu/.bashrc\n  - echo \"alias ll='ls -ltrh'\" >> /root/.bashrc\n  - echo \"syntax on\" >> /home/ubuntu/.vimrc\n  - echo \"syntax on\" >> /root/.vimrc\n  \noutput:\n  all: '| tee -a /var/log/cloud-init-output.log'\n\n"
          + content_type = "text/cloud-config"
          + filename     = "cloud-config"
        }
      + part {
          + content      = "#!/bin/bash\n\necho \"my id is vol-0ee2c76f1719d817e\"\n"
          + content_type = "text/x-shellscript"
        }
    }

  # aws_ebs_volume.volume-a will be updated in-place
  ~ resource "aws_ebs_volume" "volume-a" {
        arn               = "arn:aws:ec2:eu-west-1:123456789:volume/vol-0ee2c76f1719d817e"
        availability_zone = "eu-west-1a"
        encrypted         = true
        id                = "vol-0ee2c76f1719d817e"
        iops              = 100
        kms_key_id        = "arn:aws:kms:eu-west-1:123456789"
        size              = 10
      ~ tags              = {
          ~ "FOO" = "change-me" -> "change-mee"
        }
        type              = "gp2"
    }

  # aws_instance.foo must be replaced
-/+ resource "aws_instance" "foo" {
        ami                          = "ami-0e58c687b1228df2f"
      ~ arn                          = "arn:aws:ec2:eu-west-1:123456789:instance/i-09287e6395d4a70c7" -> (known after apply)
      ~ associate_public_ip_address  = false -> (known after apply)
      ~ availability_zone            = "eu-west-1b" -> (known after apply)
      ~ cpu_core_count               = 1 -> (known after apply)
      ~ cpu_threads_per_core         = 1 -> (known after apply)
      - disable_api_termination      = false -> null
      - ebs_optimized                = false -> null
        get_password_data            = false
      + host_id                      = (known after apply)
      ~ id                           = "i-09287e6395d4a70c7" -> (known after apply)
      ~ instance_state               = "running" -> (known after apply)
        instance_type                = "t2.micro"
      ~ ipv6_address_count           = 0 -> (known after apply)
      ~ ipv6_addresses               = [] -> (known after apply)
      + key_name                     = (known after apply)
      - monitoring                   = false -> null
      + network_interface_id         = (known after apply)
      + password_data                = (known after apply)
      + placement_group              = (known after apply)
      ~ primary_network_interface_id = "eni-0fa8114f7606056e3" -> (known after apply)
      ~ private_dns                  = "ip-10-196-133-104.eu-west-1.compute.internal" -> (known after apply)
      ~ private_ip                   = "10.196.133.104" -> (known after apply)
      + public_dns                   = (known after apply)
      + public_ip                    = (known after apply)
      ~ security_groups              = [] -> (known after apply)
        source_dest_check            = true
        subnet_id                    = "subnet-9e9e79f8"
      - tags                         = {} -> null
      ~ tenancy                      = "default" -> (known after apply)
      ~ user_data                    = "f108a43b6021c7b27c255decbed14cbabf40b43a" -> (known after apply) # forces replacement
      ~ volume_tags                  = {} -> (known after apply)
      ~ vpc_security_group_ids       = [
          - "sg-3ab74340",
        ] -> (known after apply)

      - credit_specification {
          - cpu_credits = "standard" -> null
        }

      + ebs_block_device {
          + delete_on_termination = (known after apply)
          + device_name           = (known after apply)
          + encrypted             = (known after apply)
          + iops                  = (known after apply)
          + kms_key_id            = (known after apply)
          + snapshot_id           = (known after apply)
          + volume_id             = (known after apply)
          + volume_size           = (known after apply)
          + volume_type           = (known after apply)
        }

      + ephemeral_block_device {
          + device_name  = (known after apply)
          + no_device    = (known after apply)
          + virtual_name = (known after apply)
        }

      + network_interface {
          + delete_on_termination = (known after apply)
          + device_index          = (known after apply)
          + network_interface_id  = (known after apply)
        }

      ~ root_block_device {
          ~ delete_on_termination = true -> (known after apply)
          ~ encrypted             = true -> (known after apply)
          ~ iops                  = 100 -> (known after apply)
          ~ kms_key_id            = "arn:aws:kms:eu-west-1:123456789" -> (known after apply)
          ~ volume_id             = "vol-075729a1545ed550c" -> (known after apply)
          ~ volume_size           = 8 -> (known after apply)
          ~ volume_type           = "gp2" -> (known after apply)
        }
    }

Plan: 1 to add, 1 to change, 1 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

[eosman-tibco]

I'm also dealing with this issue. The template_cloudinit_config gets rendered freshly on each plan/apply which triggers a cascading update of associated resources.

[jqassar]

I'm also experiencing this issue, but inconsistently. I'm not sure whether it indicates an issue with the templates themselves or the resources relying upon them.

[Quentin-M]

As we rely heavily on data sources (ignition, AWS IAM policies, etc), terraform 0.12 rendered all of our plans be thousands of lines long, even if there is not a single change. It's a massive PITA. Makes any changes impossible to review.

The whole point of Terraform plan is to get us useful information about what's about to happen. "known after apply" with data sources is sad.

[tdmalone]

I _think_ I'm experiencing the same problem - but with a different resource - while moving configuration from 0.11.14 to 0.12.26. This also seems kinda similar to https://github.com/hashicorp/terraform/issues/19234 which was fixed during 0.12 alpha stages.

My plan is for a VPC data source, and looks like this:

Terraform will perform the following actions:

  # module.main.module.tools_sg.data.aws_vpc.selected will be read during apply
  # (config refers to values not yet known)
 <= data "aws_vpc" "selected"  {
      + arn                     = (known after apply)
      + cidr_block              = (known after apply)
      + cidr_block_associations = (known after apply)
      + default                 = (known after apply)
      + dhcp_options_id         = (known after apply)
      + enable_dns_hostnames    = (known after apply)
      + enable_dns_support      = (known after apply)
      + id                      = "vpc-037c86f0f47fffa4a"
      + instance_tenancy        = (known after apply)
      + ipv6_association_id     = (known after apply)
      + ipv6_cidr_block         = (known after apply)
      + main_route_table_id     = (known after apply)
      + owner_id                = (known after apply)
      + state                   = (known after apply)
      + tags                    = (known after apply)
    }

The id is known, so I don't believe any of the other values should be causing the plan to defer reading.

The configuration:

data "aws_vpc" "selected" {
  id = var.vpc_id
}

Here's the interesting bit in my case: the vpc_id there is passed in to this module, from an output of another module that creates the VPC (in the case of this plan, the VPC is already created; so the ID is known).

However, inside that VPC module, the VPC itself is conditionally created, and the output is put together with a join and the splat syntax: join("", aws_vpc.main.*.id)

If I _remove_ the count on this VPC resource, the data source correctly gets read at plan time.

So it appears in my case that even though the VPC ID is known - as it already exists - if the resource has a count set on it, at some point in the plan it appears to be determined somewhere that the read has to be deferred.

I'd be interested to know if the others in this thread also have a count involved in the resources that determine the values ending up in your data source configuration? (If not - I apologise for hijacking this thread!)