Terraform: Experimental feature failure

[sethvargo]

Terraform Version

0.8.5-dev

Affected Resource(s)

-core (state, shadow graph)

Dump

1 error(s) occurred:

* apply operation: ReadDataDiff "data.template_file.workstation" had unequal configs (real, then shadow):

&terraform.ResourceConfig{ComputedKeys:[]string(nil), Raw:map[string]interface {}{"template":"${file(\"${path.module}/templates/workstation.sh.tpl\")}", "vars":[]map[string]interface {}{map[string]interface {}{"hostname":"workstation.${var.cloudflare_domain}", "vault_address":"vault.${var.cloudflare_domain}", "vault_version":"${var.vault_version}", "vault_token":"${random_id.vault-token.hex}", "access_key":"${var.access_key}", "secret_key":"${var.secret_key}", "region":"${var.region}"}}}, Config:map[string]interface {}{"vars":[]map[string]interface {}{map[string]interface {}{"vault_token":"xxx", "access_key":"aaa", "secret_key":"bbb", "region":"us-east-1", "hostname":"workstation.hashicorp.rocks", "vault_address":"vault.hashicorp.rocks", "vault_version":"0.6.4"}}, "template":"#!/usr/bin/env bash\nset -e\n\necho \"==> Updating apt cache...\"\nsudo apt-get -qq update\n\necho \"==> Installing Vault...\"\nsudo apt-get -yqq install curl git jq unzip vim\ncurl -sLo \"vault.zip\" \"https://releases.hashicorp.com/vault/${vault_version}/vault_${vault_version}_linux_amd64.zip\"\nunzip \"vault.zip\"\nsudo mv \"vault\" \"/usr/local/bin/vault\"\nsudo chmod +x \"/usr/local/bin/vault\"\nsudo rm -f \"vault.zip\"\n\necho \"==> Installing Consul Template...\"\ncurl -sLo \"consul-template.tgz\" \"https://releases.hashicorp.com/consul-template/0.18.0/consul-template_0.18.0_linux_amd64.tgz\"\ntar -zxvf \"consul-template.tgz\"\nsudo mv \"consul-template\" \"/usr/local/bin/consul-template\"\nsudo chmod +x \"/usr/local/bin/consul-template\"\nsudo rm -f \"consul-template.tgz\"\n\necho \"==> Setting Vault settings...\"\nsudo tee /etc/profile.d/vault.sh > /dev/null <<\"EOF\"\nexport VAULT_ADDR=\"https://${vault_address}\"\nexport VAULT_TOKEN=\"root\"\nalias v=\"vault\"\nalias vualt=\"vault\"\nEOF\nsource /etc/profile.d/vault.sh\n\necho \"==> Configuring Vault (postgresql)...\"\nwhile true; do\n  # Sleep until ready\n  vault status && break\n  sleep 2\ndone\nif ! vault mounts | grep postgresql; then\n  vault mount postgresql\nfi\nvault write postgresql/config/connection \\\n  connection_url=\"postgresql://localhost/myapp\"\nvault write postgresql/config/lease \\\n  lease=2m \\\n  lease_max=24h\nvault write postgresql/roles/readonly \\\n  sql=-<<\"EOF\"\nCREATE ROLE \"{{name}}\"\nWITH LOGIN PASSWORD '{{password}}'\nVALID UNTIL '{{expiration}}';\n\nGRANT SELECT ON ALL TABLES IN SCHEMA public\nTO \"{{name}}\";\nEOF\n\n# Vault AWS\necho \"Configuring Vault (aws)...\"\nif ! vault mounts | grep aws; then\n  vault mount aws\nfi\nvault write aws/config/root \\\n  access_key=\"${access_key}\" \\\n  secret_key=\"${secret_key}\" \\\n  region=\"${region}\"\nvault write aws/config/lease \\\n  lease=30s \\\n  lease_max=1m\nvault write aws/roles/developer \\\n  policy=-<<\"EOF\"\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": {\n    \"Effect\": \"Allow\",\n    \"Action\": \"ec2:DescribeInstances\",\n    \"Resource\": \"*\"\n  }\n}\nEOF\n\necho \"==> Configuring git...\"\nsudo su ubuntu \\\n  -c 'git config --global color.ui true'\nsudo su ubuntu \\\n  -c 'git config --global user.email \"[email protected]\"'\nsudo su ubuntu \\\n  -c 'git config --global user.name \"HashiCorp Webinars\"'\nsudo su ubuntu \\\n  -c 'mkdir -p ~/.cache; touch ~/.cache/motd.legal-displayed'\n\necho \"==> Creating psqlrc...\"\nsudo tee /home/ubuntu/.psqlrc > /dev/null <<\"EOF\"\n\\set QUIET 1\n\\set COMP_KEYWORD_CASE upper\n\\set PROMPT1 '%n > '\n\n\\echo 'Welcome to PostgreSQL for HashiCorp webinars!'\n\\echo 'Type \\\\q to exit.\\n'\nEOF\nsudo chown ubuntu:ubuntu /home/ubuntu/.psqlrc\n\necho \"==> Updating motd...\"\nsudo rm -rf /etc/update-motd.d/*\nsudo tee /etc/update-motd.d/00-hashicorp > /dev/null <<\"EOF\"\n#!/bin/sh\n\necho \"Welcome to this HashiCorp official webinars! We are very\"\necho \"excited to have you on board for this event.\"\nEOF\nsudo chmod +x /etc/update-motd.d/00-hashicorp\nsudo run-parts /etc/update-motd.d/\nsudo sed -i'' 's/PrintLastLog\\ yes/PrintLastLog\\ no/' /etc/ssh/sshd_config\n\necho \"==> Setting prompt....\"\nsudo tee /etc/profile.d/ps1.sh > /dev/null <<\"EOF\"\nexport PS1=\"webinars@hashicorp > \"\nEOF\n"}, raw:(*config.RawConfig)(0xc420673620)}

&terraform.ResourceConfig{ComputedKeys:[]string(nil), Raw:map[string]interface {}{"template":"${file(\"${path.module}/templates/workstation.sh.tpl\")}", "vars":[]map[string]interface {}{map[string]interface {}{"vault_token":"${random_id.vault-token.hex}", "access_key":"${var.access_key}", "secret_key":"${var.secret_key}", "region":"${var.region}", "hostname":"workstation.${var.cloudflare_domain}", "vault_address":"vault.${var.cloudflare_domain}", "vault_version":"${var.vault_version}"}}}, Config:map[string]interface {}{"template":"#!/usr/bin/env bash\nset -e\n\necho \"==> Updating apt cache...\"\nsudo apt-get -qq update\n\necho \"==> Installing Vault...\"\nsudo apt-get -yqq install curl git jq unzip vim\ncurl -sLo \"vault.zip\" \"https://releases.hashicorp.com/vault/${vault_version}/vault_${vault_version}_linux_amd64.zip\"\nunzip \"vault.zip\"\nsudo mv \"vault\" \"/usr/local/bin/vault\"\nsudo chmod +x \"/usr/local/bin/vault\"\nsudo rm -f \"vault.zip\"\n\necho \"==> Installing Consul Template...\"\ncurl -sLo \"consul-template.tgz\" \"https://releases.hashicorp.com/consul-template/0.18.0/consul-template_0.18.0_linux_amd64.tgz\"\ntar -zxvf \"consul-template.tgz\"\nsudo mv \"consul-template\" \"/usr/local/bin/consul-template\"\nsudo chmod +x \"/usr/local/bin/consul-template\"\nsudo rm -f \"consul-template.tgz\"\n\necho \"==> Setting Vault settings...\"\nsudo tee /etc/profile.d/vault.sh > /dev/null <<\"EOF\"\nexport VAULT_ADDR=\"https://${vault_address}\"\nexport VAULT_TOKEN=\"${vault_token}\"\nalias v=\"vault\"\nalias vualt=\"vault\"\nEOF\nsource /etc/profile.d/vault.sh\n\necho \"==> Configuring Vault (postgresql)...\"\nwhile true; do\n  # Sleep until ready\n  vault status && break\n  sleep 2\ndone\nif ! vault mounts | grep postgresql; then\n  vault mount postgresql\nfi\nvault write postgresql/config/connection \\\n  connection_url=\"postgresql://localhost/myapp\"\nvault write postgresql/config/lease \\\n  lease=2m \\\n  lease_max=24h\nvault write postgresql/roles/readonly \\\n  sql=-<<\"EOF\"\nCREATE ROLE \"{{name}}\"\nWITH LOGIN PASSWORD '{{password}}'\nVALID UNTIL '{{expiration}}';\n\nGRANT SELECT ON ALL TABLES IN SCHEMA public\nTO \"{{name}}\";\nEOF\n\n# Vault AWS\necho \"Configuring Vault (aws)...\"\nif ! vault mounts | grep aws; then\n  vault mount aws\nfi\nvault write aws/config/root \\\n  access_key=\"${access_key}\" \\\n  secret_key=\"${secret_key}\" \\\n  region=\"${region}\"\nvault write aws/config/lease \\\n  lease=30s \\\n  lease_max=1m\nvault write aws/roles/developer \\\n  policy=-<<\"EOF\"\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": {\n    \"Effect\": \"Allow\",\n    \"Action\": \"ec2:DescribeInstances\",\n    \"Resource\": \"*\"\n  }\n}\nEOF\n\necho \"==> Configuring git...\"\nsudo su ubuntu \\\n  -c 'git config --global color.ui true'\nsudo su ubuntu \\\n  -c 'git config --global user.email \"[email protected]\"'\nsudo su ubuntu \\\n  -c 'git config --global user.name \"HashiCorp Webinars\"'\nsudo su ubuntu \\\n  -c 'mkdir -p ~/.cache; touch ~/.cache/motd.legal-displayed'\n\necho \"==> Creating psqlrc...\"\nsudo tee /home/ubuntu/.psqlrc > /dev/null <<\"EOF\"\n\\set QUIET 1\n\\set COMP_KEYWORD_CASE upper\n\\set PROMPT1 '%n > '\n\n\\echo 'Welcome to PostgreSQL for HashiCorp webinars!'\n\\echo 'Type \\\\q to exit.\\n'\nEOF\nsudo chown ubuntu:ubuntu /home/ubuntu/.psqlrc\n\necho \"==> Updating motd...\"\nsudo rm -rf /etc/update-motd.d/*\nsudo tee /etc/update-motd.d/00-hashicorp > /dev/null <<\"EOF\"\n#!/bin/sh\n\necho \"Welcome to this HashiCorp official webinars! We are very\"\necho \"excited to have you on board for this event.\"\nEOF\nsudo chmod +x /etc/update-motd.d/00-hashicorp\nsudo run-parts /etc/update-motd.d/\nsudo sed -i'' 's/PrintLastLog\\ yes/PrintLastLog\\ no/' /etc/ssh/sshd_config\n\necho \"==> Setting prompt....\"\nsudo tee /etc/profile.d/ps1.sh > /dev/null <<\"EOF\"\nexport PS1=\"webinars@hashicorp > \"\nEOF\n", "vars":[]map[string]interface {}{map[string]interface {}{"vault_version":"0.6.4", "vault_token":"xxx", "access_key":"aaa", "secret_key":"bbb", "region":"us-east-1", "hostname":"workstation.hashicorp.rocks", "vault_address":"vault.hashicorp.rocks"}}}, raw:(*config.RawConfig)(0xc4201b0240)}

[mitchellh]

I think this is the usage of random causing this. Not your fault, just talking out loud.

Looking into it.

[mitchellh]

Hm, I can't reproduce this but I'm just going to close this down since the shadowing with the legacy graph is going away in the next [major] release anyways. If we find any easy wins here we can fix it!

[sethvargo]

Hey @mitchellh

I can reproduce this, 100% of the time, without random, but it might just fall under the category of "undefined behavior, don't do that". If you start an apply with instances consuming user-data, then alter that user-data during the apply (but before it completes), you'll get this error.

[onorua]

I was hit by this issue as well:

1 error(s) occurred:

* apply operation: ReadDataDiff "data.template_file.etcd_cloud_config" had unequal configs (real, then shadow):

&terraform.ResourceConfig{ComputedKeys:[]string(nil), Raw:map[string]interface {}{"template":"${file(\"etcd_cloud_config.yaml.tpl\")}", "vars":[]map[string]interface {}{map[string]interface {}{"etcd_discovery_url":"${file(\"nyc2\")}", "private_ip":"${element(digitalocean_droplet.etcd.*.ipv4_address_private, 0)}"}}}, Config:map[string]interface {}{"template":"[Unit]\nDescription=etcd\nDocumentation=https://github.com/coreos/etcd\nConflicts=etcd.service\nConflicts=etcd2.service\n\n[Service]\nType=notify\nRestart=always\nRestartSec=5s\nLimitNOFILE=40000\nTimeoutStartSec=0\n\nExecStart=etcd --data-dir /var/lib/etcd \\\n          --listen-client-urls http://${private_ip}:2379 \\\n          --advertise-client-urls http://${private_ip}:2379 \\\n          --listen-peer-urls http://${private_ip}:2380 \\\n          --initial-advertise-peer-urls http://${private_ip}:2380 \\\n          --discovery ${etcd_discovery_url}\n\n[Install]\nWantedBy=multi-user.target\n", "vars":[]map[string]interface {}{map[string]interface {}{"etcd_discovery_url":"https://discovery.etcd.io/76538e30608496e01b11a483a1ad71b3", "private_ip":"10.128.3.181"}}}, raw:(*config.RawConfig)(0xc420459260)}

&terraform.ResourceConfig{ComputedKeys:[]string(nil), Raw:map[string]interface {}{"template":"${file(\"etcd_cloud_config.yaml.tpl\")}", "vars":[]map[string]interface {}{map[string]interface {}{"private_ip":"${element(digitalocean_droplet.etcd.*.ipv4_address_private, 0)}", "etcd_discovery_url":"${file(\"nyc2\")}"}}}, Config:map[string]interface {}{"vars":[]map[string]interface {}{map[string]interface {}{"etcd_discovery_url":"https://discovery.etcd.io/5c0b51de7f2ed37cfc9f7ac02d0ff2ef", "private_ip":"10.128.3.181"}}, "template":"[Unit]\nDescription=etcd\nDocumentation=https://github.com/coreos/etcd\nConflicts=etcd.service\nConflicts=etcd2.service\n\n[Service]\nType=notify\nRestart=always\nRestartSec=5s\nLimitNOFILE=40000\nTimeoutStartSec=0\n\nExecStart=etcd --data-dir /var/lib/etcd \\\n          --listen-client-urls http://${private_ip}:2379 \\\n          --advertise-client-urls http://${private_ip}:2379 \\\n          --listen-peer-urls http://${private_ip}:2380 \\\n          --initial-advertise-peer-urls http://${private_ip}:2380 \\\n          --discovery ${etcd_discovery_url}\n\n[Install]\nWantedBy=multi-user.target\n"}, raw:(*config.RawConfig)(0xc4205ac5a0)}

Will be glad do provide you with any input you need. This issue happens every time I run terraform apply and it completely blocks me.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.