Terraform: "Unsupported attribute" errors on valid code that work in 0.12.17

[ryanking]

Terraform Version

0.12.18

I tested reverting to 0.12.17 (with no other changes) and I no longer get this error.

I tried aws provider 2.41.0 and 2.42.0 and both fail on 0.12.18 but succeed on 0.12.17.

Terraform Configuration Files

I have several places in our code where we use this open-source module (that we wrote) - https://github.com/chanzuckerberg/cztack/tree/master/aws-ecs-job

Here is an example that matches as close as possible the failing code, but note that when trying to do this from a clean state it works–

locals {
  env     = "foo"
  project = "bar"
  owner   = "[email protected]"
  service = "this"
}

resource aws_ecs_cluster cluster {
  name = "cluster"
}

module "ecs-role" {
  source  = "github.com/chanzuckerberg/cztack//aws-iam-ecs-task-role?ref=v0.25.0"
  project = local.project
  env     = local.env
  owner   = local.owner

  service = local.service
}


module "service" {
  source = "github.com/chanzuckerberg/cztack//aws-ecs-job?ref=v0.25.0"

  env     = local.env
  project = local.project
  service = local.service
  owner   = local.owner

  desired_count = "1"
  cluster_id    = aws_ecs_cluster.cluster.id
  task_role_arn = module.ecs-role.arn

  manage_task_definition = false
  tag_service            = false
}

Debug Output

I am still looking through the trace logs to be sure there is nothing sensitive in them (and probably need approval from our security team to post them.

But I did find these lines related to the failing resource–

2019/12/13 17:41:34 [TRACE] renamed first module.kg-monitoring.aws_ecs_task_definition.job instance in transient state due to count argument change

2019/12/13 17:30:44 [TRACE] EvalReadState: reading state for module.kg-monitoring.aws_ecs_task_definition.job
2019/12/13 17:30:44 [TRACE] UpgradeResourceState: schema version of module.kg-monitoring.aws_ecs_task_definition.job is still 1; calling provider "registry.terraform.io/-/aws" for any other minor fixups

2019-12-13T17:42:17.984-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4: 2019/12/13 17:42:17 [DEBUG] [aws-sdk-go] {"tags":[],"taskDefinition":{"compatibilities":["EC2"],"containerDefinitions":[{"command":["sh","-c","while true; do { echo -e 'HTTP/1.1 200 OK\r\n\nRunning stub server'; date; } | nc -l -p 8080; done"],"cpu":0,"environment":[],"essential":true,"image":"library/busybox:1.29","memoryReservation":4,"mountPoints":[],"name":"meta-prod-kg-monitoring","portMappings":[],"volumesFrom":[]}],"family":"meta-prod-kg-monitoring","placementConstraints":[],"requiresAttributes":[{"name":"com.amazonaws.ecs.capability.docker-remote-api.1.21"},{"name":"com.amazonaws.ecs.capability.task-iam-role"}],"revision":1,"status":"ACTIVE","taskDefinitionArn":"arn:aws:ecs:us-west-2:607813682535:task-definition/meta-prod-kg-monitoring:1","taskRoleArn":"arn:aws:iam::607813682535:role/meta-prod-kg-monitoring","volumes":[]}}
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4: 2019/12/13 17:42:17 [DEBUG] Received task definition meta-prod-kg-monitoring, status:ACTIVE
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:  {
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:   Tags: [],
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:   TaskDefinition: {
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:     Compatibilities: ["EC2"],
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:     ContainerDefinitions: [{
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         Command: ["sh","-c","while true; do { echo -e 'HTTP/1.1 200 OK\r\n\nRunning stub server'; date; } | nc -l -p 8080; done"],
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         Cpu: 0,
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         Environment: [],
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         Essential: true,
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         Image: "library/busybox:1.29",
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         MemoryReservation: 4,
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         MountPoints: [],
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         Name: "meta-prod-kg-monitoring",
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         PortMappings: [],
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         VolumesFrom: []
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:       }],
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:     Family: "meta-prod-kg-monitoring",
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:     PlacementConstraints: [],
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:     RequiresAttributes: [{
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         Name: "com.amazonaws.ecs.capability.docker-remote-api.1.21"
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:       },{
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:         Name: "com.amazonaws.ecs.capability.task-iam-role"
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:       }],
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:     Revision: 1,
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:     Status: "ACTIVE",
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:     TaskDefinitionArn: "arn:aws:ecs:us-west-2:607813682535:task-definition/meta-prod-kg-monitoring:1",
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:     TaskRoleArn: "arn:aws:iam::607813682535:role/meta-prod-kg-monitoring",
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:     Volumes: []
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4:   }
2019-12-13T17:42:17.985-0800 [DEBUG] plugin.terraform-provider-aws_v2.41.0_x4: }

Expected Behavior

I expect the plan to complete with zero changes planned.

Actual Behavior

Errors:

Error: Unsupported attribute

  on .terraform/modules/kg-monitoring/aws-ecs-job/main.tf line 6, in locals:
   6:   task_definition = "${aws_ecs_task_definition.job.family}:${aws_ecs_task_definition.job.revision}"
    |----------------
    | aws_ecs_task_definition.job is empty tuple

This value does not have any attributes.


Error: Unsupported attribute

  on .terraform/modules/kg-monitoring/aws-ecs-job/main.tf line 6, in locals:
   6:   task_definition = "${aws_ecs_task_definition.job.family}:${aws_ecs_task_definition.job.revision}"
    |----------------
    | aws_ecs_task_definition.job is empty tuple

This value does not have any attributes.


Error: Unsupported attribute

  on .terraform/modules/kg-monitoring/aws-ecs-job/outputs.tf line 11, in output "ecs_task_definition_family":
  11:   value       = aws_ecs_task_definition.job.family
    |----------------
    | aws_ecs_task_definition.job is empty tuple

This value does not have any attributes.

Steps to Reproduce

I haven't been able to reproduce it outside the context of our internal repositories, which makes me thing it is related to managing state.

But a few notes -

• it happens in each place where we are using this module
• it happens when running refresh or plan

Additional Context

References

[danieldreier]

@ryanking thanks for reporting this. This sure looks like a valid issue with 0.12.18, but without a reproduction case it's difficult for us to work on. Do you think you will be able to work on a repro that gets the state to the point where this is triggered?

[ryanking]

@danieldreier I am working on it, but haven't figured out a way to reproduce it cleanly.

I have, however found a workaround which might be a clue–

If I go back to my master branch (on 0.12.17) and do an apply operation, which may or may not change anything (have run into this in ~10 places in our code, all with this exact module), then switch to my 0.12.18 branch everything is ok.

[conorcurlett]

I'm also seeing this condition, though since I'm using an s3 backend its a lot more difficult to revert. @ryanking Are you able to do a diff in the state when you apply the workaround to see where the bug might live?

[conorcurlett]

This might help isolate the problem - I compared some states generated by different versions, and noticed that for the item in question that was erring, there was a key difference:
In terraform 12.17 it looked like

{
      "module": "module.eks.module.eks",
      "mode": "managed",
      "type": "aws_eks_cluster",
      "name": "this",
      "each": "list",      
      "provider": "provider.aws",
      "instances": [
...

and a similar example _created_ by 12.18 it was

{
      "module": "module.eks.module.eks",
      "mode": "managed",
      "type": "aws_eks_cluster",
      "name": "this",
      "provider": "provider.aws",
      "instances": [
  ...

i.e. the "each": "list", was what was causing the error.

[ryanking]

I am seeing the same pattern in state files–

1. try to run a plan with 0.12.18 and fails
2. revert to 0.12.17, state is–
   json { "module": "module.dpt-ecs-datadog-agent", "mode": "managed", "type": "aws_ecs_task_definition", "name": "job", "each": "list", "provider": "provider.aws", "instances": [ { "index_key": 0, "schema_version": 1, "attributes": { "arn": "arn:aws:ecs:us-west-2:123456789:task-definition/meta-prod-dpt-ecs-datadog-agent:2", "container_definitions": "[{\"command\":[\"sh\",\"-c\",\"while true; do { echo -e 'HTTP/1.1 200 OK\\r\\n\\nRunning stub server'; date; } | nc -l -p 8080; done\"],\"cpu\":0,\"environment\":[],\"essential\":true,\"image\":\"library/busybox:1.29\",\"memoryReservation\":4,\"mountPoints\":[],\"name\":\"meta-prod-dpt-ecs-datadog-agent\",\"portMappings\":[],\"volumesFrom\":[]}]", "cpu": "", "execution_role_arn": "", "family": "meta-prod-dpt-ecs-datadog-agent", "id": "meta-prod-dpt-ecs-datadog-agent", "ipc_mode": null, "memory": "", "network_mode": "", "pid_mode": null, "placement_constraints": [], "proxy_configuration": [], "requires_compatibilities": [], "revision": 2, "tags": {}, "task_role_arn": "arn:aws:iam::123456789:role/meta-prod-dpt-ecs-datadog-agent", "volume": [] }, "depends_on": [ "data.template_file.task", "local.name", "local.tf_managed_task" ] } ] },
3. run an apply with 0.12.17, now state is
   ```json
   {
   "module": "module.dpt-ecs-datadog-agent",
   "mode": "managed",
   "type": "aws_ecs_task_definition",
   "name": "job",
   "provider": "provider.aws",
   "instances": [
   {
   "schema_version": 1,
   "attributes": {
   "arn": "arn:aws:ecs:us-west-2:123456789:task-definition/meta-prod-dpt-ecs-datadog-agent:2",
   "container_definitions": "[{\"command\":[\"sh\",\"-c\",\"while true; do { echo -e 'HTTP/1.1 200 OK\r\n\nRunning stub server'; date; } | nc -l -p 8080; done\"],\"cpu\":0,\"environment\":[],\"essential\":true,\"image\":\"library/busybox:1.29\",\"memoryReservation\":4,\"mountPoints\":[],\"name\":\"meta-prod-dpt-ecs-datadog-agent\",\"portMappings\":[],\"volumesFrom\":[]}]",
   "cpu": "",
   "execution_role_arn": "",
   "family": "meta-prod-dpt-ecs-datadog-agent",
   "id": "meta-prod-dpt-ecs-datadog-agent",
   "ipc_mode": null,
   "memory": "",
   "network_mode": "",
   "pid_mode": null,
   "placement_constraints": [],
   "proxy_configuration": [],
   "requires_compatibilities": [],
   "revision": 2,
   "tags": {},
   "task_role_arn": "arn:aws:iam::123456789:role/meta-prod-dpt-ecs-datadog-agent",
   "volume": []
   },
   "dependencies": [
   "module.datadog-agent-role.aws_iam_role.role",
   "module.dpt-ecs-datadog-agent.aws_iam_role.task_execution_role"
   ]
   }
   ]
   },
   ````
4. Run plan with 0.12.18 and no error

[grimm26]

@danieldreier

I hit this same error while refactoring a module and changing a resource from a counted resource to normal. I made a simplified example.

• Made a bucket
• Changed the s3_bucket module's aws_s3_bucket to no longer be counted. did a terraform state mv 'module.bucket.aws_s3_bucket.b[0]' module.bucket.aws_s3_bucket.b
• a subsequent apply or plan will show:

Error: Unsupported attribute

  on s3_bucket/main.tf line 15, in output "id":
  15:   value = aws_s3_bucket.b.id
    |----------------
    | aws_s3_bucket.b is empty tuple

This value does not have any attributes.

• files are at https://gist.github.com/grimm26/e973a686526f9d054224d251b713aae6

In the second state file after I changed the module and moved the aws_S3_bucket resource, I think the problem is that even though I removed the count and it removed the "index_key": 0, entry from resources[0]{"instances"}[0]{"attributes"} it did not remove "each": "list" from resources[0]

[mf2012]

I believe this issue still occurs in 0.12.20

• I'm using it to create Openstack environment with TF 0.12.20

• By trying to load state file in backend local
  ```# Remote data source to pull stage1 info
  data "terraform_remote_state" "stage1" {
  backend = "local"

  config = {
  path = "../stage1/terraform.tfstate"
  }
  }

- state file contains in outputs:

"mgmt_subnet_id": {
  "value": "88daefd8-1b7d-4bcd-810a-43246441805f",
  "type": "string"
},

Error when trying 
**terraform plan**

Error: Unsupported attribute

on controllers.tf line 16, in module "pv01":
16: mgmt_subnet_id = "${data.terraform_remote_state.stage1.mgmt_subnet_id}"

This object has no argument, nested block, or exported attribute named
"mgmt_subnet_id".```

[v3n]

_deleted_

Previously posted repro was unrelated to this issue.

[bytejunkie]

I have seen this while refactoring a module and changing a resource from a counted resource to normal.

I worked around it by removing the "each":"list" line from the state file for the resource that was being referenced.

[bryancpowell]

FWIW I can confirm we are seeing something similar on 0.12.21. We had originally used counts on our resources, then simplified them to not use them. The error presented afterwards.

[jbardin]

Consolidating discussion on #23847 which starts with a clear reproduction and has the most 👍 s

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.