Terraform: Moving a module with "terraform state mv" loses "data." prefix for aws_caller_identity address

[nickrw]

Terraform Version

Terraform v0.7.9

Affected Resource(s)

Please list the resources as a list, for example:

• data.aws_caller_identity
• command line "terraform state mv"

Terraform Configuration Files

I created a reproduction case with this simple layout

./sourcemod/test.tf

provider "aws" {
  region = "eu-west-1"
}
data "aws_caller_identity" "current" { }

./root.tf

module "sourcemod" {
  source = "./sourcemod"
}

Expected Behavior

With the above module layout, after running terraform apply you should be able to rename a module, e.g.

terraform apply
terraform state mv module.sourcemod module.destmod
terraform plan

Actual Behavior

Subsequent plan/apply operations result in

module.sourcemod.data.aws_caller_identity.current: Refreshing state...
module.destmod.aws_caller_identity.current: Refreshing state... (ID: 2016-11-09 14:37:57.229678945 +0000 UTC)
Error refreshing state: 1 error(s) occurred:

* aws_caller_identity.current: unknown resource type: aws_caller_identity

Examining the diff between the state and the backup file shows that "data." is being dropped from the resource address:

diff --git a/terraform.tfstate.1478702283..backup b/terraform.tfstate
index 5189272..c706ebd 100644
--- a/terraform.tfstate.1478702283..backup
+++ b/terraform.tfstate
@@ -1,7 +1,7 @@
 {
     "version": 3,
     "terraform_version": "0.7.9",
-    "serial": 0,
+    "serial": 1,
     "lineage": "ba66147c-24c6-42e0-89b5-762458202496",
     "modules": [
         {
@@ -15,11 +15,11 @@
         {
             "path": [
                 "root",
-                "sourcemod"
+                "destmod"
             ],
             "outputs": {},
             "resources": {
-                "data.aws_caller_identity.current": {
+                "aws_caller_identity.current": {
                     "type": "aws_caller_identity",
                     "depends_on": [],
                     "primary": {

Manually adding the missing "data." back to the address in the state file fixes the issue, and subsequent plan/apply operations succeed.

[apparentlymart]

Hi @nickrw. Thanks for reporting this!

The state commands and data source code were being worked on concurrently, so my bet is that this was missed because the extra attribute we added to distinguish data resources from managed resources wasn't around when the state mv code was written.

In some quick spelunking in the code I wasn't able to track it down exactly, but this problem suggests that somewhere inside the module-moving code we're copying and rewriting ResourceAddress objects and failing to preserve the Mode attribute, which is what records whether a resource is a managed or a data resource.

[apparentlymart]

Aha... I think I found it:
https://github.com/hashicorp/terraform/blob/0e6e206465260723e2ca7fce6ed8eed5eda8495f/terraform/state_add.go#L113

I don't have a Go compiler handy to test right now but I expect adding an additional line to copy the Mode attribute here will fix it.

[hamstah]

Hi,
I've just had this happen with terraform 0.9.2 with terraform state mv

I used terraform state mv for all the resources in a module (I renamed the module and didn't want to delete/recreate all the resources in it).

When running plan I get

Error refreshing state: 1 error(s) occurred:
* module.client_account.aws_caller_identity.current: aws_caller_identity.current: unknown resource type: aws_caller_identity

I'm using the new state backend

Here is the relevant part of terraform state pull

           "resources": {
                "aws_caller_identity.current": {
                    "type": "aws_caller_identity",
                    "depends_on": [],
                    "primary": {
                        "id": "2017-03-31 13:51:02.764582214 +0000 UTC",
                        "attributes": {
                            "account_id": "xxxxx",
                            "id": "2017-03-31 13:51:02.764582214 +0000 UTC"
                        },
                        "meta": {},
                        "tainted": false
                    },
                    "deposed": [],
                    "provider": ""
                },
...

The data has been dropped from the resource as @nickrw reported

Using terraform state show works fine

terraform state show module.client_account.aws_caller_identity.current
id         = 2017-03-31 13:51:02.764582214 +0000 UTC
account_id = xxxxx

[hamstah]

Fixed the issue with

• terraform state pull > state.json
• Edit the file and add the data. prefix to the resource key
• terraform state push state.json

[exolab]

Having the exact same problem with aws_availability_zones and did exactly what hamstah did to fix it.

[themalkolm]

Same with aws_iam_policy_document

[frimik]

Happened with every single data source in a state mv ... plenty of prefixing had to be done.
```Terraform v0.11.3

• provider.aws v1.10.0```

[apparentlymart]

Hi all,

Given that Terraform has changed a lot since this issue was opened and addressed, it's likely that new problems have different causes even if the symptoms look similar. For those of you who are seeing what appears to be a regression of this issue, it'd be great if you could open a fresh issue and fill out the requested information in the issue template so we can have the best chance of understanding what's going on.

Thanks!

[2rs2ts]

@apparentlymart new issue: #18978

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.