Terraform: Unable to move resource to new module

[sudoforge]

Terraform Version

Terraform v0.12.0-rc1
+ provider.github v2.1.0

Terraform Configuration Files

provider "github" {
  organization = "ORG_NAME_HERE"
  version      = "~> 2.1.0"
}

resource "github_team" "foo" {
  name    = "Some Team Name"
  privacy = "closed"
}

module "foo" {
  source      = "PATH_TO_MODULE"
  name        = "Some Team Name"
}

A reference module can be as simple as:

variable "name" {
  type = string
  description = "The name of the team"
}

resource "github_team" "this" {
  name = var.name
}

Debug Output

2019/05/16 18:54:44 [INFO] Terraform version: 0.12.0 rc1 
2019/05/16 18:54:44 [INFO] Go runtime version: go1.12.1
2019/05/16 18:54:44 [INFO] CLI args: []string{"/Users/username/.local/bin/tf12rc1", "state", "mv", "github_team.foo", "module.foo.github_team.this"}
2019/05/16 18:54:44 [DEBUG] Attempting to open CLI config file: /Users/username/.terraformrc
2019/05/16 18:54:44 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/05/16 18:54:44 [INFO] CLI command args: []string{"state", "mv", "github_team.foo", "module.foo.github_team.this"}
2019/05/16 18:54:44 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2019/05/16 18:54:44 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2019/05/16 18:54:44 [DEBUG] New state was assigned lineage "2f78a26c-330c-43e7-9f71-d82486f48d23"
2019/05/16 18:54:44 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2019/05/16 18:54:44 [TRACE] Meta.Backend: instantiated backend of type <nil>
2019/05/16 18:54:44 [DEBUG] checking for provider in "."
2019/05/16 18:54:44 [DEBUG] checking for provider in "/Users/username/.local/bin"
2019/05/16 18:54:44 [DEBUG] checking for provider in ".terraform/plugins/darwin_amd64"
2019/05/16 18:54:44 [DEBUG] found provider "terraform-provider-github_v2.1.0_x4"
2019/05/16 18:54:44 [DEBUG] found valid plugin: "github", "2.1.0", "/private/tmp/foo/.terraform/plugins/darwin_amd64/terraform-provider-github_v2.1.0_x4"
2019/05/16 18:54:44 [DEBUG] checking for provisioner in "."
2019/05/16 18:54:44 [DEBUG] checking for provisioner in "/Users/username/.local/bin"
2019/05/16 18:54:44 [DEBUG] checking for provisioner in ".terraform/plugins/darwin_amd64"
2019/05/16 18:54:44 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2019/05/16 18:54:44 [TRACE] backend/local: CLI option -backup is overriding state backup path to -
2019/05/16 18:54:44 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at 
2019/05/16 18:54:44 [DEBUG] checking for provider in "."
2019/05/16 18:54:44 [DEBUG] checking for provider in "/Users/username/.local/bin"
2019/05/16 18:54:44 [DEBUG] checking for provider in ".terraform/plugins/darwin_amd64"
2019/05/16 18:54:44 [DEBUG] found provider "terraform-provider-github_v2.1.0_x4"
2019/05/16 18:54:44 [DEBUG] found valid plugin: "github", "2.1.0", "/private/tmp/foo/.terraform/plugins/darwin_amd64/terraform-provider-github_v2.1.0_x4"
2019/05/16 18:54:44 [DEBUG] checking for provisioner in "."
2019/05/16 18:54:44 [DEBUG] checking for provisioner in "/Users/username/.local/bin"
2019/05/16 18:54:44 [DEBUG] checking for provisioner in ".terraform/plugins/darwin_amd64"
2019/05/16 18:54:44 [TRACE] backend/local: CLI option -backup is overriding state backup path to -
2019/05/16 18:54:44 [TRACE] statemgr.Filesystem: preparing to manage state snapshots at terraform.tfstate
2019/05/16 18:54:44 [TRACE] statemgr.Filesystem: existing snapshot has lineage "a022c743-4cae-c580-101f-35d46d150be5" serial 1
2019/05/16 18:54:44 [TRACE] statemgr.Filesystem: locking terraform.tfstate using fcntl flock
2019/05/16 18:54:44 [TRACE] statemgr.Filesystem: writing lock metadata to .terraform.tfstate.lock.info
2019/05/16 18:54:44 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate
2019/05/16 18:54:44 [TRACE] statemgr.Filesystem: read snapshot with lineage "a022c743-4cae-c580-101f-35d46d150be5" serial 1
2019/05/16 18:54:44 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info

2019/05/16 18:54:44 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock
Error: Invalid target address

Cannot move to module.foo.github_team.this: module.foo does not exist in the
current state.

Expected Behavior

Terraform 0.11.13 (and below) would move the resource to the module, assuming you provided the full path to the resource contained within the module, e.g. module.foo.whatever_resource.resource_name.

Actual Behavior

Terraform 0.12.0-rc1 errors out, stating that module.foo doesn't exist in the current state. And that's true, it's not currently being tracked.

Steps to Reproduce

1. terraform-12-rc1 init
2. terraform-12-rc1 import github_team.foo SOME_TEAM_ID
3. terraform-12-rc1 state mv github_team.foo module.foo.github_team.this

Additional Context

N/A

References

N/A (I searched and didn't find anything; feel free to edit if reference issues are found)

[sudoforge]

It should be noted that current documentation for terraform state mv suggests that this should be possible:

• Rename a resource (test case covered above)
• Move a resource into a module (test case: example files above, command below)

$ terraform-12-rc1 state mv github_team.foo module.foo
Error: Invalid target address

Cannot move github_team.all_developers to <invalid>..: the target must also be
a whole resource.

[hwstovall]

We've had some success working around this with the following.

Let's say we have an existing resource identified by ABC123 that was created by Terraform as a resource called module.old.resource_type.foo.

# Import the component created by the old module into the new module.
# This create entries for the new module in the state.
terraform import module.new.resource_type.foo ABC123

# Delete the old module's state entry for that component.
terraform state rm module.old.resource_type.foo

From here on out, the new module will exist in state and terraform state mv should work as expected.

While this has worked for us, I want to make sure it is clear that this could potentially lead to state inconsistencies if not done correctly. Use this approach with caution.

[sudoforge]

To be clear, @hwstovall's comment is not a method for moving modules and resources, but a full deletion of the old and import of the new. This is the obvious way to accomplish the end goal, but requires importing the resource(s) that are being "moved", which means you need to know something about the resource -- the format of the import string, and any details such as the resource name, ID, or other such property.

This issue was created to highlight that the new version of the command line client was unable to move A to B, which does _not_ require any knowledge about the resource, and does _not_ require re-importing the resource.

[pauldraper]

Also happens with final 0.12 versions. (I'm using 0.12.1.)

Can't figure out a good way around this; @hwstovall's (incomplete) workaround is the best I can find.

And I have to deal with "resource aws_cloudwatch_log_stream doesn't support import"

[sudoforge]

@pauldraper You could manually modify the state file, but this isn't recommended. If you have already migrated your plans to 0.12, however, that remains your only option for resources which do not support imports.

[bohdanyurov-gl]

Moving tf resource:
module.project.module.service-dbm-ecr.aws_ecr_repository.main ->
module.project.module.service-dbm-ecr.aws_ecr_repository.this[0]

Error: Invalid target address

Cannot move module.project.module.service-dbm-ecr.aws_ecr_repository.main to
<invalid>..: the target must also be a whole resource.

Terraform version 0.12.1

[sudoforge]

@bohdanyurov-gl Thanks for reporting that you're experiencing the same issue!

In the future, a simple 👍 reaction on the original comment is a great way to "me too" an issue.

[pauldraper]

@sudoforge thanks for the tip; modifying the state file manually was indeed the solution I used.

[pauldraper]

Alright, here is can move a resource to a new module in 0.12, with minimal risk: use a dummy resource (e.g. null_resource) in the new module, and -target its creation before moving your actual resources.

Version 1 (initial)

main.tf

module "example" {
  number = 1
  source = "./example"
}

example/main.tf

variable "number" {}

resource "local_file" "file" {
  count = var.number
  content = "content"
  filename = "${path.module}/file-${count.index}.txt"
}

# terraform init
# terraform apply -auto-approve
module.example.local_file.file[0]: Creating...
module.example.local_file.file[0]: Creation complete after 0s [id=040f06fd774092478d450774f5ba30c5da78acc8]

Version 2 (update, and failed mv)

main.tf

module "example" {
  number = 0
  source = "./example"
}

module "example2" {
  number = 1
  source = "./example"
}

example/main.tf

variable "number" {}

resource "local_file" "file" {
  count = var.number
  content = "content"
  filename = "${path.module}/file-${count.index}.txt"
}

# terraform state mv module.example.local_file.file module.example2.local_file.file

Error: Invalid target address

Cannot move to module.example2.local_file.file: module.example2 does not exist
in the current state.

Version 3 (create dummy, and successfully mv)

main.tf

module "example" {
  number = 0
  source = "./example"
}

module "example2" {
  number = 1
  source = "./example"
}

example/main.tf

variable "number" {}

resource "local_file" "file" {
  count = var.number
  content = "content"
  filename = "${path.module}/file-${count.index}.txt"
}

resource "null_resource" "null" {}

# terraform init

# terraform apply -auto-approve -target=module.example2.null_resource.null
module.example2.null_resource.null: Creating...
module.example2.null_resource.null: Creation complete after 0s [id=885875772923890418]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

# terraform state mv module.example.local_file.file module.example2.local_file.file
Move "module.example.local_file.file" to "module.example2.local_file.file"
Successfully moved 1 object(s).

Version 4 (remove dummy)

main.tf

module "example" {
  number = 0
  source = "./example"
}

module "example2" {
  number = 1
  source = "./example"
}

example/main.tf

variable "number" {}

resource "local_file" "file" {
  count = var.number
  content = "content"
  filename = "${path.module}/file-${count.index}.txt"
}

# terraform apply -auto-approve
module.example2.null_resource.null: Refreshing state... [id=2880189106572971671]
module.example2.local_file.file[0]: Refreshing state... [id=040f06fd774092478d450774f5ba30c5da78acc8]
module.example.null_resource.null: Creating...
module.example.null_resource.null: Creation complete after 0s [id=6086028694931568030]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

---

(I did run into #21529 when I was trying this with more complex configurations, but I didn't record how I got around that.)

[navaati]

Hi.

Nice workaround with the dummy in the module, although it's impractical when you're getting the module from git or something like that.

So here is a way to do the exact same thing without modifying your module source, hacking the statefile instead. Of course the usual warning apply, that could eat your statefile:

terraform state pull |jq --arg module_path module.dali --arg null_id $(uuidgen) '.resources+=[{module:$module_path,mode:"managed",type:"null_resource",name:"dummy",provider:"provider.null",instances:[{schema_version:0,attributes:{id:$null_id,triggers:null},private:"bnVsbA=="}]}]|.serial+=1' |terraform state push -

You can use it by modifying the "module.mymodule" to the path of the actual module you want to move resources to. You need the jq and uuidgen commands.

Then do your move:

terraform state mv resource_type.my_resource module.mymodule.resource_type.my_resource # Now that works
terraform init # to download the null provider, that's needed at this point
terraform apply # That will delete the null resource, since it's not present in the code

Now, that is still a horrible hack :).

Dear Hashicorp, this bug is breaking basically all module-related refactoring possibilities, I really think it needs love fast !

[apottere]

If you use the workaround provided by @navaati, you can use the following command to remove the dummy null_resource after moving your resources into the module. In my case I wasn't able to plan or apply because I didn't have a provider for the dummy null_resource.

terraform state rm module.<name>.null_resource.dummy

[jcarlson]

@jbardin Why was this issue closed? I am still seeing this behavior in Terraform 0.12.5. Either this defect still exists, or the documentation should be updated to indicate that it is not possible.

In my use case, I am trying to refactor a large module into sub-modules. I am trying to do something like:

terraform state mv \
  module.monolith.random_string.password \
  module.monolith.module.credentials.random_string.password

I get the following error:

Error: Invalid target address

Cannot move to
module.monolith.module.credentials.random_string.password:
module.monolith.module.credentials does not exist in the current
state.

From the documentation:
https://www.terraform.io/docs/commands/state/mv.html#example-move-a-resource-into-a-module

The module will be created if it doesn't exist.

[teamterraform]

Hi @jcarlson, the Close message above has a link to the PR that closed the issue, which was merged after 0.12.6. It will be included in the next release.

[jcarlson]

Apologies, I missed that PR link. Thank you!

[sandangel]

hi I'm facing the same issue with terraform 0.12.6

tf state mv aws_iam_role.ecs_cloud_watch_read_role module.monitoring.module.iam.aws_iam_role.ecs_cloud_watch_read_role

Error: Invalid target address

Cannot move to
module.monitoring.module.iam.aws_iam_role.ecs_cloud_watch_read_role:
module.monitoring.module.iam does not exist in the current state.

# the old state here
tf state show aws_iam_role.ecs_cloud_watch_read_role
# aws_iam_role.ecs_cloud_watch_read_role:
resource "aws_iam_role" "ecs_cloud_watch_read_role" {
    arn                   = "arn:aws:iam::229482903727:role/ecsCloudWatchReadRole"
    assume_role_policy    = jsonencode(
        {
            Statement = [
                {
                    Action    = "sts:AssumeRole"
                    Effect    = "Allow"
                    Principal = {
                        Service = "ecs-tasks.amazonaws.com"
                    }
                },
            ]
            Version   = "2012-10-17"
        }
    )
    create_date           = "2019-08-15T08:09:15Z"
    force_detach_policies = false
    id                    = "ecsCloudWatchReadRole"
    max_session_duration  = 3600
    name                  = "ecsCloudWatchReadRole"
    path                  = "/"
    tags                  = {}
    unique_id             = "AROATK3R7HCX65ATUVF7C"
}

tf show tfplan
  - aws_iam_role.ecs_cloud_watch_read_role

  - aws_iam_role_policy.ecs_cloud_watch_read_policy

-/+ module.monitoring.aws_ecs_task_definition.z_grafana_td (new resource required)

  ~ module.monitoring.module.ecs_cluster.aws_launch_template.ecs_cluster_lt

  # I created the new resource here
  + module.monitoring.module.iam.aws_iam_role.ecs_cloud_watch_read_role

  + module.monitoring.module.iam.aws_iam_role.ecs_efs_instance_role

  + module.monitoring.module.iam.aws_iam_role_policy.ecs_cloud_watch_read_policy

  + module.monitoring.module.iam.aws_iam_role_policy.ecs_rexray_efs_policy

  + module.monitoring.module.iam.aws_iam_role_policy.ecs_rexray_volume_policy

  + module.monitoring.module.iam.aws_iam_role_policy_attachment.ecs_for_ec2_role

  + module.monitoring.module.iam.aws_iam_role_policy_attachment.efs_read_only_access

tf --version
Terraform v0.12.6
+ provider.aws v2.24.0
+ provider.cloudflare v1.17.1

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.