terraform import fails when modules use for_each

[favoretti]

Given this module code:

module "labels" {
  for_each = local.regions
  source   = "git::[email protected]:v3/xxx/common/terraform-null-label?ref=latest"
  context  = local.az_network[local.subscription_name].labels_context
  location = each.value
  names    = local.environments
  tags = {
    eai            = "xxx"
    terraform-plan = "xxx/xxx-terraform/nonprod"
  }
}

When trying to import a resource we observe the following behavior:

terraform import 'module.xxx_output_sa["primary"].azurerm_storage_container.this["xxx-output"]' 'https://xxx.blob.core.windows.net/xxx-output'
Acquiring state lock. This may take a few moments...
module.xxx_output_sa["primary"].azurerm_storage_container.this["xxx-output"]: Importing from ID "https://xxx.blob.core.windows.net/xxx-output"...
module.xxx_output_sa["primary"].azurerm_storage_container.this["xxx-output"]: Import prepared!
  Prepared azurerm_storage_container for import
module.xxxr_output_sa["primary"].azurerm_storage_container.this["xxx-output"]: Refreshing state... [id=https://xxx.blob.core.windows.net/xxx-output]

Error: Invalid index

  on atlas-integration.tf line 4, in module "atlas_integration_spn":
   4:   labels_context = module.labels["primary"].context
    |----------------
    | module.labels is object with no attributes

The given key does not identify an element in this collection value.


Error: Invalid index

  on databricks.tf line 4, in module "databricks":
   4:   labels_context             = module.labels[each.key].context
    |----------------
    | each.key is "primary"
    | module.labels is object with no attributes

The given key does not identify an element in this collection value.


Error: Invalid index

  on databricks.tf line 21, in module "databricks_secret_scope":
  21:   labels_context      = module.labels[each.key].context
    |----------------
    | each.key is "primary"
    | module.labels is object with no attributes

The given key does not identify an element in this collection value.

Terraform version:

Terraform v0.13.2

[alisdair]

Thanks for reporting this issue, @favoretti!

Unfortunately there isn't enough detail here to understand what the cause of the issue is. Can you provide a smaller but complete configuration which has the same effect, so that we can reproduce it? If possible please use resources from the null or random providers, to make reproduction simpler. Thanks!

[favoretti]

@alisdair

Hmm, not sure how to make this easy, but let's try this...

Folder structure:

(⎈ |eastus2-fxs-oscar-np-fxc:build-agents)➜  ~/terraform-tests $ find .
.
./test.tf
./modules
./modules/databricks
./modules/databricks/null.tf
./modules/labels
./modules/labels/null.tf

test.tf:

provider "azurerm" {}

locals {

  regions = {
    primary = "eastus2"
    #secondary = "centralus"
  }
}

module "labels" {
  for_each = local.regions
  source   = "./modules/labels"
  context  = {}
  location = each.value
  names    = ["foo", "bar"]
}

module "databricks" {
  for_each                   = local.regions
  source                     = "./modules/databricks"
  labels_context             = module.labels[each.key].context
  resource_group_name        = "fake"
  log_analytics_workspace_id = "fake"
}

modules/labels/null.tf:

variable "context" {
  type    = map
  default = {}
}

variable "location" {
  type    = string
  default = "eastus2"
}

variable "names" {
  type    = list
  default = []
}

resource "null_resource" "labels" {
}

output "context" {
  value = var.context
}

modules/databricks/null.tf:

variable "labels_context" {
  type    = map
  default = {}
}

variable "resource_group_name" {
  type    = string
  default = "fake"
}

variable "log_analytics_workspace_id" {
  type    = string
  default = "fake"
}

resource "null_resource" "labels" {
}

Now after initialization I run the following command:

(⎈ |eastus2-fxs-oscar-np-fxc:build-agents)➜  ~/terraform-tests $ terraform import -allow-missing-config azurerm_resource_group.bla /subscriptions/340958304958039458/resourceGroups/bla
azurerm_resource_group.bla: Importing from ID "/subscriptions/340958304958039458/resourceGroups/bla"...
azurerm_resource_group.bla: Import prepared!
  Prepared azurerm_resource_group for import
azurerm_resource_group.bla: Refreshing state... [id=/subscriptions/340958304958039458/resourceGroups/bla]

Error: Cannot import non-existent remote object

While attempting to import an existing object to azurerm_resource_group.bla,
the provider detected that no object exists with the given id. Only
pre-existing objects can be imported; check that the id is correct and that it
is associated with the provider's configured region or endpoint, or use
"terraform apply" to create a new remote object for this resource.


Error: Invalid index

  on /Users/vlazarenko/terraform-tests/test.tf line 22, in module "databricks":
  22:   labels_context             = module.labels[each.key].context
    |----------------
    | each.key is "primary"
    | module.labels is object with no attributes

The given key does not identify an element in this collection value.

terraform console does see the module.labels contents:

(⎈ |eastus2-fxs-oscar-np-fxc:build-agents)➜  ~/terraform-tests $ terraform console
> module.labels
{
  "primary" = {
    "context" = {}
  }
}

Ignore the fact that resource doesn't exist, it's about the following error, which renders the whole import failing, even if resource exists. Does this help?

[joegajeckyj]

I am having the same issues, let me add my output:

Terraform Version

Terraform v0.13.3

variables,tf

variable list_resource_groups {
type = set(string)
default = []
}

tfvars
list_resource_groups = [
"perimeter",
"hj",
"dnszone",
"mgmt",
"db"
]

main.tf

module resourcegroups {
source = "git::[email protected]:xxxxxx-terraform-registry.git//Azure/Modules/resourcegroup"
for_each = var.list_resource_groups
name = "${var.customer_prefix}-${each.key}-${var.environment}"
location = var.location
environment = var.environment
}

module keyvault {
source = "git::[email protected]:xxxxxx-terraform-registry.git//Azure/Modules/keyvault"
app_name = var.app_name
location = var.location
resource_group_name = module.resourcegroups["mgmt"].resource_group.name
tenant_id = var.tenant_id
}

module storage_diag {
source = "git::[email protected]:xxxxxx-terraform-registry.git//Azure/Modules/storage"
storage_name = "di" # must be lowercase
storage_account_count = 1
app_name = var.app_name
location = var.location
resource_group_name = module.resourcegroups["mgmt"].resource_group.name
allowed_ips = null
allowed_subnets = [module.subnets_perimeter["bastion-internal"].subnet, module.subnets_management["restricted"].subnet, module.subnets_hj["trusted"].subnet, module.subnets_db["restricted"].subnet]
}

module
main.tf
resource azurerm_resource_group resource_group {
name = var.name
location = var.location
tags = {
environment = var.environment
}
}

output.tf
output resource_group {
value = "${azurerm_resource_group.resource_group}"
description = "Output list object from the resource_group resource"
}

Output
terraform import "module.vnetgateway_expressroute.azurerm_virtual_network_gateway.vnet_virtualgateway" "/subscriptions/XXXXXXX/resourceGroups/X-perimeter-prod/providers/Microsoft.Network/virtualNetworks/X-perimeter-prod-vnet"
module.vnetgateway_expressroute.azurerm_virtual_network_gateway.vnet_virtualgateway: Importing from ID "/subscriptions/XXXXXXX/resourceGroups/X-perimeter-prod/providers/Microsoft.Network/virtualNetworks/X-perimeter-prod-vnet"...
module.vnetgateway_expressroute.azurerm_virtual_network_gateway.vnet_virtualgateway: Import prepared!
Prepared azurerm_virtual_network_gateway for import
module.vnetgateway_expressroute.azurerm_virtual_network_gateway.vnet_virtualgateway: Refreshing state... [id=/subscriptions/XXXXXXX/resourceGroups/X-perimeter-prod/providers/Microsoft.Network/virtualNetworks/X-perimeter-prod-vnet]

Error: Invalid index

on \Azure\Environments\Prod\main.tf line 46, in module "keyvault":
46: resource_group_name = module.resourcegroups["mgmt"].resource_group.name
|----------------
| module.resourcegroups is object with no attributes

The given key does not identify an element in this collection value.

Error: Invalid index

on \Azure\Environments\Prod\main.tf line 56, in module "storage_diag":
56: resource_group_name = module.resourcegroups["mgmt"].resource_group.name
|----------------
| module.resourcegroups is object with no attributes

The given key does not identify an element in this collection value.

Expected Behavior
module.vnetgateway_expressroute.azurerm_virtual_network_gateway.vnet_virtualgateway: Import prepared!
Prepared azurerm_virtual_network_gateway for import
module.vnetgateway_expressroute.azurerm_virtual_network_gateway.vnet_virtualgateway: Refreshing state... [id=/subscriptions/xxxxxxxx/resourceGroups/excelicareuk-perimeter-prod/providers/Microsoft.Network/virtualNetworks/excelicareuk-perimeter-prod-vnet]

Actual Behavior
on each subsequent use of the resourcegroups module the import refresh fails with apparently no attributes in the module

Error: Invalid index

Steps to Reproduce

terraform import "module.vnetgateway_expressroute.azurerm_virtual_network_gateway.vnet_virtualgateway"
Additional Context

terraform apply works, but import doesn't!

this happens on any import, the failure is always on the module which contains the for_each.

[alisdair]

Thanks, that is very helpful! I have a slightly simpler reproduction case in our issue reproductions:

main.tf:

locals {
  xs = toset(["foo"])
}

module "a" {
  for_each = local.xs
  source   = "./a"
}

module "b" {
  for_each = local.xs
  source   = "./b"
  y = module.a[each.key].y
}

a/main.tf:

output "y" {
  value = "bar"
}

b/main.tf:

variable "y" {
  type = string
}

resource "random_pet" "pet" {
  prefix = var.y
}

Repro:

• terraform init
• (Optionally terraform apply but it doesn't change the behaviour)
• terraform import -allow-missing-config random_string.test test

Result:

$ terraform-0.13.3 import -allow-missing-config random_string.test test
random_string.test: Importing from ID "test"...
random_string.test: Import prepared!
  Prepared random_string for import
random_string.test: Refreshing state... [id=test]

Error: Invalid index

  on /Users/alisdair/repro/26258/main.tf line 13, in module "b":
  13:   y = module.a[each.key].y
    |----------------
    | each.key is "foo"
    | module.a is object with no attributes

The given key does not identify an element in this collection value.

This worked in 0.13.0, and therefore may have been broken by #25890, but I'm not sure how to fix it. Marking as confirmed for now.

[favoretti]

@alisdair Any news on this? Import is completely unusable for us at this moment and this is unfortunately a quite often used feature with Azure timeouts and such...

[favoretti]

BTW, I reverted #25890 locally and it indeed fixed the problem, so it's definitely that change that caused this behavior.

[alisdair]

@favoretti I don't have any further news on this at this time, sorry. I have a failing test case but no ideas about how to fix this bug without breaking the bug that was fixed in #25890.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.