Terraform-provider-azurerm: Error: Invalid index. The given key does not identify an element in this collection value.

[andrewCluey]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.20

• provider.azurerm v1.42.0

Affected Resource(s)

• azurerm_virtual_machine
• azurerm_role_assignment

Terraform Configuration Files

Initial terraform configuration (Other resources not shown here including Network interface etc). This deploys perfectly.

resource "azurerm_virtual_machine" "virtual_machine" {
  count                                 = 2
  name                                 = "${format("vm-%03d", count.index + 1)}"
  location                             = var.location
  resource_group_name    = var.rg
  network_interface_ids     = [element(azurerm_network_interface.df_network_interface.*.id, count.index)]
  vm_size                              = "Standard_B2s"
  license_type                  = "Windows_Server"

Modified TF configuration (adding 'identity' block and new resource for role assignment to the VM).

resource "azurerm_virtual_machine" "virtual_machine" {
  count                                 = var.VMCount
  name                                 = "${format("vm-%03d", count.index + 1)}"
  location                             = var.location
  resource_group_name    = var.rg
  network_interface_ids     = [element(azurerm_network_interface.df_network_interface.*.id, count.index)]
  vm_size                              = "Standard_B2s"
  license_type                  = "Windows_Server"
  identity {
    type = "SystemAssigned"
  }

```hcl
resource "azurerm_role_assignment" "df_contributor" {
count = local.create_data_factory ? var.VMCount : 0
scope = azurerm_data_factory.factory.id
role_definition_name = "Reader"
principal_id = "${azurerm_virtual_machine.virtual_machine[count.index].identity.0.principal_id}"
depends_on = ["azurerm_virtual_machine.virtual_machine"]
}

### Debug Output
<script src="https://gist.github.com/andrewCluey/1ed407e821af6fde065141c25df71b08.js"></script>
<!---
Please provide a link to a GitHub Gist containing the complete debug output. Please do NOT paste the debug output in the issue; just paste a link to the Gist.

To obtain the debug output, see the [Terraform documentation on debugging](https://www.terraform.io/docs/internals/debugging.html).
--->

### Panic Output

<!--- If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the `crash.log`. --->

### Expected Behavior

The identity block for the VM should first be applied, then the Role assignment should apply. 

### Actual Behavior
On both plan and apply the error message (below) is returned...

```hcl
Error: Invalid index

  on main.tf line 207, in resource "azurerm_role_assignment" "df_contributor":
 207:   principal_id         = "${azurerm_virtual_machine.df_virtual_machine[count.index].identity.0.principal_id}"
    |----------------
    | azurerm_virtual_machine.virtual_machine is tuple with 2 elements
    | count.index is 0

The given key does not identify an element in this collection value.

Seems to be that the 'identity' block for the VM is not applied before the role assignment resource is applied.

If I comment out the Role_Assignment resource, ad re-apply, then the VM is modified in place with the 'identity' block. If i then re-enable the 'role_assignment' resource and re-apply again, it's all fine.

This seems to suggest a issue in the ordering of dependencies between the 'identity' block in the VM_resource and the role_assignment resource.

Steps to Reproduce

1. Created a terraform config for a new Azure VM resource with count attribute set. Using latest Terraform version and Azure provider.
2. terraform apply
3. Add an identity block into the azurerm_virtual_machine resource for 'systemassigned'.
4. add a new resource into the terraform config to assign a role to the VM (azurerm_role_assignment). Using count and count.index to identity the correct principal IDs.
5. terraform apply.
6. Error: Invalid index

Important Factoids

References

• #0000

[tombuildsstuff]

hi @andrewCluey

Thanks for opening this issue.

Taking a look into this it appears that this bug is due to the field principal_id not being returned from the Azure API - which is why this field is unset.

Whilst traditionally we'd look into fixing this so that the principal_id was populated with an empty value - the azurerm_virtual_machine (and azurerm_virtual_machine_scale_set) resources are being superseded in the upcoming version 2.0 of the Azure Provider (more details can be found in #2807) and thus are in a feature-frozen state at this time.

A Beta containing these new resources is available in version 1.43 of the Azure Provider and as such I'm going to suggest taking a look to see if the upcoming azurerm_windows_virtual_machine resource instead would fit your needs here instead (which I can confirm contains this behaviour so the principal_id field is always set)?

Thanks!

[andrewCluey]

Thanks Tom, although I've just tested this and I'm getting the same error.

I've created a VM using the new resource (azurerm_windows_virtual_machine). Resource I used to create the VMs is below.

• Terraform v0.12.20
• provider.azurerm v1.43.0

Initial resource to create 2 VMs.

resource "azurerm_windows_virtual_machine" "df_virtual_machine" {
    count               = 2
    name                = "${format("vm-%03d", count.index + 1)}"
    resource_group_name = data.azurerm_resource_group.PlatformRG.name
    location            = data.azurerm_resource_group.PlatformRG.location
    size                = "Standard_B2s"
    admin_username      = "aduser"
    admin_password      = "Passw0rd*" 

    network_interface_ids = [
      element(azurerm_network_interface.df_network_interface.*.id, count.index)
      ]

I then added the identity block to the VM resource

  resource "azurerm_windows_virtual_machine" "df_virtual_machine" {
    count               = 2
    name                = "${format("vm-%03d", count.index + 1)}"
    resource_group_name = data.azurerm_resource_group.PlatformRG.name
    location            = data.azurerm_resource_group.PlatformRG.location
    size                = "Standard_B2s"
    admin_username      = "aclure"
    admin_password      = "Passw0rd*" 
    network_interface_ids = [
      element(azurerm_network_interface.df_network_interface.*.id, count.index)
      ]
  identity {
    type = "SystemAssigned"
  }

And also a 'role_assignment' resource.

resource "azurerm_role_assignment" "df_reader" {
  count                = local.create_data_factory ? var.VMCount : 0
  scope                = azurerm_data_factory.factory.id
  role_definition_name = "Reader"
  principal_id         = "${azurerm_windows_virtual_machine.df_virtual_machine[count.index].identity.0.principal_id}"
  depends_on           = ["azurerm_windows_virtual_machine.df_virtual_machine"]
  }

When I apply or 'plan' these modifications, I get the error again (Error: Invalid index)

As before, if I comment out the role_assignment, and just leave in the 'new' 'identity' block on the VM resource, this modifies the VM in place.

Having just applied the config with the VM 'identity', re-enabling the role_assignment resource and applying the config works fine and I get the desired result.

If the VM resource and role_assignment is completely new, then it's not a problem and works first time. Seems to be when modifying an existing VM with the identity block (and getting the principal_id output).

Thanks

[gilbert-hsu]

same issue with

azurerm_virtual_machine
azurerm_virtual_machine_extension

resource "azurerm_virtual_machine_extension" "example" {
  count                = 2
  name                 = example
  virtual_machine_id   = azurerm_virtual_machine.vm-example[count.index].id
  publisher            = "Microsoft.Azure.Diagnostics"
  type                 = "LinuxDiagnostic"
  type_handler_version = "3.0"
  settings             = data.template_file.vm-example-setting[count.index].rendered
  protected_settings   = data.template_file.vm-example-protected-setrting.rendered
  }
}

Error: Invalid index

  on vm-example.tf line 107, in resource "azurerm_virtual_machine_extension" "vm-example":
 107:   virtual_machine_id   = azurerm_virtual_machine.vm-example[count.index].id
    |----------------
    | azurerm_virtual_machine.vm-exaple is empty tuple
    | count.index is 0

The given key does not identify an element in this collection value.

Error: Invalid index

  on vm-example.tf line 107, in resource "azurerm_virtual_machine_extension" "vm-example":
 107:   virtual_machine_id   = azurerm_virtual_machine.vm-example[count.index].id
    |----------------
    | azurerm_virtual_machine.vm-exaple is empty tuple
    | count.index is 1

The given key does not identify an element in this collection value.

[jeanpaulsmit]

Hi @tombuildsstuff this "Taking a look into this it appears that this bug is due to the field principal_id not being returned from the Azure API - which is why this field is unset." also seems to be the case for azurerm_function_app where azurerm_function_app.fa.identity[0].principal_id breaks when being used.

[A30001546]

@here check if this post is useful
https://stackoverflow.com/questions/59875966/output-for-principal-id-for-multiple-azure-app-services-through-terraform

[jgspratt]

I upgraded from Terraform 12.10 to 12.24 and got a very similar issue with an AWS stack.

 Error: Invalid index
   on alb.tf line 32, in module "redacted":
   32:   target_redacted_list           = [aws_instance.redacted[0].id, aws_instance.redacted[1].id]
     |----------------
     | aws_instance.redacted is empty tuple
 The given key does not identify an element in this collection value.
 Error: Invalid index
   on alb.tf line 32, in module "redacted":
   32:   target_redacted_list           = [aws_instance.redacted[0].id, aws_instance.redacted[1].id]
     |----------------
     | aws_instance.redacted is empty tuple
 The given key does not identify an element in this collection value.
...

Downgrading to 12.10 fixed the issue. AWS Provider version 2.55.

[eedwards36]

Downgrading to Terraform 12.10 from 12.24 also resolved my issue with Azurerm version 2.3.0 of trying to output the ADF principal_id:
Error: Unsupported attribute in output "adf_principal_id": value = "${azurerm_data_factory.foo.identity.0.principal_id}" azurerm_data_factory.foo is empty tuple. This value does not have any attributes.

[elastic2ls-awiechert]

Hi,

Downgrading to Terraform 12.10 from 12.24 also resolved my issue with Azurerm version 2.3.0 of trying to output the ADF principal_id:
Error: Unsupported attribute in output "adf_principal_id": value = "${azurerm_data_factory.foo.identity.0.principal_id}" azurerm_data_factory.foo is empty tuple. This value does not have any attributes.

Hi, just downloaded older bin. But now I can't use the state I have issues with.

Error refreshing state: state snapshot was created by Terraform v0.12.24, which is newer than current v0.12.10; upgrade to Terraform v0.12.24 or greater to work with this state

That's realy annoying I even can't destroy the resources.

[jgspratt]

You can downgrade by replacing your binary with an older one which you can
download from https://releases.hashicorp.com/terraform/

On Mon, May 4, 2020, 06:05 elastic2ls-awiechert [e-mail]
wrote:

Hi,

Downgrading to Terraform 12.10 from 12.24 also resolved my issue with
Azurerm version 2.3.0 of trying to output the ADF principal_id:
Error: Unsupported attribute in output "adf_principal_id": value =
"${azurerm_data_factory.foo.identity.0.principal_id}"
azurerm_data_factory.foo is empty tuple. This value does not have any
attributes.

Hi, how to do that downgrade? I run into a similar issue.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/terraform-providers/terraform-provider-azurerm/issues/5675#issuecomment-623374416,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI63K7XRUMLC2BPYXU6JU3RP2HM3ANCNFSM4KS6XVUQ
.

[hotdawg789]

Same behavior when going from 12.07 to 12.24. This is a PITA right now and we also cant destroy resources. we could roll back version, but upgraded because we needed new feature on 12.24.

Error: Invalid index

  on resources.tf line 448, in resource "aws_volume_attachment" "webber01_80":
 448:   instance_id = aws_instance.webber[0].id
    |----------------
    | aws_instance.webber is empty tuple

The given key does not identify an element in this collection value.

[davegallant]

I believe this is related to: https://github.com/hashicorp/terraform/pull/22846

[mrlesmithjr]

I have just experienced this for the first time today after several weeks of no issues on 0.12.28 using AzureRM modules.

Error: Invalid index

  on ../../modules/services/loadbalancers.tf line 44, in resource "azurerm_network_interface_backend_address_pool_association" "wordpress_db":
  44:   network_interface_id    = azurerm_network_interface.wordpress_db[count.index].id
    |----------------
    | azurerm_network_interface.wordpress_db is tuple with 2 elements
    | count.index is 2

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../../modules/services/vms.tf line 79, in resource "azurerm_linux_virtual_machine" "wordpress_db":
  79:   network_interface_ids           = [azurerm_network_interface.wordpress_db[count.index].id]
    |----------------
    | azurerm_network_interface.wordpress_db is tuple with 2 elements
    | count.index is 2

The given key does not identify an element in this collection value.

[taylorturner]

Seeing the same issue with AWS...

resource "aws_vpc" "this" {
  count = var.create_vpc ? 1 : 0

  cidr_block = var.cidr_block
  enable_dns_hostnames = true

  tags = merge(
    var.custom_tags,
    {
      Name                                        = "${var.name}-vpc"
      "kubernetes.io/cluster/${var.name}" = "shared"
    }
  )
}

output "vpc_id" {
  description = "The ID of the VPC"
  value = aws_vpc.this[0].id
}

terraform apply errors out

Error: Invalid index

  on .terraform/modules/vpc/outputs.tf line 13, in output "vpc_id":
  13:   value = aws_vpc.this[0].id
    |----------------
    | aws_vpc.this is empty tuple

The given key does not identify an element in this collection value.

Current Terraform version looks like it's out of date so I'll try updating.

❯ terraform --version
Terraform v0.12.25
+ provider.aws v2.70.0
+ provider.helm v1.0.0
+ provider.kubernetes v1.11.3
+ provider.local v1.4.0
+ provider.null v2.1.2
+ provider.random v2.3.0
+ provider.template v2.1.2

Your version of Terraform is out of date! The latest version
is 0.12.28. You can update by downloading from https://www.terraform.io/downloads.html

[tombuildsstuff]

👋

Based on the information above it appears that this is an issue with Terraform Core rather than specific to Azure, as such I'm going to close this for the moment - but would you mind opening an issue on the Terraform Core repository so that someone from the Core team can take a look into this?

Thanks!

[jgspratt]

Just so everyone gets the notification, taylorturner was kind enough to migrate this issue for us for Terraform Core: https://github.com/hashicorp/terraform/issues/25578 . Please subscribe to it/vote for it/etc. as appropriate for your situation.

Thank you, taylorturner!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!