Terraform-provider-azurerm: terraform destroy is failed to clean up resources with error "Operation 'startTenantUpdate' is not allowed on VM 'VM-Test' since the VM is marked for deletion"

[surajmuthreja]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.24

• provider.azurerm v2.6.0
• provider.random v2.2.1
• provider.template v2.1.2

Affected Resource(s)

• azurerm_subnet_route_table_association
• azurerm_network_interface
• azurerm_virtual_machine

Terraform Configuration Files

# Configure the Microsoft Azure Provider
provider "azurerm" {
  subscription_id = var.subscription_id
  client_id       = var.client_id
  client_secret   = var.client_secret
  tenant_id       = var.tenant_id
  version         = "~>2.6.0"
  features {}
}

provider "random" {
  version = "~> 2.2"
}

provider "template" {
  version = "~> 2.1"
}

# Create a resource group
resource "azurerm_resource_group" "test_rg" {
  name     = var.resource_group
  location = var.location

  tags = {
    environment = "TestVMDeployment"
  }
}

# Add template to use custom data for VM-1:
data "template_file" "user_data_vm_1" {
  template = file("vm_1.sh")

  vars = {
    hostname_vm = var.hostname_vm
    vm-1_mgmt_ip  = azurerm_network_interface.vm-1_nic_1.private_ip_address
    sshkey       = var.ssh_key
  }
}

# Add template to use custom data for VM-2:
data "template_file" "user_data_vm_2" {
  template = file("vm_2.sh")

  vars = {
    sshkey      = var.ssh_key
    vm-1_mgmt_ip = azurerm_network_interface.vm-1_nic_1.private_ip_address
  }
}

# Add template to use custom data for VM-3:
data "template_file" "user_data_vm_3" {
  template = file("vm_3.sh")

  vars = {
    vm-2_mgmt_ip  = azurerm_network_interface.vm-3_nic_1.private_ip_address
    hostname_vm = var.hostname_vm
    vm-1_mgmt_ip  = azurerm_network_interface.vm-1_nic_1.private_ip_address
    sshkey       = var.ssh_key
  }
}

# Create virtual network
resource "azurerm_virtual_network" "TestNetwork" {
  name                = "Test_VPC"
  address_space       = [var.vpc_address_space]
  location            = var.location
  resource_group_name = azurerm_resource_group.test_rg.name

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Route Table
resource "azurerm_route_table" "test_udr" {
  name                = "TestRouteTable"
  location            = var.location
  resource_group_name = azurerm_resource_group.test_rg.name
}

# Add Route in Route Table
resource "azurerm_route" "test_route" {
  name                   = "TestRoute"
  resource_group_name    = azurerm_resource_group.test_rg.name
  route_table_name       = azurerm_route_table.test_udr.name
  address_prefix         = "0.0.0.0/0"
  next_hop_type          = "VirtualAppliance"
  next_hop_in_ip_address = azurerm_network_interface.vm-2_nic_2.private_ip_address
}

# Create Management Subnet
resource "azurerm_subnet" "mgmt_subnet" {
  name                 = "MGMT-NET"
  resource_group_name  = azurerm_resource_group.test_rg.name
  virtual_network_name = azurerm_virtual_network.TestNetwork.name
  address_prefix = cidrsubnet(azurerm_virtual_network.TestNetwork.address_space[0],var.newbits_subnet,1,)
}

# Create Traffic Subnet
resource "azurerm_subnet" "subnet-2" {
  name                 = "Subnet-2"
  resource_group_name  = azurerm_resource_group.test_rg.name
  virtual_network_name = azurerm_virtual_network.TestNetwork.name
  address_prefix = cidrsubnet(azurerm_virtual_network.TestNetwork.address_space[0],var.newbits_subnet,2,)
}

# Create Traffic Subnet for VM-2 and Branch
resource "azurerm_subnet" "subnet-3" {
  name                 = "Subnet-3"
  resource_group_name  = azurerm_resource_group.test_rg.name
  virtual_network_name = azurerm_virtual_network.TestNetwork.name
  address_prefix = cidrsubnet(azurerm_virtual_network.TestNetwork.address_space[0],var.newbits_subnet,3,)
}

# Associate Route Table to Subnet
resource "azurerm_subnet_route_table_association" "subnet_rt_table" {
  subnet_id      = azurerm_subnet.subnet-2.id
  route_table_id = azurerm_route_table.test_udr.id
  depends_on     = [azurerm_subnet.subnet-2, azurerm_route_table.test_udr]
}

# Create Public IP for VM-1
resource "azurerm_public_ip" "ip_vm_1" {
  name                = "PublicIP_VM_1"
  location            = var.location
  resource_group_name = azurerm_resource_group.test_rg.name
  allocation_method   = "Dynamic"

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Public IP for VM-2
resource "azurerm_public_ip" "ip_vm_2" {
  name                = "PublicIP_VM_2"
  location            = var.location
  resource_group_name = azurerm_resource_group.test_rg.name
  allocation_method   = "Dynamic"

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Public IP for VM-2 Interface-2
resource "azurerm_public_ip" "ip_vm_2_int_2" {
  name                = "PublicIP_VM_2_INT_2"
  location            = var.location
  resource_group_name = azurerm_resource_group.test_rg.name
  allocation_method   = "Static"

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Public IP for VM-3
resource "azurerm_public_ip" "ip_vm_3" {
  name                = "PublicIP_VM_3"
  location            = var.location
  resource_group_name = azurerm_resource_group.test_rg.name
  allocation_method   = "Dynamic"

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Network Security Groups and rules for VM-1
resource "azurerm_network_security_group" "nsg_vm_1" {
  name                = "VM-1-NSG"
  location            = var.location
  resource_group_name = azurerm_resource_group.test_rg.name
  security_rule {
    name                       = "Security_Rule_TCP"
    priority                   = 151
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_port_range          = "*"
    destination_port_ranges    = ["22", "443", "8080", "8443"]
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }
  security_rule {
    name                       = "Security_Rule_UDP"
    priority                   = 201
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Udp"
    source_port_range          = "*"
    destination_port_ranges    = ["12345"]
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }
  security_rule {
    name                       = "Security_Rule_Outbound"
    priority                   = 251
    direction                  = "Outbound"
    access                     = "Allow"
    protocol                   = "*"
    source_port_range          = "*"
    destination_port_range     = "*"
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Network Security Groups and rules for VM-2
resource "azurerm_network_security_group" "nsg_vm_2" {
  name                = "VM-2-NSG"
  location            = var.location
  resource_group_name = azurerm_resource_group.test_rg.name
  security_rule {
    name                       = "Security_Rule_TCP"
    priority                   = 151
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_port_range          = "*"
    destination_port_ranges    = ["22", "2022", "1024-1120", "8443"]
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }
  security_rule {
    name                       = "Security_Rule_UDP"
    priority                   = 201
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Udp"
    source_port_range          = "*"
    destination_port_ranges    = ["1234"]
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }
  security_rule {
    name                       = "Security_Rule_Outbound"
    priority                   = 251
    direction                  = "Outbound"
    access                     = "Allow"
    protocol                   = "*"
    source_port_range          = "*"
    destination_port_range     = "*"
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Network Security Groups and rules for VM-3
resource "azurerm_network_security_group" "nsg_vm_3" {
  name                = "VM-3-NSG"
  location            = var.location
  resource_group_name = azurerm_resource_group.test_rg.name
  security_rule {
    name                       = "Security_Rule_TCP"
    priority                   = 151
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_port_range          = "*"
    destination_port_ranges    = ["22", "8080", "8443", "1233"]
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }
  security_rule {
    name                       = "Security_Rule_UDP"
    priority                   = 201
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Udp"
    source_port_range          = "*"
    destination_port_ranges    = ["1234", "123"]
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }
  security_rule {
    name                       = "Security_Rule_Outbound"
    priority                   = 251
    direction                  = "Outbound"
    access                     = "Allow"
    protocol                   = "*"
    source_port_range          = "*"
    destination_port_range     = "*"
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Management network interface for VM-1
resource "azurerm_network_interface" "vm-1_nic_1" {
  name                 = "VM-1_NIC1"
  location             = var.location
  resource_group_name  = azurerm_resource_group.test_rg.name
  enable_ip_forwarding = "true"

  ip_configuration {
    name                          = "VM-1_NIC1_Configuration"
    subnet_id                     = azurerm_subnet.mgmt_subnet.id
    private_ip_address_allocation = "dynamic"
    public_ip_address_id          = azurerm_public_ip.ip_vm_1.id
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Southbound network interface for VM-1
resource "azurerm_network_interface" "vm-1_nic_2" {
  name                 = "VM-1_NIC2"
  location             = var.location
  resource_group_name  = azurerm_resource_group.test_rg.name
  enable_ip_forwarding = "true"

  ip_configuration {
    name                          = "VM-1_NIC2_Configuration"
    subnet_id                     = azurerm_subnet.subnet-2.id
    private_ip_address_allocation = "dynamic"
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Management network interface for VM-2
resource "azurerm_network_interface" "vm-2_nic_1" {
  name                 = "VM-2_NIC1"
  location             = var.location
  resource_group_name  = azurerm_resource_group.test_rg.name
  enable_ip_forwarding = "true"

  ip_configuration {
    name                          = "VM-2_NIC1_Configuration"
    subnet_id                     = azurerm_subnet.mgmt_subnet.id
    private_ip_address_allocation = "dynamic"
    public_ip_address_id          = azurerm_public_ip.ip_vm_2.id
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Northbound network interface for VM-2
resource "azurerm_network_interface" "vm-2_nic_2" {
  name                 = "VM-2_NIC2"
  location             = var.location
  resource_group_name  = azurerm_resource_group.test_rg.name
  enable_ip_forwarding = "true"

  ip_configuration {
    name                          = "VM-2_NIC2_Configuration"
    subnet_id                     = azurerm_subnet.subnet-2.id
    private_ip_address_allocation = "dynamic"
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Southbound network interface for VM-2
resource "azurerm_network_interface" "vm-2_nic_3" {
  name                 = "VM-2_NIC3"
  location             = var.location
  resource_group_name  = azurerm_resource_group.test_rg.name
  enable_ip_forwarding = "true"

  ip_configuration {
    name                          = "VM-2_NIC3_Configuration"
    subnet_id                     = azurerm_subnet.subnet-3.id
    private_ip_address_allocation = "dynamic"
    public_ip_address_id          = azurerm_public_ip.ip_vm_2_int_2.id
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Management network interface for VM-3
resource "azurerm_network_interface" "vm-3_nic_1" {
  name                 = "VM-3_NIC1"
  location             = var.location
  resource_group_name  = azurerm_resource_group.test_rg.name
  enable_ip_forwarding = "true"

  ip_configuration {
    name                          = "VM-3_NIC1_Configuration"
    subnet_id                     = azurerm_subnet.mgmt_subnet.id
    private_ip_address_allocation = "dynamic"
    public_ip_address_id          = azurerm_public_ip.ip_vm_3.id
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create Southbound network interface for VM-3
resource "azurerm_network_interface" "vm-3_nic_2" {
  name                 = "VM-3_NIC2"
  location             = var.location
  resource_group_name  = azurerm_resource_group.test_rg.name
  enable_ip_forwarding = "true"

  ip_configuration {
    name                          = "VM-3_NIC2_Configuration"
    subnet_id                     = azurerm_subnet.subnet-2.id
    private_ip_address_allocation = "dynamic"
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Associate security group to VM-1 Management Network Interface
resource "azurerm_network_interface_security_group_association" "vm_1_mgmt_nic_nsg" {
  network_interface_id      = azurerm_network_interface.vm-1_nic_1.id
  network_security_group_id = azurerm_network_security_group.nsg_vm_1.id
  depends_on                = [azurerm_network_interface.vm-1_nic_1, azurerm_network_security_group.nsg_vm_1]
}

# Associate security group to VM-1 Southbound Network Interface
resource "azurerm_network_interface_security_group_association" "vm_1_sb_nic_nsg" {
  network_interface_id      = azurerm_network_interface.vm-1_nic_2.id
  network_security_group_id = azurerm_network_security_group.nsg_vm_1.id
  depends_on                = [azurerm_network_interface.vm-1_nic_2, azurerm_network_security_group.nsg_vm_1]
}

# Associate security group to VM-2 Management Network Interface
resource "azurerm_network_interface_security_group_association" "vm_2_mgmt_nic_nsg" {
  network_interface_id      = azurerm_network_interface.vm-2_nic_1.id
  network_security_group_id = azurerm_network_security_group.nsg_vm_2.id
  depends_on                = [azurerm_network_interface.vm-2_nic_1, azurerm_network_security_group.nsg_vm_2]
}

# Associate security group to VM-2 Northbound Network Interface
resource "azurerm_network_interface_security_group_association" "vm_2_nb_nic_nsg" {
  network_interface_id      = azurerm_network_interface.vm-2_nic_2.id
  network_security_group_id = azurerm_network_security_group.nsg_vm_2.id
  depends_on                = [azurerm_network_interface.vm-2_nic_2, azurerm_network_security_group.nsg_vm_2]
}

# Associate security group to VM-2 Southbound Network Interface
resource "azurerm_network_interface_security_group_association" "vm_2_sb_nic_nsg" {
  network_interface_id      = azurerm_network_interface.vm-2_nic_3.id
  network_security_group_id = azurerm_network_security_group.nsg_vm_2.id
  depends_on                = [azurerm_network_interface.vm-2_nic_3, azurerm_network_security_group.nsg_vm_2]
}

# Associate security group to VM-3 Management Network Interface
resource "azurerm_network_interface_security_group_association" "vm_3_mgmt_nic_nsg" {
  network_interface_id      = azurerm_network_interface.vm-3_nic_1.id
  network_security_group_id = azurerm_network_security_group.nsg_vm_3.id
  depends_on                = [azurerm_network_interface.vm-3_nic_1, azurerm_network_security_group.nsg_vm_3]
}

# Associate security group to VM-3 Southbound Network Interface
resource "azurerm_network_interface_security_group_association" "vm_3_sb_nic_nsg" {
  network_interface_id      = azurerm_network_interface.vm-3_nic_2.id
  network_security_group_id = azurerm_network_security_group.nsg_vm_3.id
  depends_on                = [azurerm_network_interface.vm-3_nic_2, azurerm_network_security_group.nsg_vm_3]
}

# Generate random text for a unique storage account name
resource "random_id" "randomId" {
  keepers = {
    resource_group = azurerm_resource_group.test_rg.name
  }

  byte_length = 4
}

# Create storage account for boot diagnostics of VM-1 VM
resource "azurerm_storage_account" "storageaccountvm1" {
  name                     = "vm1diag${random_id.randomId.hex}"
  resource_group_name      = azurerm_resource_group.test_rg.name
  location                 = var.location
  account_tier             = "Standard"
  account_replication_type = "LRS"

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create storage account for boot diagnostics of VM-2 VM
resource "azurerm_storage_account" "storageaccountvm2" {
  name                     = "vm2diag${random_id.randomId.hex}"
  resource_group_name      = azurerm_resource_group.test_rg.name
  location                 = var.location
  account_tier             = "Standard"
  account_replication_type = "LRS"

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create storage account for boot diagnostics of VM-3 VM
resource "azurerm_storage_account" "storageaccountvm3" {
  name                     = "vm3diag${random_id.randomId.hex}"
  resource_group_name      = azurerm_resource_group.test_rg.name
  location                 = var.location
  account_tier             = "Standard"
  account_replication_type = "LRS"

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create VM-1 Virtual Machine
resource "azurerm_virtual_machine" "vm-1" {
  name                         = "VM-1"
  location                     = var.location
  resource_group_name          = azurerm_resource_group.test_rg.name
  depends_on                   = [azurerm_network_interface.vm-1_nic_1, azurerm_network_interface.vm-1_nic_2]
  network_interface_ids        = [azurerm_network_interface.vm-1_nic_1.id, azurerm_network_interface.vm-1_nic_2.id]
  primary_network_interface_id = azurerm_network_interface.vm-1_nic_1.id
  vm_size                      = var.VM-1_vm_size

  storage_os_disk {
    name              = "VM-1_OSDisk"
    caching           = "ReadWrite"
    create_option     = "FromImage"
    managed_disk_type = "Standard_LRS"
  }

  storage_image_reference {
    id = var.image_VM-1
  }

  os_profile {
    computer_name  = var.hostname_vm
    admin_username = "devops"
    custom_data    = data.template_file.user_data_vm_1.rendered
  }

  os_profile_linux_config {
    disable_password_authentication = true
    ssh_keys {
      path     = "/home/devops/.ssh/authorized_keys"
      key_data = var.ssh_key
    }
  }

  boot_diagnostics {
    enabled     = "true"
    storage_uri = azurerm_storage_account.storageaccountvm1.primary_blob_endpoint
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create VM-2 Virtual Machine
resource "azurerm_virtual_machine" "vm-2" {
  name                         = "VM-2"
  location                     = var.location
  resource_group_name          = azurerm_resource_group.test_rg.name
  depends_on                   = [azurerm_network_interface.vm-2_nic_1, azurerm_network_interface.vm-2_nic_2, azurerm_network_interface.vm-2_nic_3]
  network_interface_ids        = [azurerm_network_interface.vm-2_nic_1.id, azurerm_network_interface.vm-2_nic_2.id, azurerm_network_interface.vm-2_nic_3.id]
  primary_network_interface_id = azurerm_network_interface.vm-2_nic_1.id
  vm_size                      = var.VM-2_vm_size

  storage_os_disk {
    name              = "VM-2_OSDisk"
    caching           = "ReadWrite"
    create_option     = "FromImage"
    managed_disk_type = "Standard_LRS"
  }

  storage_image_reference {
    id = var.image_VM-2
  }

  os_profile {
    computer_name  = "test-vm-2"
    admin_username = "devops"
    custom_data    = data.template_file.user_data_vm_2.rendered
  }

  os_profile_linux_config {
    disable_password_authentication = true
    ssh_keys {
      path     = "/home/devops/.ssh/authorized_keys"
      key_data = var.ssh_key
    }
  }

  boot_diagnostics {
    enabled     = "true"
    storage_uri = azurerm_storage_account.storageaccountvm2.primary_blob_endpoint
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

# Create VM-3 Virtual Machine
resource "azurerm_virtual_machine" "vm-3" {
  name                         = "VM-3"
  location                     = var.location
  resource_group_name          = azurerm_resource_group.test_rg.name
  depends_on                   = [azurerm_network_interface.vm-3_nic_1, azurerm_network_interface.vm-3_NIC_2]
  network_interface_ids        = [azurerm_network_interface.vm-3_nic_1.id, azurerm_network_interface.vm-3_NIC_2.id]
  primary_network_interface_id = azurerm_network_interface.vm-3_nic_1.id
  vm_size                      = var.VM-3_vm_size

  storage_os_disk {
    name              = "VM-3_OSDisk"
    caching           = "ReadWrite"
    create_option     = "FromImage"
    managed_disk_type = "Standard_LRS"
  }

  storage_image_reference {
    id = var.image_VM-3
  }

  os_profile {
    computer_name  = "VM-3"
    admin_username = "devops"
    custom_data    = data.template_file.user_data_vm_3.rendered
  }

  os_profile_linux_config {
    disable_password_authentication = true
    ssh_keys {
      path     = "/home/devops/.ssh/authorized_keys"
      key_data = var.ssh_key
    }
  }

  boot_diagnostics {
    enabled     = "true"
    storage_uri = azurerm_storage_account.storageaccountvm3.primary_blob_endpoint
  }

  tags = {
    environment = "TestVMDeployment"
  }
}

data "azurerm_public_ip" "vm-1_pub_ip" {
  name                = azurerm_public_ip.ip_vm_1.name
  resource_group_name = azurerm_resource_group.test_rg.name
  depends_on          = [azurerm_virtual_machine.vm-1]
}

data "azurerm_public_ip" "vm-2_pub_ip" {
  name                = azurerm_public_ip.ip_vm_2.name
  resource_group_name = azurerm_resource_group.test_rg.name
  depends_on          = [azurerm_virtual_machine.vm-2]
}

data "azurerm_public_ip" "vm-3_pub_ip" {
  name                = azurerm_public_ip.ip_vm_3.name
  resource_group_name = azurerm_resource_group.test_rg.name
  depends_on          = [azurerm_virtual_machine.vm-3]
}

Debug Output

Panic Output

Expected Behavior

The resources should be successfully created with terraform apply and deleted with terraform destroy.

Actual Behavior

Resource deletion is failing sometime on Route Table association deletion or sometime at NIC card deletion.

Error: Error waiting for update of Network Interface "VM-2_NIC2" (Resource Group "ResourceGroup_Test"): Code="InternalServerError" Message="An error occurred." Details=[]

Sometime getting below error:

Error: Error waiting for update of Network Interface "VM-3_NIC1" (Resource Group "ResourceGroup_Test"): Code="OperationNotAllowed" Message="Operation 'startTenantUpdate' is not allowed on VM 'VM-3' since the VM is marked for deletion. You can only retry the Delete operation (or wait for an ongoing one to complete)." Details=[]

Steps to Reproduce

1. terraform apply
2. terraform destroy

References

• #4330

As mentioned in Issue #4330, raising new issue to track this.
I have even tried the workaround to add the depends_on attribute on almost every resource but still destroy is failing.

This used to be working fine when I was using:
Terraform Version : v0.11.8
AzureRM provider Version: v1.27.1

[bhavin192]

I was facing the second error you mentioned, adding dependency on azurerm_network_interface_security_group_association in azurerm_virtual_machine like this https://github.com/bhavin192/terraform-azure-yugabyte/commit/960a037a5ec09ce8b942294491a67af323e76cab
Seems to be solving the issue for me tried apply and destroy 3 to 4 times.

$ terraform version
Terraform v0.12.24
+ provider.azurerm v2.7.0
+ provider.null v2.1.2

[ArcturusZhang]

Hi @surajmuthreja thanks for this issue.
After a brief reading, I assume this may get more abstract like this issue. In my opinion this should be an issue with terraform core since terraform cannot properly handle the dependencies when the one depends on other should be first modified before the deletion is applied.

And for a valid workaround in this case, adding an explicit dependency should work.

[surajmuthreja]

I was facing the second error you mentioned, adding dependency on azurerm_network_interface_security_group_association in azurerm_virtual_machine like this bhavin192/terraform-azure-yugabyte@960a037
Seems to be solving the issue for me tried apply and destroy 3 to 4 times.

$ terraform version
Terraform v0.12.24
+ provider.azurerm v2.7.0
+ provider.null v2.1.2

Adding dependency for 'azurerm_network_interface_security_group_association' in VM did the trick. I had dependency added for all most of the resources except this 'azurerm_network_interface_security_group_association'. Its difficult to debug for which dependency it was cribbing for.
Thanks for your help!

[surajmuthreja]

Hi @surajmuthreja thanks for this issue.
After a brief reading, I assume this may get more abstract like this issue. In my opinion this should be an issue with terraform core since terraform cannot properly handle the dependencies when the one depends on other should be first modified before the deletion is applied.

And for a valid workaround in this case, adding an explicit dependency should work.

Hi @ArcturusZhang ,

Thanks for your analysis here. I was going through the issue mentioned by you on Terraform Core and they are suspecting it to be issue with azurerm provider as mentioned in comment.

I had seen several issues around this deletion of resources failing due to dependencies not resolved properly but there is no proper resolution on this.
I would suggest, can you guys (Terraform Core Team & Azurerm provider team) get together and provide some solution around this.

[bhavin192]

Its difficult to debug for which dependency it was cribbing for.

Yes, even we were trying to figure out since last 3 to 4 days. I just gave it a try after reading solution from OpenShift folks, https://github.com/openshift/installer/commit/2ee265b62c51cfe8218395ea9c02bf4125120196

[tombuildsstuff]

👋

Taking a look through here there's two separate issues occurring here:

Error: Error waiting for update of Network Interface "VM-2_NIC2" (Resource Group "ResourceGroup_Test"): Code="InternalServerError" Message="An error occurred." Details=[]

We automatically retry 500's several times - so unfortunately this is a bug within the Azure API.

Error: Error waiting for update of Network Interface "VM-3_NIC1" (Resource Group "ResourceGroup_Test"): Code="OperationNotAllowed" Message="Operation 'startTenantUpdate' is not allowed on VM 'VM-3' since the VM is marked for deletion. You can only retry the Delete operation (or wait for an ongoing one to complete)." Details=[]

The root-cause of this is an issue in Terraform Core where the dependency tree isn't walked correctly, which is being tracked in https://github.com/hashicorp/terraform/issues/24663 - since this isn't something that we're able to fix within the Azure Provider directly, I'm going to close this in favour of that issue, would you mind subscribing to that issue for updates?

Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!