Terraform-provider-azurerm: Unable to destroy VM protected items

[ranokarno]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Hi,
recently I am unable to delete VM protected items. I suspect this could be due to Azure backend change.
from Azure portal it mentioned VM is deleted in deferred delete but Terraform expect Not Found

Terraform (and AzureRM Provider) Version

Terraform v0.11.11
AzureRm 1.33.1

Affected Resource(s)

• azurerm_recovery_services_protected_vm

Terraform Configuration Files

resource "azurerm_recovery_services_protected_vm" "vm1" {
  resource_group_name = "${azurerm_resource_group.example.name}"
  recovery_vault_name = "${azurerm_recovery_services_vault.example.name}"
  source_vm_id        = "${azurerm_virtual_machine.example.id}"
  backup_policy_id    = "${azurerm_recovery_services_protection_policy_vm.example.id}"
}

Expected Behavior

azurerm_recovery_services_protected_vm.vm1: destroy completed.

Actual Behavior

azurerm_recovery_services_protected_vm.vm1: Still destroying... (ID: <vm_resource_id>, 30m0s elapsed)

Error: Error applying plan:

1 error(s) occurred:

* azurerm_recovery_services_protected_vm.vm1 (destroy): 1 error(s) occurred:

* azurerm_recovery_services_protected_vm.vm1: Error waiting for the Recovery Service Protected VM "VM;iaasvmcontainerv2;vm_resource_group;test_vm1" to be false (Resource Group "recovery_vault_resource_group") to provision: timeout while waiting for state to become 'NotFound' (last state: 'Found', timeout: 30m0s)

Steps to Reproduce

1. terraform apply
2. terraform destroy

[Dev-Mivl]

Hi Rano, did you find a workaround for this solution? My plan wants to destroy these before applying the rest of the plan.

[marcpascual]

unfortunately as stated in https://github.com/MicrosoftDocs/azure-docs/issues/39329, there are no plans to revert the change or make it a toggle, similar to AKV.

[ranokarno]

@Dev-Mivl ,
workaround is to manually remove Terraform State related with azurerm_protected_item resource address.
AzureRm provider may need to change the return code for this resource.

in my case, i will run terraform state rm <resource_address>
you can refer resource addressing for more info, or run terraform state list

[ivanov-aleksander]

The problem is that your Azure Backup Vault has configured Soft delete. As a resul the backup data is retained for 14 additional days, allowing the recovery of that backup item with no data loss. These additional 14 days retention of backup data in the "soft delete" state don’t incur any cost to the customer.

[katbyte]

@ranokarno, @ivanov-aleksander,

Currently there is no way to turn off soft delete, and despite the API "soft-deleting" the VM it returns an error. This has been raised with the Azure service team and they have talked about potential fixes.. however i don't have date for when they will make the them.

[ThoHah]

As there is a manual way (.1 deactivating soft state for the vault, 2.deleting backup data, 3. destroying the vault) this should be possible via API too.

[tombuildsstuff]

@ThoHah the API for disabling Soft Delete has only recently become available (in the last few weeks), so whilst this is likely now possible - unfortunately we've not gotten to it yet; there's an issue tracking support for this however :)

[dpbustos]

Is it possible to add an option to retain backup data on destroy of the azurerm_recovery_services_protected_vm resource? (This would be in the scenario where soft-delete has been disabled in the vault... Via the Portal, when you select Stop Backup you get the option to delete or retain backup data)

[jcanongfi]

This issue concern azurerm 1.xx and azurerm_recovery_services_protected_vm.
But the is still present on azurerm 2.xx and azurerm_backup_protected_vm.
I look at the code and the principe is in 2 steps :

• Sending the DELETE request on azure API.
• Loop waiting for state switching from "Found" to "NotFound".
  (File : terraform-provider-azurerm/azurerm/internal/services/recoveryservices/resource_arm_backup_protected_vm.go: 203)
  Because a lot of us have configured a soft delete policy, it could be interesting that the loop (step2) beeing optional.
  Is it possible to add an optional resource property (boolean type) for getting the "state loop" optional ?

[perlboy]

@jcanongfi Yes it's still there. Destroying a azurerm_backup_protected_vm resource with soft delete enabled results in endless loop (I guess for 14 days). What would be ideal is if azurerm_backup_protected_vm destroy could consider soft delete as "destroyed". I have no problems with soft delete existing but would prefer to avoid direct state manipulation to work around it hanging on destroy (even after manual deletion).