Terraform-provider-azurerm: azurerm_resource_group_template_deployment: running 'terraform apply' succeeds, than fails on successive attempts

[geekzter]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

terraform -v
Terraform v0.13.4
+ provider registry.terraform.io/hashicorp/azurerm v2.31.0
+ provider registry.terraform.io/hashicorp/random v2.3.0

Affected Resource(s)

• azurerm_resource_group_template_deployment

Thanks for creating an resource that can destroy ARM resources!

Terraform Configuration Files

locals {
  suffix                       = random_string.suffix.result
  tags                         = map(
    "application",               "Dev/Test Lab",
    "environment",               "repro",
    "provisioner",               "terraform",
    "suffix",                    local.suffix,
    "workspace",                 terraform.workspace
  )
}
# Random resource suffix, this will prevent name collisions when creating resources in parallel
resource random_string suffix {
  length                       = 4
  upper                        = false
  lower                        = true
  number                       = false
  special                      = false
}

resource azurerm_resource_group lab_resource_group {
  name                         = "lab-${terraform.workspace}-${local.suffix}"
  location                     = var.location
  tags                         = local.tags
}

resource azurerm_dev_test_lab lab {
  name                         = "${azurerm_resource_group.lab_resource_group.name}-lab"
  location                     = azurerm_resource_group.lab_resource_group.location
  resource_group_name          = azurerm_resource_group.lab_resource_group.name

  tags                         = local.tags
}

resource azurerm_virtual_network custom_network {
  name                         = "${azurerm_resource_group.lab_resource_group.name}-network"
  location                     = azurerm_resource_group.lab_resource_group.location
  resource_group_name          = azurerm_resource_group.lab_resource_group.name
  address_space                = ["10.1.0.0/16"]

  tags                         = local.tags
}

resource azurerm_subnet custom_subnet {
  name                         = "CustomSubnet"
  virtual_network_name         = azurerm_virtual_network.custom_network.name
  resource_group_name          = azurerm_virtual_network.custom_network.resource_group_name
  address_prefixes             = ["10.1.1.0/24"]
}

resource azurerm_resource_group_template_deployment custom_network_association {
  name                         = "${azurerm_resource_group.lab_resource_group.name}-network-association"
  resource_group_name          = azurerm_resource_group.lab_resource_group.name
  deployment_mode              = "Incremental"

  template_content             = file("${path.module}/lab-network-association.json")
  parameters_content           = templatefile("${path.module}/lab-network-association-parameters.json",
    {
      lab_id                   = azurerm_dev_test_lab.lab.id
      virtual_network_id       = azurerm_virtual_network.custom_network.id
      virtual_network_subnet_id= azurerm_subnet.custom_subnet.id
    }
  )

  debug_level                  = "requestContent"
}

lab-network-association.json:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "labId": {
            "type": "string"
        },
        "virtualNetworkId": {
            "type": "string"
        },
        "virtualNetworkSubnetId": {
            "type": "string"
        }
    },
    "variables": {
        "labVirtualNetworkId": "[concat(parameters('labId'),'/virtualnetworks/', split(parameters('virtualNetworkId'),'/')[8])]"
    },
    "resources": [
        {
            "apiVersion": "2016-05-15",
            "type": "Microsoft.DevTestLab/labs",
            "id": "[parameters('labId')]",
            "name": "[split(parameters('labId'),'/')[8]]",
            "location": "[resourceGroup().location]",
            "resources": [
                {
                    "apiVersion": "2016-05-15",
                    "id": "[variables('labVirtualNetworkId')]",
                    "name": "[split(parameters('virtualNetworkId'),'/')[8]]",
                    "type": "virtualNetworks",
                    "dependsOn": [
                        "[parameters('labId')]"
                    ],
                    "properties": {
                        "description": "Existing Compute virtual network associated as part of the lab",
                        "externalProviderResourceId": "[parameters('virtualNetworkId')]",
                        "subnetOverrides": [
                            {
                                "name": "[split(parameters('virtualNetworkSubnetId'),'/')[10]]",
                                "resourceId": "[parameters('virtualNetworkSubnetId')]",
                                "useInVmCreationPermission": "Allow",
                                "usePublicIpAddressPermission": "Allow"
                            }
                        ]
                    }
                }
            ]
        }
    ],
    "outputs": {
        "labVirtualNetworkId": {
            "type": "string",
            "value": "[variables('labVirtualNetworkId')]"
        }
    }
}

lab-network-association-parameters.json:

{
    "labId": {
        "value": "${lab_id}"
    },
    "virtualNetworkId": {
        "value": "${virtual_network_id}"
    },
    "virtualNetworkSubnetId": {
        "value": "${virtual_network_subnet_id}"
    }
}

Debug Output

Panic Output

Expected Behavior

Running 'terraform apply' a second time should be idempotent and succeed as the first 'terraform apply' also succeeeds

Actual Behavior

Running 'terraform apply' a second time yields an error:

Error: validating Template Deployment "lab-default-vlac-network-association" (Resource Group "lab-default-vlac"): requesting validating: resources.DeploymentsClient#Validate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidTemplate" Message="Deployment template validation failed: 'The value for the template parameter 'labId' at line '1' and column '266' is not provided. Please see https://aka.ms/resource-manager-parameter-files for usage details.'." AdditionalInfo=[{"info":{"lineNumber":1,"linePosition":266,"path":"properties.template.parameters.labId"},"type":"TemplateViolation"}]

  on main.tf line 50, in resource "azurerm_resource_group_template_deployment" "custom_network_association":
  50: resource azurerm_resource_group_template_deployment custom_network_association {

Steps to Reproduce

1. terraform init
2. terraform apply
3. terraform apply

Important Factoids

References

[tombuildsstuff]

Taking a look through this is likely this: https://github.com/terraform-providers/terraform-provider-azurerm/blob/756cf9fc8acacc7b988b8ec29284d058494ca5b6/azurerm/internal/services/resource/template_deployment_resource_group_resource.go#L190-L196 (and here in the subscription resource) where I guess we'll need to always set these fields, since they contain values? Needs an acceptance test to confirm

[tinablonjazz]

Hello, I have also encountered this issue.
As a temporary workaround, I inserted following block:

/*
Temporary workaround for subsequent azurerm_resource_group_template_deployment bug in terraform.
Resource can be created or destroyed, but will do no updates.
*/
lifecycle {
ignore_changes = all
}

My arm deployment deploys workspace-based App Insights and I know I do not want to do any changes at the moment as well as don't expect any change in the future for this particular resource. Still, would be great if this gets a permanent and less "dirty" fix.

If you'll need to do changes at yours, it's sufficient to change the deployment name - although it will force new resource creation (e.g. my App Insights kept the same name, but the instrumentation_key has been changed)

[lodejard]

Very frustrating bug to encounter!

@geekzter if you're still having this problem another workaround to consider is to add an additional unused parameter

  parameters_content            = templatefile("${path.module}/lab-network-association-parameters.json",
    {
      lab_id                    = azurerm_dev_test_lab.lab.id
      virtual_network_id        = azurerm_virtual_network.custom_network.id
      virtual_network_subnet_id = azurerm_subnet.custom_subnet.id
      template_md5              = md5(file("${path.module}/lab-network-association.json"))
    }
  )

The template_md5 doesn't need to be used by any resources in the template - but at least whenever the the template file is seen to have changed the resource provider will also send parameters_content as they will also always have a change. Still enables the resource to be skipped when the file is not different.

[geekzter]

Thanks @lodejard . I used the old azurerm_template_deployment, which works for my scenario.

[TylerLu]

I found an easy workaround - use a dynamic name. For example:

resource "azurerm_resource_group_template_deployment" "api_connections" {
  name = "api-connections-${formatdate("YYMMDDhhmmss", timestamp())}"
  ...
}