Terraform-provider-azurerm: Deploying Azure App Service Environment fails

[Phydeauxman]

Trying to deploy an App Service Environment (ASE) v2 in Azure. Since there is currently no Terraform ASE resource, attempting to use ARM template resource method. ASEs take anywhere from 1 to 2 hours to provision. Each time I run the deployment, the Terraform deployment errors out after exactly 1 hour. The deployment in Azure continues and eventually completes successfully but I am left without an up-to-date state file and I am not able to collect the outputs I desire from the deployment. Once the Terraform deployment fails, if I wait until the deployment in Azure is complete and then re-run the Terraform deployment...the Terraform deployment will complete successfully.

Terraform Version

Terraform v0.11.3
+ provider.azurerm v1.2.0

Terraform Configuration Files

Terraform config file:

provider "azurerm" {
  subscription_id = "${var.sub_id}"
}

data "terraform_remote_state" "rg" {
  backend = "azurerm"

  config {
    storage_account_name = "${var.tfstate_storage_account}"
    container_name       = "${var.tfstate_container}"
    key                  = "${var.tfstate_rgstate_file}"
    access_key           = "${var.tfstate_access_key}"
  }
}

resource "azurerm_resource_group" "ase_rg" {
  name     = "${var.ilbase_rg_name}"
  location = "${data.terraform_remote_state.rg.rglocation}"
}

resource "azurerm_template_deployment" "ase" {
  name                = "ILBASE_ARM_template"
  resource_group_name = "${azurerm_resource_group.ase_rg.name}"

  template_body = <<DEPLOY

  {
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
      "ilbase_name": {
        "type": "string"
      },
      "ilbase_domain_name": {
        "type": "string"
      },
      "ilbase_subnet_name": {
        "type": "string"
      },
      "ilbase_rglocation": {
        "defaultValue": "East US",
        "type": "string"
      },
      "vnet_id": {
        "type": "string"
      }
    },
    "variables": {
    },
    "resources": [
      {
        "apiVersion": "2016-09-01",
        "type": "Microsoft.Web/hostingEnvironments",
        "name": "[parameters('ilbase_name')]",
        "kind": "ASEV2",
        "location": "[parameters('ilbase_rglocation')]",
        "properties": {
          "name": "[parameters('ilbase_name')]",
          "location": "[parameters('ilbase_rglocation')]",
          "virtualNetwork": {
            "Id": "[parameters('vnet_id')]",
            "Subnet": "[parameters('ilbase_subnet_name')]"
          },
          "internalLoadBalancingMode": "Web, Publishing",
          "multiSize": "Standard_D1_V2",
          "multiRoleCount": 2,
          "workerPools": null,
          "ipsslAddressCount": 0,
          "dnsSuffix": "[parameters('ilbase_domain_name')]",
          "networkAccessControlList": [],
          "frontEndScaleFactor": 15,
          "apiManagementAccountId": null,
          "suspended": false,
          "dynamicCacheEnabled": null,
          "clusterSettings": null
        }
      }
    ],
    "outputs": {
    }
  }

  DEPLOY

  # these key-value pairs are passed into the ARM Template's `parameters` block
  parameters {
    "vnet_id"            = "${data.terraform_remote_state.rg.vnetid}"
    "ilbase_subnet_name" = "${data.terraform_remote_state.rg.sn2name}"
    "ilbase_name"        = "${var.ilbase_name}"
    "ilbase_domain_name" = "${var.ilbase_domain_name}"
  }

  deployment_mode = "Incremental"
}

Shell script used to deploy config:

#!/bin/bash
set -eo pipefail

# The block below will grab the access key for the storage account that is used
# to store state files

subscription_name="Development"
tfstate_storage_resource_group="terraform-state-rg"
tfstate_storage_account="mystatefile"

az account set --subscription "$subscription_name"
tfstate_storage_access_key=$(
  az storage account keys list \
  --resource-group "$tfstate_storage_resource_group" \
  --account-name "$tfstate_storage_account" \
  --query '[0].value' -o tsv
)

echo ""
echo "Terraform state storage account access key:"
echo $tfstate_storage_access_key
echo ""

terraform apply \
  -var "tfstate_access_key=$tfstate_storage_access_key"

Debug Output

https://gist.github.com/Phydeauxman/0f9aa3d1c1379c36e2f8f420d0ae345e

Expected Behavior

Deployment should have completed successfully and state file should have been updated

Actual Behavior

After exactly 1 hour the deployment errored out with the following error:

azurerm_template_deployment.ase: Error creating deployment: azure#WaitForCompletion: context has been cancelled: StatusCode=200 -- Original Error: context deadline exceeded

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. ./apply.sh

[metacpp]

@Phydeauxman since we're adding the ASE resource in #869 and we met the similar long time running but fail issue, and we're working closely with service team to see if there's anything wrong on their side, will share more updates later.

[Phydeauxman]

@metacpp Thanks for the update. I also have an open ticket with the Azure service team on this issue because the behavior exists also when deploying from VS 2017 using the ARM template I use in my Terraform config.

[tombuildsstuff]

@metacpp heads up that this is an issue caused during the upgrade to the latest version of the Azure SDK (v12) where the handling of timeouts has changed; I'm looking into this at the moment since it affects other resources too (as part of #1006)

[tombuildsstuff]

hey @Phydeauxman

Since the root cause for this is a duplicate of #171 I'm going to close this in favour of that issue (rather than having multiple issues tracking the same thing) - which we'll be fixing in the near future.

Thanks!

[rifaterdemsahin]

updates are appreciated ... (👍

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!