Terraform-provider-azurerm: azurerm_frontdoor does not allow for key vault creation to occur in the same apply

[slynickel]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

terraform {
  required_version = ">= 0.12.21" # version tested on
  backend "local" {
  }
}

provider "azurerm" {
  version = "=2.0.0" # same issue occurs on 1.44.0
  # subscription_id = "" # redacted, we always set these though
  # tenant_id = "" # redacted, we always set these though
  features {
  }
}

data "azurerm_client_config" "current" {
}

locals {
  frontdoor_name = "terraformtoyexample"
  kvpartialname = "36dg23fgs2"
}

resource "azurerm_resource_group" "rg" {
    name     = "frontdoor"
    location = "northcentralus"
}

resource "azurerm_key_vault" "kvfd" {
  name = "kvfd-${local.kvpartialname}"
  location                                     = azurerm_resource_group.rg.location
  resource_group_name                          = azurerm_resource_group.rg.name
  tenant_id                                    = data.azurerm_client_config.current.tenant_id
  enabled_for_deployment = false
  enabled_for_disk_encryption = false
  enabled_for_template_deployment = false
  purge_protection_enabled = false
  soft_delete_enabled = false
  sku_name = "standard"
  network_acls {
    default_action             = "Allow"
    bypass                     = "AzureServices" # Really we just need Azure Front Door, but not sure terraform provides a method to do that
  }
}

resource "azurerm_key_vault_access_policy" "frontdoor" {
  key_vault_id = azurerm_key_vault.kvfd.id
  tenant_id    = data.azurerm_client_config.current.tenant_id
    object_id    = "daa778f6-798e-4f28-ac66-6d6d2be6f6cc" # az ad sp show --id ad0e1c7e-6d38-4ba4-9efd-0bc77ba9f037 | jq .objectId << Azure Front Door Application
  certificate_permissions = [
    "Get"
  ]
  secret_permissions = [
    "Get"
  ]
  key_permissions = [   ]
}

resource "azurerm_key_vault_certificate" "example" {
  name         = "example"
  key_vault_id = azurerm_key_vault.kvfd.id

  certificate {
    contents = filebase64("/tmp/test.txt") # file needs to exist but doesnt need to be valid certificate for this issue
    password = ""
  }
  certificate_policy {
    issuer_parameters {
      name = "Unknown"
    }
    key_properties {
      exportable = true
      key_size   = 4096
      key_type   = "RSA"
      reuse_key  = false
    }
    secret_properties {
      content_type = "application/x-pkcs12"
    }
  }
}

# can switch between secret and certificate both will fail
resource "azurerm_key_vault_secret" "examplesecret" {
  name = "examplesecret"
  key_vault_id = azurerm_key_vault.kvfd.id
  value = filebase64("/tmp/test.txt") # file needs to exist but doesnt need to be valid certificate for this issue
}

resource "azurerm_frontdoor" "frontdoor" {
  name                                         = local.frontdoor_name
  friendly_name                                = "frontdoor" // not clear how this is used
  location                                     = azurerm_resource_group.rg.location
  resource_group_name                          = azurerm_resource_group.rg.name
  enforce_backend_pools_certificate_name_check = true

  frontend_endpoint {
    name                              = "defaultFrontDoorFE"
    host_name                         = "${local.frontdoor_name}.azurefd.net" // for the default front end this has to match exactly
    session_affinity_enabled          = false
    custom_https_provisioning_enabled = false
  }

  # NOTE to get this to work you have to register the front door service principal
  # az ad sp create --id ad0e1c7e-6d38-4ba4-9efd-0bc77ba9f037
  frontend_endpoint {
    name                              = "example1FE"
    host_name                         = "doesntmatterforthis.example.com"
    session_affinity_enabled          = false
    custom_https_provisioning_enabled = true
    custom_https_configuration {
      certificate_source = "AzureKeyVault"
      azure_key_vault_certificate_vault_id = azurerm_key_vault.kvfd.id
      azure_key_vault_certificate_secret_name = azurerm_key_vault_certificate.example.name
      azure_key_vault_certificate_secret_version = azurerm_key_vault_certificate.example.version
      # can use either both return the same error, I thought based on #5468 that you had to use secret resources, and certificates weren't support both fail
      # azure_key_vault_certificate_secret_name = azurerm_key_vault_secret.examplesecret.name
      # azure_key_vault_certificate_secret_version = azurerm_key_vault_secret.examplesecret.version
    }
  }

  routing_rule {
    name               = "example1FEdefaultAppServiceBERR"
    accepted_protocols = ["Https"]
    patterns_to_match  = ["/*"]
    frontend_endpoints = ["defaultFrontDoorFE", "example1FE"]
    forwarding_configuration {
      forwarding_protocol = "HttpsOnly"
      backend_pool_name   = "defaultAppServiceBE"
      cache_use_dynamic_compression = false
    }
    enabled = true
  }

  backend_pool {
    name = "defaultAppServiceBE"

    backend {
      enabled = true
      host_header = "alsodoesntmatterforthis.example.com"
      address     = "alsodoesntmatterforthis.example.com"
      http_port   = 80
      https_port  = 443
      priority    = 1
      weight      = 50
    }

    load_balancing_name = "defaultAppServiceLB"
    health_probe_name   = "defaultAppServiceHP"
  }

  backend_pool_load_balancing {
    name                            = "defaultAppServiceLB"
    sample_size                     = 4
    successful_samples_required     = 2
    additional_latency_milliseconds = 0
  }

  backend_pool_health_probe {
    name                = "defaultAppServiceHP"
    path                = "/"
    protocol            = "Http"
    interval_in_seconds = 255
  }

  load_balancer_enabled = true
  depends_on = [
    azurerm_key_vault.kvfd,
    azurerm_key_vault_certificate.example,
    azurerm_key_vault_secret.examplesecret # shouldnt need anything in this block
  ]
}

Debug Output

I don't want to post the entire debug output since I don't want to redact specifics.

2020-02-27T17:20:49.570-0600 [DEBUG] plugin.terraform-provider-azurerm_v2.0.0_x5: [DEBUG] Determining which Resource Providers require Registration
2020-02-27T17:20:49.570-0600 [DEBUG] plugin.terraform-provider-azurerm_v2.0.0_x5: [DEBUG] All required Resource Providers are registered
2020/02/27 17:20:49 [DEBUG] Resource instance state not found for node "azurerm_resource_group.rg", instance azurerm_resource_group.rg
2020/02/27 17:20:49 [DEBUG] ReferenceTransformer: "azurerm_resource_group.rg" references: []
2020/02/27 17:20:49 [DEBUG] ReferenceTransformer: "data.azurerm_client_config.current" references: []
2020/02/27 17:20:49 [DEBUG] Resource instance state not found for node "azurerm_key_vault.kvfd", instance azurerm_key_vault.kvfd
2020/02/27 17:20:49 [DEBUG] ReferenceTransformer: "azurerm_key_vault.kvfd" references: []
2020/02/27 17:20:49 [DEBUG] Resource instance state not found for node "azurerm_key_vault_certificate.example", instance azurerm_key_vault_certificate.example
2020/02/27 17:20:49 [DEBUG] ReferenceTransformer: "azurerm_key_vault_certificate.example" references: []
2020/02/27 17:20:49 [DEBUG] Resource instance state not found for node "azurerm_key_vault_secret.examplesecret", instance azurerm_key_vault_secret.examplesecret
2020/02/27 17:20:49 [DEBUG] ReferenceTransformer: "azurerm_key_vault_secret.examplesecret" references: []
2020/02/27 17:20:49 [DEBUG] Resource instance state not found for node "azurerm_key_vault_access_policy.frontdoor", instance azurerm_key_vault_access_policy.frontdoor
2020/02/27 17:20:49 [DEBUG] ReferenceTransformer: "azurerm_key_vault_access_policy.frontdoor" references: []
2020/02/27 17:20:49 [WARN] Provider "registry.terraform.io/-/azurerm" produced an invalid plan for azurerm_key_vault_certificate.example, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .certificate_policy[0].x509_certificate_properties: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
2020/02/27 17:20:49 [DEBUG] Resource instance state not found for node "azurerm_frontdoor.frontdoor", instance azurerm_frontdoor.frontdoor
2020/02/27 17:20:49 [DEBUG] ReferenceTransformer: "azurerm_frontdoor.frontdoor" references: []
2020/02/27 17:20:49 [ERROR] <root>: eval: *terraform.EvalDiff, err: Error creating Front Door "terraformtoyexample" (Resource Group "frontdoor"): "frontend_endpoint":"example1FE" "custom_https_configuration" is invalid, all of the following keys must have values in the "custom_https_configuration" block: "azure_key_vault_certificate_secret_name", "azure_key_vault_certificate_secret_version", and "azure_key_vault_certificate_vault_id"
2020/02/27 17:20:49 [ERROR] <root>: eval: *terraform.EvalSequence, err: Error creating Front Door "terraformtoyexample" (Resource Group "frontdoor"): "frontend_endpoint":"example1FE" "custom_https_configuration" is invalid, all of the following keys must have values in the "custom_https_configuration" block: "azure_key_vault_certificate_secret_name", "azure_key_vault_certificate_secret_version", and "azure_key_vault_certificate_vault_id"
2020/02/27 17:20:49 [INFO] backend/local: plan operation completed

Error: Error creating Front Door "terraformtoyexample" (Resource Group "frontdoor"): "frontend_endpoint":"example1FE" "custom_https_configuration" is invalid, all of the following keys must have values in the "custom_https_configuration" block: "azure_key_vault_certificate_secret_name", "azure_key_vault_certificate_secret_version", and "azure_key_vault_certificate_vault_id"

  on frontdoor.tf line 89, in resource "azurerm_frontdoor" "frontdoor":
  89: resource "azurerm_frontdoor" "frontdoor" {


2020-02-27T17:20:49.613-0600 [DEBUG] plugin: plugin process exited: path=/Users/REDACTED/CloudServices/cloudlab/toy-frontdoor/.terraform/plugins/darwin_amd64/terraform-provider-azurerm_v2.0.0_x5 pid=55102
2020-02-27T17:20:49.613-0600 [DEBUG] plugin: plugin exited

Expected Behavior

The plan would have succeeded.

Actual Behavior

Plan threw an error for a resource that has not yet been created.

Steps to Reproduce

1. terraform init
2. terraform plan (terraform apply will also produce the error)

Important Factoids

• You do not need a valid certificate to run this example, a file just needs to exist at /tmp/test.txt
• In the example above I have set it up so you can easily switch between using a certificate or secret key vault resource. Given #5468 I was thinking you had to use a secret resource but both fail.
• ALSO if you create the key vault first (say by setting front door count =0 or just commenting it out) and then add in the front door it works fine. This works with certificate resource. I haven't tried with a secret resource.

References

• #5468 << this is an enhancement request, overall I'm finding this doesn't work at all, with a certificate or a secret.

[WodansSon]

@slynickel thanks for opening this issue, the front door resource assumes that the Key Vault has already been configured correctly prior to running the apply. If you wish to create both in the same configuration file you will need to add a depends on to your front door block to force it to wait until the key vault and the certificates have been created before creating the front door resource.

depends_on = [azurerm_key_vault_certificate.example,]

Azure Key Vault certificates: If you already have a certificate, you can upload it directly to your Azure Key Vault account or you can create a new certificate directly through Azure Key Vault from one of the partner CAs that Azure Key Vault integrates with. Upload your certificate as a certificate object, rather than a secret.

Important: You must upload the certificate in PFX format without password protection.

[slynickel]

@WodansSon you'll note in my fully functional example (if you put an empty file at /tmp/test.txt) I explicitly provide the following block inside of the front door resource. This should ensure that the front door resource waits for the key vault and key vault certificate to be created. It does not. It errors prior to running the plan.

  depends_on = [
    azurerm_key_vault.kvfd,
    azurerm_key_vault_certificate.example,
    azurerm_key_vault_secret.examplesecret # shouldnt need anything in this block
  ]

It would appear there is an issue where the front door resource doesn't allow these to be filled in after. Adding the depends block does not fix the issue.

[kingsleyadam]

depends_on = [azurerm_key_vault_certificate.example,]

I added this to my configuration, assuming this would address the issue, but I still get the same error as @slynickel

My workaround, for now, was to create the Key Vault certificate entry in a separate terraform plan. Then run the FrontDoor plan.

But because the certificate and front door are so dependent on one another, I'd like to have them in the same plan.

[matt-FFFFFF]

I'm having trouble writing a test for this scenario. The issue is that Front Door must use a valid certificate from Key Vault, self-signed won't work.

Therefore, when trying to codify an acceptance test in the azurerm/internal/services/frontdoor/tests/resource_arm_front_door_test.go file we need to be able to supply a valid cert.

We could create one with Let's Encrypt, but this means referencing another provider - I don't think this will work using the acceptance test framework.

We could supply in based on the contents of a file, but this breaks the automation.

Anybody got any ideas?

[slynickel]

Short of procuring a certificate from a valid CA I'm not sure of a good way to test. That's how I was testing this.

FWIW, it would appear other folks +1'ed this but I've switched to using the managed cert to avoid this. Also Key vault doesn't have great HA options for this so the managed seemed like a better option.

[WodansSon]

@slynickel, unfortunately this is currently by design given the current architecture of the API. You must first create the key vault and then add front door to the key vaults access policy. There are a number of steps that need to happen before you can use your custom domain. Given that this is currently by design I am going to close this issue, feel free to re-open the issue if you believe this is in error.

[matt-FFFFFF]

I don't believe this is by design. If the resources were created in the correct order, as defined by the implicit and explicit dependencies in the plan, then it should be possible to create the key, the key vault and the Front Door in one apply step.

The problem is that the validate stage fails as it is looking for the resource details of the to-be-created key vault secret. No calls have been made to the networking resource provider at the time of the validate so it is not an Azure API limitation.

The validate stage should recognise that the current plan will create the required resources and allow the application of the plan, substituting in the resource details once created.

Multiple apply steps should not be required to do this.

[matt-FFFFFF]

@slynickel Managed certs did not support apex domains. E.g. xyz.com. They only support www.xyz.com.

[robdavey555]

I'm hitting the same issue as the op.

I have tried adding this suggestion without success:

depends_on = [azurerm_key_vault.example, azurerm_key_vault_certificate.example, azurerm_key_vault_access_policy]

This is my section for frontend_endpoint within the FrontDoor config

    custom_https_provisioning_enabled = true
    custom_https_configuration {
      certificate_source                         = "AzureKeyVault"
      azure_key_vault_certificate_vault_id       = azurerm_key_vault.example.id
      azure_key_vault_certificate_secret_name    = azurerm_key_vault_certificate.example.name
      azure_key_vault_certificate_secret_version = azurerm_key_vault_certificate.example.version
    }

It seems like values (below) from the creation of my keyvault resources are not being returned for use within the creations of my frontdoor resource.

azurerm_key_vault.example.id
azurerm_key_vault_certificate.example.name
azurerm_key_vault_certificate.example.version

If I hard code the values from a manually added certificate, it will work. This confirms The granted access between FrontDoor and KeyValut works.

Is there anything I should be aware of?

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!