Terraform-provider-azurerm: API Management (azurerm_api_management) with custom domain.

[msivers]

I'm trying to script API Management with custom domain where certificate is obtained from Key Vault.

I seem to have a catch-22 where I am defining azurerm_api_management with the Identiity property set to 'SystemAssigned' and with host_configuration > proxy > key_vault_id set to 'azurerm_key_vault_secret..id'... The problem here is that at this stage the system assigned managed identity has not been given permission to Key Vault (which I have defined in the next step using azurerm_key_vault_access_policy.
rs.html#provider-versions). --->

Affected Resource(s)

• azurerm_api_management
• azurerm_key_vault_access_policy
• azurerm_key_vault_secret

Terraform Configuration Files

data "azurerm_key_vault_secret" "core_apim" {
  name      = "SslCert"
  vault_uri = "${azurerm_key_vault.core.vault_uri}"
}

resource "azurerm_api_management" "core" {
  name                = "${var.res_prefix}-${var.environment}-apim"
  location            = "${var.primary_location}"
  resource_group_name = "${azurerm_resource_group.core.name}"
  publisher_name      = "HELLO WORLD"
  publisher_email     = "[email protected]"

  sku {
    name     = "${var.apim_tier_name}"
    capacity = "${var.apim_unit_count}"
  }

  identity
  {
    type = "SystemAssigned"
  }

  hostname_configuration {
    proxy {
      host_name = "${replace(replace(replace(var.environment, "/^d.*$/", "dev-api"), "/^t.*$/", "test-api"), "/^p.*$/", "api")}.mydomain.net"
      key_vault_id = "${data.azurerm_key_vault_secret.core_apim.id}"  # TODO: Handle count
    }
  }
}

resource "azurerm_key_vault_access_policy" "core_apim" {
  key_vault_id = "${azurerm_key_vault.core.id}"

  tenant_id = "${azurerm_api_management.core.identity.0.tenant_id}"
  object_id = "${azurerm_api_management.core.identity.0.principal_id}"

  key_permissions = [
    "backup", "create", "decrypt", "delete", "encrypt", "get", "import", "list", "purge", "recover", "restore", "sign", "unwrapKey", "update", "verify", "wrapKey"
  ]

  secret_permissions = [
    "backup", "delete", "get", "list", "purge", "recover", "restore", "set"
  ]

  certificate_permissions = [
    "backup", "create", "delete", "deleteissuers", "get", "getissuers", "import", "list", "listissuers", "managecontacts", "manageissuers", "purge", "recover", "restore", "setissuers", "update"
  ]
}

Expected Behavior

API Management setup with custom domain (using cert from Key Vault).

Actual Behavior

Error applying plan....

azurerm_api_management.core: Error creating/updating API Management Service "rp-dev-apim" (Resource Group "rd-dev-core-rg"): apimanagement.ServiceClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="BadRequest" Message="Request to resource 'https://{subdomain}.vault.azure.net/secrets/RedPlatSslCert/5b_________e4_________fe_____2b?api-version=7.0' failed with StatusCode: Forbidden for RequestId: . Exception message: Operation returned an invalid status code 'Forbidden'"

[msivers]

This scenario using ARM is described here... https://docs.microsoft.com/en-us/azure/api-management/api-management-howto-use-managed-service-identity#use-the-managed-service-identity-to-access-other-resources

[wasfree]

Hi @msivers,
after looking deeper into this it looks like a chicken vs. egg causality dilemma. I'm not sure if this can/should be solved in terraform. It would make more scene, if Azure had the possibility to add an predefined service principal to API Management instead of managed Identity (similar like for an AKS Cluster).

Then we could do something like this:

  service_principal {
    client_id     = "00000000-0000-0000-0000-000000000000"
    client_secret = "00000000000000000000000000000000"
  }

[simongh]

Could we solve this using a provisioner script? That would then mean removing the ability to set the cert using key vault within the resource. The provisioner would run once the api management resource is created and therefore so is the service principal.

Could we also fix it by moving the cert configuration off the azurerm_api_management resource and into it's own resource. You'd still have a problem ensuring it ran after the key vault access policy resource though, unless it had an explicit dependency setup.

[jeanpaulsmit]

I managed to fix this, with the assistance of Microsoft.
Things are a bit different than I thought (read the tip section in https://docs.microsoft.com/en-us/azure/api-management/configure-custom-domain).
My steps are:

• Create Key Vault
• Set kv access policy to import+get certificate for executing process to allow import of certificate
• Import certificate in certificate store in kv
• Create APIM instance with managed identity, but leave host configuration section out
• Set kv access policy to 'get' for secret+certificate to allow APIM identity to read
• Use Powershell to set APIM host configuration using New-AzApiManagementCustomHostnameConfiguration + keyvault secret id

This solves the chicken-egg problem.

[molinch]

The best would be to have a separate resource to set the custom domains. Then the chicken/egg problem is solved.
@tombuildsstuff there are a bunch of open APIM related issues, do you know if there will be another round of implementation? At the moment a lot of stuff still has to go through Powershell scripts, which is a pity because HCL really rocks!

[prashantshetage]

"User Assigned Managed Identities" feature for API Management is in preview.
Once it will be in GA, can we leverage this feature to retrieve certificates/secrets from Key vault to fix this issue?

[w0ut0]

Can we split the proxy from the APIM instances? Sort of like we do for App Services: https://www.terraform.io/docs/providers/azurerm/r/app_service_custom_hostname_binding.html

This way you can create resources in order:

• APIM + managed identity
• key vault access policy
• custom hostname

I tried using 'User Assigned Managed Identities', but it's currently not possible to use this identity to access the KeyVault for the custom hostname certificate.

[SaileswarSaha]

As of now, this issue can be solved only by azurerm 1.44 version with the below considerations -->

We need to split the creation of the resources in below order :

1. Key Vault.
2. APIM + managed identity (system assigned)
3. Add APIM principal id into --> key vault access policy
4. Again update APIM with custom hostname certificate.

We've already raised a hasicorp ticket to make this available 2.7 onward & shared TF logs using 1.44 as requested.

[sirlatrom]

"User Assigned Managed Identities" feature for API Management is in preview.
Once it will be in GA, can we leverage this feature to retrieve certificates/secrets from Key vault to fix this issue?

For what it's worth, I tried this approach, which is why I created #6783. Unfortunately, it does not work. Furthermore, I was told it is not planned to be supported anytime soon, i.e. only System Assigned Identity is used when retrieving Key Vault Certificates.

[schlbra]

Any updates on this PR?

[sirlatrom]

I've created a PR for a separate resource: https://github.com/terraform-providers/terraform-provider-azurerm/pull/8228

[schlbra]

I've created a PR for a separate resource: #8228

Your efforts on this PR are much appreciated @sirlatrom! Thank you for your contributions.

[hashibot[bot]]

This has been released in version 2.36.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 2.36.0"
}
# ... other configuration ...

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!