Azure-cli: Application Gateway KeyVault support

[amitsriva]

Application Gateway has recently added support for Azure Key Vault certificates. Please enable this on CLI as well.

[tjprescott]

@amitsriva I need more information to be able to start this. Please provide the following. In the future, we have created a special issue template for service teams requesting CLI feature work (which is where these questions are pulled from)

Description of Feature
Provide a brief description of the feature. A link to conceptual documentation may be helpful too.

Minimum API Version Required
What is the minimum API version of your service required to implement your feature?

Swagger PR Link
Provide a link to the closed PR for the rest API specs repo. Note that this PR should be to the
master branch of the rest API specs repo.

Target Date (Optional)
If you have a target date for release of this feature, please provide it. That will help us
schedule the work.

[tjprescott]

Moving to the backlog until we get sufficient information from the service team to move forward.

[akshaysngupta]

Description of Feature

KeyVault integration with AppGW enables customers to specify a keyvault secret Id in place of a base 64 encoded certificate/pfx file in the appgw to setup a https listener.
Customer needs to perform 3 steps in order to make this scenario possible.

1. Create dependency resources: Create a user assigned identity and keyvault. Soft-delete feature must be enabled on the keyvualt in order for appgw to accept the secret. Appgw will throw a descriptive exception here.
2. Configure KeyVault: Set access policy on the keyvault to allow GET Secret operation by the user assigned identity.
3. Create/Update Application Gateway: Assign the above user assigned identity to AppGW by using the identity top-level tag present in the appgw. They also need to specify the secret Url `https:///secrets/ in the Ssl Certificate section.
   If no identity has been assigned to Appgw, then this operation will fail.

Note: Currently, We don't support keyVaultSecretId field in the trusted root certificates sub-resource. So please omit that field from CLI.

Minimum API Version Required

Network 2018-12-01

Swagger PR Link

https://github.com/Azure/azure-rest-api-specs/pull/4387

Target Date (Optional)

If you have a target date for release of this feature, please provide it. That will help us
schedule the work.

[tjprescott]

Thanks for the info @akshaysngupta. We'll get this work scheduled in an upcoming sprint

[tjprescott]

Bumping to Sprint 65 due to higher priority direction from NRP.

[tjprescott]

Moved to Sprint 70 to support other NRP work in 65. Arranged offline.

[nexxai]

@tjprescott Do you have any idea when Sprint 70 should be expected to kick off? Just wondering if it's reasonably expected to happen in the next few weeks, or whether we should be looking for other solutions.

[tjprescott]

Sprint 70 is around September. https://github.com/Azure/azure-cli/milestones

[MyronFanQiu]

@nexxai Talked with @MikhailTryakhov offline. I will start to implement this feature.

[MyronFanQiu]

@akshaysngupta Hello. I want to confirm something in detail.

• Application gw uses managed identity to obtain the key vault. Secret Id must be set in the ssl certification section. Right?
• Does ManagedServiceIdentity support setting a list of user identities associated with application gw?
• Trusted root certificates don't support keyVaultSecretId even though they are already defined in the swagger spec.

[MyronFanQiu]

cc/ @nexxai

[nexxai]

I'm not totally clear on the requirements; I'm just a Terraform user that has a project temporarily hung up on this API support. If there's something I can help with from a user perspective, I'd love to help contribute however I can.

[akshaysngupta]

@MyronFanQiu

• Yes, AppGW uses userAssignedIdentity to access secrets in KeyVault. Yes, SecretId is set in the SslCertificate as keyVaultSecretId field.
• No, as of now, AppGw usage of identity is constrained. We only allow one userAssignedIdentity on gateway. Remaining are rejected in validation.
• Yes, we might add support for keyvault to trusted root certificates, but for now, please skip it in the CLI.

[MyronFanQiu]

I'm not totally clear on the requirements; I'm just a Terraform user that has a project temporarily hung up on this API support. If there's something I can help with from a user perspective, I'd love to help contribute however I can.

Oh. Sorry. 😄 I thought you were the PM of our service's team.

[MyronFanQiu]

@MyronFanQiu

• Yes, AppGW uses userAssignedIdentity to access secrets in KeyVault. Yes, SecretId is set in the SslCertificate as keyVaultSecretId field.
• No, as of now, AppGw usage of identity is constrained. We only allow one userAssignedIdentity on gateway. Remaining are rejected in validation.
• Yes, we might add support for keyvault to trusted root certificates, but for now, please skip it in the CLI.

Thanks.

[MyronFanQiu]

@akshaysngupta Hi. Can the user both set cert-data/cert-password and keyvaultsecretID?

[MyronFanQiu]

@akshaysngupta Hello. Now I can use the command based on the python SDK to assign or remove identity from existed ApplicationGW. However, I cannot create an application gateway with identity information. Azure CLI will use ARM Template to create application gateway. I will met an error during deployment.

Deployment failed. Correlation ID: c34fa9e2-7540-4538-86c2-13775a2b895e. {                                 
  "error": {                                                                                               
    "code": "InvalidIdentityValues",                                                                       
    "message": "Invalid value for the identities '/subscriptions/xxxxxxxx-1bf0-4dda-aec3-cb9272f09590/resou
rceGroups/fanqiu-test/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identitytest'. The 'UserAs
signedIdentities' property keys should only be empty json objects, null or the resource exisiting property.
"                                                                                                          
  }                                                                                                        
}                                                                                                          

I also tried the following processes.

• Create a application gateway
• Assign identity to the application gateway
• Download the template of the created application gateway from the portal.
• Delete the application gateway
• Use the template in Azure portal to deploy a new application gateway
  However, I met same error.
  Could you please check this error and we can discuss the solution.

[akshaysngupta]

@MyronFanQiu For user assigned identity, the field in the put should look like this:

"identity" : {
  "type": "userAssigned",
  "userAssignedIdentities": {
    "<identityResourceId>": {}
  }
}

[AdamFuzumTewelde]

Azure application gateway does support azure keyvault certificate integration, but some provisional settings must be done first.

• Assign managed Identity to the Application gateway using one of the two commands , I prefer the second one.

1. az identity create --name {application gateway name} --resource-group {resource group} --location {service location}
   OR
2. az network application-gateway identity assign --resource-group {resource group}
   --gateway-name {application gateway name} --identity "/subscriptions/{subscription_id/resourceGroups/{resource group}web/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{application gateway name}"

• Enable azure keyvault soft delete
  az resource update --id $(az keyvault show --name {keyvault name}-o tsv | awk '{print $1}') --set properties.enableSoftDelete=true

• allow all networks, or the virtual network which the application gateway belongs to have access to the fire wall
  

• add the application gateway to the access policy

the you can go to you listener and add the keyvault certificate

[AmarReddy522]

Hi @AdamFuzumTewelde ,

I have added AGW and Managed Identity with same name (Name: XXX-ApplicationGateway) and ran the step 2 for assigning the userAssignedIdentity. In the Key Vault attached the access policy as 'get' to the Manged Identity (Name: XXX-ApplicationGateway).

Below is the exception we still see

Error Creating/Updating Application Gateway "XXXX-ApplicationGateway" (Resource Group "RGName"):
network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error:
Code="ApplicationGatewayKeyVaultSecretRequiresUserAssignedIdentity" Message="Application Gateway
'/subscriptions/{subscriptionID}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/applicationGateways
/XXX-ApplicationGateway' requires a 'UserAssigned' Identity with 'get' access policy to the referenced KeyVault. Please provide so by using top level 'Identity' property."

Command:
az network application-gateway identity assign -g RGName --gateway-name XXXX-ApplicationGateway --identity /subscriptions/{subscriptionID}/resourceGroups/{resourceGroupName}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/XXX-ApplicationGateway

Please help on this exception.