Azure-cli: `az vm create --secrets` fails with "not a properly encoded Base64 string"

[neilpeterson]

Description

I am using the following commands to create a keyvault, a key value pair secret, and then a VM.

keyvault=mykeyvault007
keyvaultrg=testkvrg007
vmFront=vmfront

az group create --name $keyvaultrg --location westus

az keyvault create --name $keyvault --resource-group $keyvaultrg --enabled-for-deployment
az keyvault secret set --vault-name $keyvault --name 'sqlpassword' --value 'Password12'

secret=$(az keyvault secret list-versions --vault-name $keyvault --name sqlpassword --query "[?attributes.enabled].id" --output tsv)
vm_secret=$(az vm format-secret --secret "$secret")

az vm create --resource-group $keyvaultrg --name $vmFront --image UbuntuLTS --generate-ssh-keys --custom-data cloud-init-front.txt --secrets "$vm_secret"

The secret is successfully returned and formatted using az vm format-secret.

[ { "sourceVault": { "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/testkvrg007/providers/Microsoft.KeyVault/vaults/mykeyvault007" }, "vaultCertificates": [ { "certificateUrl": "https://mykeyvault007.vault.azure.net/secrets/sqlpassword/8e531e8ea9e948e6a8c0b81b6b4977b2" } ] } ]

However the following is thrown:

At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details. {
  "status": "Failed",
  "error": {
    "code": "ResourceDeploymentFailure",
    "message": "The resource operation completed with terminal provisioning state 'Failed'.",
    "details": [
      {
        "code": "CertificateImproperlyFormatted",
        "message": "The secret retrieved from https://mykeyvault007.vault.azure.net/secrets/sqlpassword/8e531e8ea9e948e6a8c0b81b6b4977b2 is not a properly encoded Base64 string."
      }
    ]
  }
}  Correlation ID: 3f9d790c-ab44-4670-b0a9-d2068487d47e

I see that the az keyvault secret set command has an --encoding argument, however this only applies when providing a file, which I am not.

I can reproduce the issue on a Macbook Pro.

---

Environment summary

Install Method: How did you install the CLI? (e.g. pip, interactive script, apt-get, Docker, MSI, nightly)
apt-get

CLI Version: What version of the CLI and modules are installed? (Use az --version)

azure-cli (2.0.3)

acr (2.0.1)
acs (2.0.3)
appservice (0.1.3)
batch (2.0.1)
cloud (2.0.1)
component (2.0.1)
configure (2.0.3)
container (0.1.3)
core (2.0.3)
documentdb (0.1.3)
feedback (2.0.1)
find (0.0.2)
iot (0.1.3)
keyvault (2.0.1)
lab (0.0.2)
monitor (0.0.2)
network (2.0.3)
nspkg (2.0.0)
profile (2.0.3)
redis (0.2.0)
resource (2.0.3)
role (2.0.2)
sql (2.0.1)
storage (2.0.3)
vm (2.0.3)

Python (Linux) 2.7.6 (default, Oct 26 2016, 20:30:19)
[GCC 4.8.4]

OS Version: What OS and version are you using?
WSL / Windows 10

Shell Type: What shell are you using? (e.g. bash, cmd.exe, Bash on Windows)
Bash on Windows

[tjprescott]

@neilpeterson I believe this command was intended for passing the private portion of a certificate (which is stored as a secret in KeyVault) into a VM. @devigned can confirm.

Is there a reason you want to save the secret value onto the VM instead of just having your VM retrieve the value from KeyVault?

[neilpeterson]

Thanks, I had considered that as the case, but it is not super clear from existing docs / help.

I am trying to use cloud-init to configure an app at VM deployment time. I would like to grab a secret from keyvault and use this in a configuration file / mysql connection string. Beyond building logic into the app for key retrieval, is there another option?

Thanks

[devigned]

@tjprescott to be accurate, it's both, the private and the public PEM within the /var/lib/waagent/ directory of the VM.

I too would have liked to have string based secrets added a VM, but alas, it's not possible today (cc: @squillace @sauryadas). It would be even cooler to be able to provide a file name for the secret.

As of right now, the path you can go down is to provision a certificate to a VM and Azure CLI (or Azure PowerShell) and use that credential to authenticate as a service principal. After using this non-interactive authentication, you can use Azure CLI from that machine to access any secrets you desire.

See this script for an example: https://github.com/devigned/level1/blob/master/scripts/provision.sh

The above script was built using the master branch as of today. Some of the features will land in the nightly builds tomorrow.

[devigned]

In that case, @neilpeterson, I have something else for you. The cloud-init that goes with the above script: https://github.com/devigned/level1/blob/master/scripts/cloud-config-template.yml

Hope that helps!

[neilpeterson]

Thanks @devigned, just checking that out. To confirm, in this case the CLI would be installed on the targeted VM and then something like az keyvault show would be used to return the secret value?

Thanks to both for the quick response +@tjprescott

[devigned]

@neilpeterson, yes, precisely. The cloud-init installs the CLI and pre-authenticates with the Key Vault cert via vm-secrets.

The deploy user should be ready to execute any command against the subscription it was provisioned.

If one would like to limit the RBAC privileges of the service principal, then you can specify the roles on az ad sp create-for-rbac via --scopes. Default scope is contributor to the subscription.

[devigned]

@tjprescott perhaps, az vm format-secret --secret "$secret" should fail if the secret is not a certificate. Thoughts? It would be nicer than waiting for the VM to fail provisioning.

[neilpeterson]

@devigned thanks a bunch.

This issue can be closed. Is there a proper way to enter a feature request to plump secrets into the az vm create --secrets argument?

[devigned]

@neilpeterson I'd suggest just creating a new issue, describe your scenario, and how you'd like to see it implemented. Perhaps, also link to this issue for some background. Thanks!

[tjprescott]

@devigned if the command is only intended for importing certificates, then perhaps it would make most sense to change the name to "format-certificate", have it accept KeyVault and certificate name and then it gets the "secret" for you.

[devigned]

Seems reasonable. Perhaps, this calls for a feature discussion.

[squillace]

+1

[rbigeard]

Should I understand form this discussion that the current example sitting at https://docs.microsoft.com/fi-fi/azure/virtual-machines/linux/tutorial-secure-web-server is actually misleading and should be amended? I am getting the exact same error as OP by following it (which is not surprising after reading this thread).

(edited): the same error occurs using an openssl generated certificate that has been imported into vault. using a certificate generated by 'az keyvault certificate create' is fine. The documentation is not super clear on those encoding issues.