Azure-pipelines-tasks: Azure Key Vault task converts %3B in secrets to ;

[coreyasmith]

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: AzureKeyVault@1 (https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks/AzureKeyVaultV1)

Environment

• Server - Azure Pipelines
• Agent - Hosted, windows-2019

Issue Description

The Azure Key Vault task is converting all instances of %3B in a secret to ;.

To reproduce, create a secret in a key vault with %3B in the value somewhere. Use the Azure Key Vault task to download the secret and observe that all instances of %3B in the value are converted to ;.

For the screenshots below I created a secret with the value a%3B:

This issue seems to be isolated to the task because neither the Azure CLI nor the Azure PowerShell commandlets are affected:

Task logs

Pre-job: Azure Key Vault

2020-07-31T15:31:03.1714670Z SubscriptionId: lkjfdsalkjfdsa.
2020-07-31T15:31:03.1716309Z Key vault name: asdfjklasdfjkl.
2020-07-31T15:31:03.1717118Z ##[debug]set SYSTEM_UNSAFEALLOWMULTILINESECRET=true
2020-07-31T15:31:03.1731143Z ##[debug]Processed: ##vso[task.setvariable variable=SYSTEM_UNSAFEALLOWMULTILINESECRET;issecret=false;]true
2020-07-31T15:31:03.1732048Z ##[debug]Downloading selected secrets from subscriptionId: lkjfdsalkjfdsa, vault: asdfjklasdfjkl
2020-07-31T15:31:03.1734255Z ##[debug]Promise for downloading secret value for: TestEncoding
2020-07-31T15:31:03.1743918Z Downloading secret value for: TestEncoding.
2020-07-31T15:31:03.1776021Z ##[debug][POST]https://login.windows.net/***/oauth2/token/
2020-07-31T15:31:03.7384649Z ##[debug][GET]https://asdfjklasdfjkl.vault.azure.net/secrets/TestEncoding?api-version=2016-10-01
2020-07-31T15:31:03.9161950Z ##[debug]SYSTEM_DONOTMASKMULTILINESECRETS=undefined
2020-07-31T15:31:03.9164575Z ##[debug]set TestEncoding=********
2020-07-31T15:31:03.9186635Z ##[debug]Processed: ##vso[task.setvariable variable=TestEncoding;issecret=true;]***
2020-07-31T15:31:03.9191134Z ##[debug]set TestEncoding=********
2020-07-31T15:31:03.9199821Z ##[debug]Processed: ##vso[task.setvariable variable=TestEncoding;issecret=true;]***
2020-07-31T15:31:03.9202240Z ##[debug]task result: Succeeded
2020-07-31T15:31:03.9217134Z ##[debug]Processed: ##vso[task.complete result=Succeeded;]
2020-07-31T15:31:03.9493121Z ##[section]Finishing: Azure Key Vault: asdfjklasdfjkl

Error logs

The task does not produce any errors.

[coreyasmith]

I just tested all hex values in the ASCII table and %5D (converted to ]) is affected as well. No other values seem to be affected.

[sandupetrasco]

It looks like the backtick symbol ( ` ) is also affected by this bug. If there is at least one backtick contained by the secret, we can meet these situations:

• The secret contained in the KeyVault: a`Sa <-> The secret returned by the AzureKeyVaultTask: aSa
• The secret contained in the KeyVault: a`na <-> The secret returned by the AzureKeyVaultTask: a(newline)a
  etc.
  @tauhid621 do you have any updates related to this issue?

[XristophD]

I also do have issues here that seems to be related:
Secrets with the following format are being shortened:

Secret: aaaa$bbbbb@!c#cc
Value returned: aaaa@!c#cc

Seems like $bbbbb is shortened away.

[sandupetrasco]

@XristophD, indeed, we also found out the issue with the dollar sign ($). It looks like the Azure Key Vault task is, somehow, parsing the output.
In order to solve that, we managed to come with a .yml template solution, that has a AzurePowerShell task that uses the Set-AzKeyVaultAccessPolicy and Get-AzKeyVaultSecret commands to have access to the Azure Key Vault and extract all the info. Then, you can simply use the Write-Host "##vso[task.setvariable variable=$($secret.Name);]$secretValueText" command, to set the environment variables.
This solution works as the AzureKeyVault task should, fixing the issues we introduced above 😉:

parameters:
  azureServiceConnection: ""
  azureSubscriptionName: ""
  vaultName: "$(Agent.Name)-kv"
  secretsFilter: "*"

steps:
- powershell: |
    [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
    $azExists = Get-InstalledModule | where {$_.Name -eq "Az"}
    if ($azExists) {
      Write-Host "Az module exists on this machine. Skipping install..."
    } else {
      Write-Host "Installing Az module..."
      Install-Module -Name Az -AllowClobber -Scope CurrentUser -Force
    }
    Get-InstalledModule Az -AllVersions
  displayName: Check and install Az Module

- task: AzurePowerShell@4
  inputs:
    azureSubscription: ${{ parameters.azureServiceConnection }}
    azurePowerShellVersion: latestVersion
    scriptType: inlineScript
    inline: |
      $azureSubscription = "${{ parameters.azureSubscriptionName }}"
      $vaultName = "${{ parameters.vaultName }}"
      $secretsFilter = "${{ parameters.secretsFilter }}"
      Set-AzContext -Subscription $azureSubscription | Out-Null
      $Context = Get-AzContext
      $AzureDevOpsServicePrincipal = Get-AzADServicePrincipal -ApplicationId $Context.Account.Id
      Write-Host "AzureDevOpsServicePrincipal $AzureDevOpsServicePrincipal"
      Write-Host "`nSetting KeyVault Access Policy..."
      Set-AzKeyVaultAccessPolicy -VaultName $vaultName -ObjectId $AzureDevOpsServicePrincipal.Id -PermissionsToSecrets get,list -PassThru
      Write-Host "`nGetting the Secrets to be fetched..."
      [System.Collections.ArrayList] $secrets = @()
      if ($secretsFilter -eq "*") {
        $secretsList = Get-AzKeyVaultSecret -VaultName $vaultName | % { $_.Name }
      } else {
        $secretsList = $secretsFilter.Split(',')
      }
      foreach ($secretItem in $secretsList) {
        $secret = Get-AzKeyVaultSecret -VaultName $vaultName -Name $secretItem -Verbose
        if ($secret) {
          $secrets.Add($secret) > $null
        } else {
          Write-Warning "The secret $secretItem does not exist in vault $vaultName"
        }
      }
      Write-Host "`nGetting secrets from KeyVault into Env variables..."
      foreach ($secret in $secrets) {
        Write-Host "Fetching secret $($secret.Name)"
        $secretValueText = [System.Net.NetworkCredential]::new("", $secret.SecretValue).Password 
        Write-Host "##vso[task.setvariable variable=$($secret.Name);]$secretValueText"
      }
      Write-Host "`nFinished fetching the secrets from KeyVault."
  displayName: 'Get AzureKeyVault Secrets into Environment variables'