Azure-pipelines-tasks: Update to Azure Powershell task breaks AzContext in background job

[PavelPikat]

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: Azure Powershell

Environment

• Server - Azure Pipelines or TFS on-premises?

  Azure Pipelines

  • If using Azure Pipelines, provide the account name, team project name, build definition name/build number:
    Account: amcsgroup
    Project: Routing
    Release definition name: routing_qa3_f0
    ReleaseId: 4612

• Agent - Hosted or Private:
  Hosted

  • If using Hosted agent, provide agent queue name:
    Azure Pipelines

Issue Description

Since yesterday, 23rd of July, Azure Powershell tasks started failing with an error "Your Azure credentials have not been set up or have expired, please run Connect-AzAccount to set up your Azure credentials." when passing AzContext to a background job.

Task logs

[Enable debug logging and please provide the zip file containing all the logs for a speedy resolution]

Troubleshooting

Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Error logs

Reproduction steps:

1. Create a release with Azure PowerShell V4 or V5 task using the following params:
   

2. Use the following reproduction script

$Context = Get-AzContext

$ScriptBlock = {
    param($Azcontext)
    $ErrorActionPreference = "Stop"
try {
    Get-AzKeyVault -VaultName %somename% -ResourceGroupName  %some-rg% -AzureRMContext $Azcontext | Out-Null
    Write-Output "Done."
}
catch
{
Write-Error $_
}
}

@(1, 2, 3) | ForEach-Object {
    Start-Job -Name $_ -ScriptBlock $ScriptBlock -ArgumentList $Context | Out-Null
}
Start-Sleep -Seconds 5
@(1, 2, 3) | ForEach-Object {
    Get-Job -Name $_ | Receive-Job
}

[v-kyljon]

Similar result can be achieved by running AzurePowershellV4 with pwsh on Win2016 or Win2019 images

`trigger:

• none

pool:
vmImage: 'windows-latest'

steps:

• task: AzurePowerShell@5
  inputs:
  azureSubscription: 'TestSub'
  ScriptType: 'InlineScript'
  Inline: |
  $PSVersionTable
  Get-AzWebAppCertificate -ResourceGroupName TestWebApp
  preferredAzurePowerShellVersion: '4.3.0'
  pwsh: true

• task: AzurePowerShell@4
  inputs:
  azureSubscription: 'TestSub'
  ScriptType: 'InlineScript'
  Inline: |
  $PSVersionTable
  Get-AzWebAppCertificate -ResourceGroupName TestWebApp
  azurePowerShellVersion: '4.3.0' # Latest has same result
  pwsh: true`

Results:
`##[debug]STDERR: Get-AzWebAppCertificate: D:\a_temp\1b94dc83-5241-4e64-85ab-bb0b39bcf9a3.ps1:5

[debug]STDERR: Line |

[debug] 5 | Get-AzWebAppCertificate -ResourceGroupName TestWebApp

[debug] | ~~~~~~~~~~~~~

[debug] | No subscription found in the context. Please ensure that the credentials you provided are authorized

[debug] | to access an Azure subscription, then run Connect-AzAccount to login.

[debug]

[debug]

[debug]Exit code: 1

[debug]Leaving Invoke-VstsTool.

[error]Get-AzWebAppCertificate: D:\a_temp\1b94dc83-5241-4e64-85ab-bb0b39bcf9a3.ps1:5

Line |
5 | Get-AzWebAppCertificate -ResourceGroupName TestWebApp
| ~~~~~~~~~~~~~
| No subscription found in the context. Please ensure that the credentials you provided are authorized
| to access an Azure subscription, then run Connect-AzAccount to login.
`
Noted: AzurePowershellV5 uses version 1.9.0 of the Az.accounts module while V4 uses 1.9.1

From the commits around this timeframe I haven't found any changes related to the tasks that could cause these responses but it seems similar to #9319

Since V5 outputs the Tls1.2 setting located in the debug for V4 as well because I wasn't sure if this may have an impact ##[debug]TLS 1.2 already present in session.

'Disable-AzContextAutosave' related?

[chshrikh]

@20shivangi to check this once

[20shivangi]

@PavelPikat Earlier we were using set-azcontext command in the scope of currentUser, due to which all the non-azureps tasks were able to use the Az context so we limited the context to the scope of process. But due to this, some pipelines started breaking where people were using az context created by azureps task.
So for now, we have reverted this change back. Our change has started being deployed, soon it will reach all the rings.

[roofiq]

Hey guys, that's quite a big bug. We had several pipelines running using this configuration and they all are failing now.

E: context is being used in quite a lot of taks.
Even when we are uploading ARM templates to storage account, this command does not work Set-AzCurrentStorageAccount.

Do you have any workaround, that we can apply so context will still be taken using this task? This change breaks our concept.

Thanks,
Rafal

[PavelPikat]

@roofiq We ended up adding something like Save-AzContext -Path C:\context.json and Import-AzContext -Path C:\context.json to get the AzContext inside of a background job.
The first job is guaranteed to fail, so we also added a dummy job that runs first and ignores any exceptions.

[lmoelleb]

Picking up the context in a later step is not the only use-case breaking. An inline script launching another PowerShell.exe fails as well, even though everything is in the same build step. Sure, not the cleanest of things, but not exactly something you expect to break outside a major version update.

[LeonB87]

Not sure if this is exactly the same issue, but our pipeline breaks aswell. we have the following task in our pipeline;

- task: AzurePowerShell@4
  displayName: 'KeyVault'
  inputs:
    azureSubscription: 'azure environment'
    ScriptPath: '$(Pipeline.Workspace)/drop/scripts/Check-KeyVault/Check-KeyVault.ps1'
    ScriptArguments: '-KeyVaultName $(keyvaultname)'
    azurePowerShellVersion: LatestVersion
    pwsh: true

The script does one call to Azure

$keyVault = Get-AzKeyVault -VaultName $keyVaultName

And the pipeline is failing telling us to Run Connect-AzAccount to login.

last time it ran the powershell module was version 4.171.1

[PavelPikat]

@LeonB87 Are you executing the Az cmdlet in a ScriptBlock? If so, the context is lost if it's not passed into the block

[v-kyljon]

Hi All,
I have two workarounds that I have tested so far.

1. Update the task version to 5.* or use AzurePowerShell@5
2. Run on Windows Powershell (set pwsh: false)

[LeonB87]

@PavelPikat The powershell script itself doesn't contain a scriptblock.

param(
    [string][parameter(Mandatory = $true)] $keyVaultName
)

$keyVault = Get-AzKeyVault -VaultName $keyVaultName

if ($keyvault) {
    Write-Host "##vso[task.setvariable variable=keyVaultExists]true"
    Write-Output "Variable for keyvault is set to true".
} else {
    Write-Host "##vso[task.setvariable variable=keyVaultExists]false"
    Write-Output "Variable for keyvault is set to false".
}

[roofiq]

Hey. So if you disable Powershell Core on this task, it will run as before this change. Powershell core is creating another process which is loosing the context and all.
It should defaultly run on Core, so it is implementation bug.

[PavelPikat]

The initial issue I created, including its repro steps use v5 without PowerShell Core, so if anything changed now, then it must be due to some fix. I'll re-test my code again

[LeonB87]

Tested work arounds mentioned by v-kyljon on my side. AzurePowerShell@5 using Core works, aswell as AzurePowerShell@4 Without Core.

[PavelPikat]

Yeah, seems like it's fixed now. The old release still fails, but if I create new deployment using the same task and code, then it seem to work.

[20shivangi]

Yeah, seems like it's fixed now. The old release still fails, but if I create new deployment using the same task and code, then it seem to work.

@PavelPikat Can you let me know the version of task in which issue is not occuring now ?

[chshrikh]

@roofiq I am trying to understand the scenario - what do you mean by "Even when we are uploading ARM templates to storage account,"....which task are you using to upload the template ?

[roofiq]

@chshrikh, nevermind because it all works when disabling Core in v4. But we are uploading ARM templates to storage with Azure PowerShell task so it was all connected :)

[harre096]

I think I'm having this error too. Switching to version 5.* doesn't seem to have helped. The error I get from my background job is

[error]Your Azure credentials have not been set up or have expired, please run Connect-AzAccount to set up your Azure credentials.

[AmrutaKawade]

I think I'm having this error too. Switching to version 5.* doesn't seem to have helped. The error I get from my background job is

[error]Your Azure credentials have not been set up or have expired, please run Connect-AzAccount to set up your Azure credentials.

@harre096 are you still facing this issue? we have reverted our changes.

[harre096]

I think I'm having this error too. Switching to version 5.* doesn't seem to have helped. The error I get from my background job is

[error]Your Azure credentials have not been set up or have expired, please run Connect-AzAccount to set up your Azure credentials.

@harre096 are you still facing this issue? we have reverted our changes.

For my script that uses background jobs and runs in AzureDevops. I have been using 5.* successfully for a while. I tested 4.* today and that seems to be working again too.

[AmrutaKawade]

Thanks for confirming. Closing this issue