Azure-pipelines-tasks: Getting 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage"

[kurt-hardy]

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: AzCopy.exe exited with non-zero exit code while uploading files to blob storage

Environment

Azure Pipelines
awsmtech -> SUAZ-Brochure -> Release-7

Agent type - Hosted Agent
Agent pool - Azure pipelines.
Machine name - 'WIN-D10U51A55TG'

Issue Description

AzureBlob File Copy not working after moving repository to a different account.
Tried versions 2, 3 and 4.

Task logs

Same error every time:

2020-10-19T15:01:20.9521504Z ##[error]Upload to container: '$web' in storage account: 'developmentstoragesuazbr' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme
2020-10-19T15:01:21.0371977Z ##[section]Finishing: AzureBlob File Copy

Full out of task

2020-10-19T15:00:30.5360288Z ##[section]Starting: AzureBlob File Copy
2020-10-19T15:00:30.5592006Z ==============================================================================
2020-10-19T15:00:30.5592430Z Task         : Azure file copy
2020-10-19T15:00:30.5592776Z Description  : Copy files to Azure Blob Storage or virtual machines
2020-10-19T15:00:30.5593097Z Version      : 3.175.3
2020-10-19T15:00:30.5593371Z Author       : Microsoft Corporation
2020-10-19T15:00:30.5593764Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-file-copy
2020-10-19T15:00:30.5594730Z ==============================================================================
2020-10-19T15:00:33.6443861Z ##[command]Import-Module -Name C:\modules\azurerm_6.13.1\AzureRM\6.13.1\AzureRM.psd1 -Global
2020-10-19T15:01:05.6166860Z ##[command]Clear-AzureRmContext -Scope Process
2020-10-19T15:01:06.2328619Z ##[command]Clear-AzureRmContext -Scope CurrentUser -Force -ErrorAction SilentlyContinue
2020-10-19T15:01:07.0778275Z ##[command]Add-AzureRMAccount -ServicePrincipal -Tenant *** -Credential System.Management.Automation.PSCredential -Environment AzureCloud
2020-10-19T15:01:09.8358573Z ##[command] Set-AzureRmContext -SubscriptionId cebbdf78-436c-4635-a408-02bd5ab3b1bf -TenantId ***
2020-10-19T15:01:18.7205051Z Uploading files from source path: 'D:\a\r1\a\SUAZ\SUAZ-Brochure-Development' to storage account: 'developmentstoragesuazbr' in container: '$web' with blob prefix: ''
2020-10-19T15:01:18.7560431Z ##[command] & "AzCopy\AzCopy.exe" /Source:"D:\a\r1\a\SUAZ\SUAZ-Brochure-Development" /Dest:"https://developmentstoragesuazbr.blob.core.windows.net/`$web" /@:"D:\a\_temp\fc47a77b-f422-4e73-ac4f-e78a8ad3785e" /XO /Y /SetContentType /Z:"AzCopy" /V:"AzCopy\AzCopyVerbose_4d16913a-cff5-45a1-a420-df419b3cc836.log" /S
2020-10-19T15:01:19.4989817Z [2020/10/19 15:01:19][ERROR] Error parsing source location "D:\a\r1\a\SUAZ\SUAZ-Brochure-Development": Failed to enumerate directory D:\a\r1\a\SUAZ\SUAZ-Brochure-Development\ with file pattern *. The system cannot find the path specified. (Exception from HRESULT: 0x80070003) For more details, please type "AzCopy /?:Source" or use verbose option /V.
2020-10-19T15:01:19.6446932Z ##[command]Disconnect-AzureRmAccount -Scope Process -ErrorAction Stop
2020-10-19T15:01:20.1788652Z ##[command]Clear-AzureRmContext -Scope Process -ErrorAction Stop
2020-10-19T15:01:20.9521504Z ##[error]Upload to container: '$web' in storage account: 'developmentstoragesuazbr' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme
2020-10-19T15:01:21.0371977Z ##[section]Finishing: AzureBlob File Copy

[ojasp]

I had the same issue, and after adding the pipeline user to the _Storage Blob Data Owner_ & _Owners_ role I was able to upload a file to the blob.

[kurt-hardy]

I'm using an ARM service connection and its created by me. When I look at the service connection role, its set to contributor which (I believe) is higher than the Storage owner right?

Thanks for your help

[ojasp]

The only thing that the Contributor role does not do is allow the user to assign roles in Azure RBAC, that is something that is allowed for the Owner role. AzCopy task may internally check to see if the user has particular access. You might be able to see exactly where the task fails by setting _System.Debug=true_ in the pipeline variable and setting _--log-level=DEBUG_ in the optional arguments section in this task.

Anyways, in this case you have to explicitly add the Storage owner role. I ended up also assigning Owner role for the pipeline user to this storage account as well because it is all inclusive, and in my use case, the storage account is only being used by this task.

[kurt-hardy]

Woo. Thank you for that! System.Debug=true is a very neat trick! Turns out it was someone putting an underscore on the alias and me not being able to read! But that sorted it. Thanks

[Frederic-roamler]

I had the issue with version 4, changing it back to v2 solved it for me

[wadeem]

For me it was solved with version 4, as described here:

https://brettmckenzie.net/2020/03/23/azure-pipelines-copy-files-task-authentication-failed/