Azure-storage-azcopy: Transfert with Azure Stack 1901 Storage accounts is not working

[jbpaux]

Which version of the AzCopy was used?

Note: The version is visible when running AzCopy without any argument

AzCopy 10.0.8-Preview

Which platform are you using? (ex: Windows, Mac, Linux)

Windows

What command did you run?

Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.

azcopy cp ".\WindowsTH-RSAT_WS_1709-x64.msu" "https://storageaccount.blob.region.azurestack.external/updates?sv=2015-04-05&ss=bqt&srt=sco&sp=rwdlacup&se=2019-03-08T16:34:18Z&st=2019-03-08T08:34:18Z&spr=https&sig=XXXXXXX"

What problem was encountered?

Job d8dfe284-2d69-a849-663b-e1ebe3a94db1 summary
Elapsed Time (Minutes): 0.0333
Total Number Of Transfers: 1
Number of Transfers Completed: 0
Number of Transfers Failed: 1
Number of Transfers Skipped: 0
TotalBytesTransferred: 0
Final Job Status: Failed
In the logs on the PUT request :
RESPONSE Status: 400 The value for one of the HTTP headers is not in the correct format.

How can we reproduce the problem in the simplest way?

Use azcopy with an azure stack storage account

Have you found a mitigation/solution?

Use an old version of azcopy as instructed on https://docs.microsoft.com/en-us/azure/azure-stack/user/azure-stack-storage-transfer#azcopy

This preview version should allow transfert with azure stack appliance to truly have the "hybrid" promise of Microsoft. I guess this is relative ou API level of Storage RP that are different. Maybe add an argument on azcopy to specify if it's a stack and api level should be downgraded.
Thank you

[zezha-msft]

Hi @jbpaux, thanks for reaching out! We appreciate your feedback.

We do support the latest Azure Stack with service version "2018-03-28", due to the version of Storage SDK that we use. Is that not the version you are using?

If "2018-03-28" is indeed the version that you are using, could you please send me the log file (the SAS signatures are scrubbed to protect your credential)? Thank you!

[jbpaux]

Azure Stack (version 1901) only support older API version as explained here : https://docs.microsoft.com/en-us/azure/azure-stack/user/azure-stack-profiles-azure-resource-manager-versions We cannot target more recent API.
When looking on the Stack itself, we have the following API latest version:
az provider list --query "[?namespace=='Microsoft.Storage'].resourceTypes[].[resourceType,apiVersions[0]]" --output table

ResourceType | Latest API Version
------------ | --------------------
checkNameAvailability | 2017-10-01
locations | 2017-10-01
locations/quotas | 2017-10-01
operations | 2017-10-01
storageaccounts | 2017-10-01
storageAccounts/blobServices | 2017-10-01
storageAccounts/queueServices | 2017-10-01
storageAccounts/tableServices | 2017-10-01
usages | 2017-10-01

[zezha-msft]

Hi @jbpaux, thanks for the info!

We are limited by the Go Blob SDK whose earliest stable preview version only supports 2018-03-28. But you bring up a great point, since in practice a lot of Azure Stack users might be stuck with an older version. However, the risk in letting users overwrite the service version AzCopy uses is that SDKs are only tested for the specific service version they support, not arbitrary older versions of the service.

@seguler @artemuwka what do you think?

[jbpaux]

Yes I know :/ Azure Stack is usually 1 year behind regarding api versions. Would be great to think about us :D Event AzCopy v8 is not compatible. I have to install a 7.3 version to make it work. I was hopping v10 would unify that. You should talk to Azure Stack PM too.

[zezha-msft]

Hi @jbpaux, in 10.1.0 we've added limited support for Azure Stack via an environment variable that could override the service version. It's called AZCOPY_DEFAULT_SERVICE_API_VERSION and you could see its current value in ./azcopy env.

I've tested with an Azure Stack instance that has the version 2017-11-09.