Azure-cli: "az storage blob upload-batch" no longer works

[patrickgalbraith]

Describe the bug

Using mcr.microsoft.com/azure-cli:latest fails with "No such file or directory: '$web'". However using mcr.microsoft.com/azure-cli:2.9.1 works.

To Reproduce

Using mcr.microsoft.com/azure-cli:latest

$ az storage blob upload-batch -s ./build -d '$web' --account-name $AZURE_STORAGE_ACCOUNT_NAME --account-key $AZURE_STORAGE_ACCESS_KEY

The command failed with an unexpected error. Here is the traceback:
 [Errno 2] No such file or directory: '$web'
 Traceback (most recent call last):
   File "/usr/local/lib/python3.6/site-packages/knack/cli.py", line 215, in invoke
     cmd_result = self.invocation.execute(args)
   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 654, in execute
     raise ex
   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 718, in _run_jobs_serially
     results.append(self._run_job(expanded_arg, cmd_copy))
   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 709, in _run_job
     cmd_copy.exception_handler(ex)
   File "/usr/local/lib/python3.6/site-packages/azure/cli/command_modules/storage/__init__.py", line 325, in new_handler
     raise ex
   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 688, in _run_job
     result = cmd_copy(params)
   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 325, in __call__
     return self.handler(*args, **kwargs)
   File "/usr/local/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 765, in default_command_handler
     return op(**command_args)
   File "/usr/local/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 305, in storage_blob_upload_batch
     if_none_match=if_none_match, timeout=timeout)
   File "/usr/local/lib/python3.6/site-packages/azure/cli/command_modules/storage/util.py", line 225, in wrapper
     return True, func(*args, **kwargs)
   File "/usr/local/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 281, in _upload_blob
     return upload_blob(*args, **kwargs)
   File "/usr/local/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 451, in upload_blob
     return type_func[blob_type]()
   File "/usr/local/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 444, in upload_block_blob
     return client.create_blob_from_path(**create_blob_args)
   File "/usr/local/lib/python3.6/site-packages/azure/multiapi/storage/v2018_11_09/blob/blockblobservice.py", line 453, in create_blob_from_path
     count = path.getsize(file_path)
   File "/usr/local/lib/python3.6/genericpath.py", line 50, in getsize
     return os.stat(filename).st_size
 FileNotFoundError: [Errno 2] No such file or directory: '$web'

Expected behavior

Command to finish successfully.

Environment summary

Docker image sha256:55a65eee1172e112a466d82671410fd2b3d1bed4f7c144e4f375dca2f2c7347f for mcr.microsoft.com/azure-cli

[najuste]

Having the same issue, thanks for posting. Checking your suggestion for using diff image

[PaulBunker]

I'm also having the same issue, I have reverted to using vs code plugin to deploy since delete task nuked my site.

[Woap]

Same issue here

[andrehaupt]

Same here

[FUGIFA]

Same here.

[yonzhan]

add to S174

[akhilrajmailbox]

same here

[fenfenzhong92]

same here

[bacongobbler]

add to S174

Hi @yonzhan. Looking at that milestone, it is planned to be released on August 25th (3 weeks from now).

Code complete: 08/19/2020 Build to Cloud Shell: 08/21/2020 Official Release: 08/25/2020

Is that an accurate timeline here? I'm trying to determine whether I should submit a patch to revert back to v2.9.1 or wait for a hotfix, as this is a major blocker for those using Azure Storage for static website hosting.

Thanks!

[goosseno]

Having the same issue here. FileNotFoundError: [Errno 2] No such file or directory:

[GraemeMalcolm]

Me too - duplicate bug: https://github.com/Azure/azure-cli/issues/14648

[kpm-at-hfi]

same here

[kpm-at-hfi]

Still failing for me. In case I'm doing something wrong, here's what I'm doing as a GitHub Action:

~~~yaml
- name: Login to Azure
uses: Azure/login@v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}

build stuff

- name: Publish app
  uses: Azure/[email protected]
  with:
    azcliversion: latest
    inlineScript: |
      az storage blob upload-batch -s $GITHUB_WORKSPACE/dist -d \$web --account-name ${{ secrets.STORAGE_ACCOUNT_NAME }}

~~~

This was working at Mon, 03 Aug 2020 19:28:44 GMT.

[trendoid]

I had the same issue. 2.9.1 worked but latest didn't

https://github.com/Azure/cli/issues/19

Failed:

name: Publish styles
uses: Azure/[email protected]
with:
azcliversion: latest
inlineScript:

az storage blob upload-batch -s $GITHUB_WORKSPACE/.out-styles -d ui-component-library/styles/${{ steps.publishVersion.outputs.version }} --account-name ${{ secrets.STORAGE_ACCOUNT_NAME }} --account-key ${{ secrets.STORAGE_ACCOUNT_KEY }}

Worked

name: Publish styles
uses: Azure/[email protected]
with:
azcliversion: 2.9.1
inlineScript:

[bacongobbler]

Posting here for those looking for a workaround with the azure-blob-storage-upload github action:

name: Upload To Azure Blob Storage
on:
  push:
    branches:
      - master
jobs:
  upload:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: bacongobbler/azure-blob-storage-upload@main
        with:
          source_dir: dist
          container_name: $web
          connection_string: ${{ secrets.ConnectionString }}
          sync: false

https://github.com/bacongobbler/azure-blob-storage-upload/commit/79465224a1ed32bc66105a825839e9f71a544fa0 was the fix reported by a few community members (thanks to @linkeshkanna for reporting the bug and following up with confirmation on a fix).

Changing azcliversion to 2.9.1 as @trendoid suggested also works for the Azure/cli action.

[kpm-at-hfi]

Thanks @trendoid and @bacongobbler... I had wrongly tried updating the image tag (not the azcliversion) to 2.9.1.

[lamaks]

@yonzhan any ETA to provide the fix or workaround for .sh script?

[akhilrajmailbox]

The latest azure-cli version have the issue, try to use the old version, it solved the issue for me, I revert back to version: 2.1.0
If you don't have any specific dependencies on the azure-cli latest version, then I try to use the old version, that may help

for ubuntu 18 : azure-cli=2.1.0-1~bionic

[patrickgalbraith]

@akhilrajmailbox There should be a 2.9.1 package I would try azure-cli=2.9.1-1~bionic.

The issue just seems to be with 2.10+.

[danmana]

It worked after switching my gitlab build from

deploy:
  image: microsoft/azure-cli

to

deploy:
  image: mcr.microsoft.com/azure-cli:2.9.1

I didn't even notice until now that I was using an older docker image microsoft/azure-cli that was moved to mcr.microsoft.com/azure-cli

[Juliehzl]

Hi @patrickgalbraith, the fix is done in attached PR. Currently you could find artifacts here https://dev.azure.com/azure-sdk/public/_build/results?buildId=484329&view=artifacts&type=publishedArtifacts with the fix and we will release the hotfix version soon.

[lgmorand]

@danmana like you I was using the oldest docker image but since we are impacted by the CLI, it seems that even the oldest one has been updated with the broken CLI version isn't it ?

[danmana]

@lgmorand yes, it looks like there was an update also on the old docker image microsoft/azure-cli. The latest tag was updated 2 days ago, the other tags are all 1 year old ...
I switched to mcr.microsoft.com/azure-cli:2.9.1 with no side effects

[yonzhan]

We will let you know after the hotfix version is ready.

[lgmorand]

@yonzhan it should be interesting to understand why someone updated the old docker image at the same time. it should be definitly frozen (or now, fixed and then frozen)

[yonzhan]

Add @fengzhou-msft to help with docker image.

[Juliehzl]

Hi all,
We just released hotfix version 2.10.1 and please see https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest for installation.
Homebrew for MacOS will be available tomorrow after the PR merged.

[lgmorand]

Could you please tell us when the docker image will also be updated and thus updated.

thanks for the quick fix

[fengzhou-msft]

@lgmorand we have updated the docker image with the latest tag, you can also use the version tag mcr.microsoft.com/azure-cli:2.9.1

[lgmorand]

Thanks, but I couldn't use the tag 2.9.1, because I'm using the github action from someone else who relies on the latest tag. but you fixed it and now everything is working again 👍

[bacongobbler]

To clarify, this issue has been mitigated, and a fix has been made available in the latest release of the Azure CLI. If that assumption is correct, should this ticket be closed, or are there further action items needed to be taken here by the Azure CLI devs preventing this ticket from being closed?

[lgmorand]

IMHO, it can be closed as we just need to take the latest version of the CLI (and because docker image has also been updated)