Azure-storage-azcopy: RESPONSE Status: 404 The specified resource does not exist.

Created on 8 Jul 2019 · 9Comments · Source: Azure/azure-storage-azcopy

Which version of the AzCopy was used?

Note: The version is visible when running AzCopy without any argument

azcopy version 10.2.1

Which platform are you using? (ex: Windows, Mac, Linux)

Linux

What command did you run?

Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.

azcopy cp 1.txt https://XXXXXXXXXXXXXxx.blob.core.windows.net/XXXXXXXXX?st=2019-07-08T15%3A15%3A04Z&se=2119-07-09T15%3A15%3A00Z&sp=racwl&sv=2018-03-28&sr=c&sig=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxxx

What problem was encountered?

Final Job Status: Failed

azcopy for win 10 works fine with same params

SAS token copied from azure portal (also issued it via storage explorer with same error)

some logs
`
2019/07/08 15:29:43 AzcopyVersion 10.2.1
2019/07/08 15:29:43 OS-Environment linux
2019/07/08 15:29:43 OS-Architecture amd64
2019/07/08 15:29:43 Job-Command cp 1.txt https://XXXXXXXXXXXXXXXxx.blob.core.windows.net/XXXXXXXXXXXXXxx?sv=2018-03-28
2019/07/08 15:29:43 Number of CPUs: 2
2019/07/08 15:29:43 Max file buffer RAM 1.000 GB
2019/07/08 15:29:43 Max open files when downloading: 1048042
2019/07/08 15:29:43 Max concurrent transfer initiation routines: 64
2019/07/08 15:29:43 PERF: primary performance constraint is Unknown. States: R: 0, D: 0, W: 0, F: 0, B: 0, T: 0
2019/07/08 15:29:43 JobID=XXXXXXXXXXXXXXXXXXXXXXXXXXXx, credential type: Anonymous
2019/07/08 15:29:43 scheduling JobID=XXXXXXXXXXXXXXXXXXXXXXXXxx, Part#=0, Transfer#=0, priority=0
2019/07/08 15:29:43 INFO: [P#0-T#0] has worker 29 which is processing TRANSFER
2019/07/08 15:29:43 INFO: [P#0-T#0] Starting transfer: Source "1.txt" Destination "https://XXXXXXXXXXXXXXXXXXXx.blob.core.windows.net/XXXXXXXXXXXXXxx/1.txt?sv=2018-03-28". Specified chunk size 8388608
2019/07/08 15:29:43 ==> OUTGOING REQUEST (Try=1)
PUT https://XXXXXXXXXXXXXXXXXXx.blob.core.windows.net/XXXXXXXXXXXXXXXX/1.txt?sv=2018-03-28&timeout=901
Content-Length: [8]
User-Agent: [AzCopy/10.2.1 Azure-Storage/0.7 (go1.12; linux)]
X-Ms-Blob-Cache-Control: []
X-Ms-Blob-Content-Disposition: []
X-Ms-Blob-Content-Encoding: []
X-Ms-Blob-Content-Language: []
X-Ms-Blob-Content-Md5: []
X-Ms-Blob-Content-Type: [text/plain; charset=utf-8]
X-Ms-Blob-Type: [BlockBlob]
X-Ms-Client-Request-Id: [XXXXXXXXXXXXXXXXXXXXXXXXx]
X-Ms-Version: [2018-03-28]

2019/07/08 15:29:44 ==> REQUEST/RESPONSE (Try=1/790.39998ms, OpTime=790.575224ms) -- RESPONSE SUCCESSFULLY RECEIVED
PUT https://XXXXXXXXXXXXXXXXXxx.blob.core.windows.net/XXXXXXXXXXXXXx/1.txt?sv=2018-03-28&timeout=901
Content-Length: [8]
User-Agent: [AzCopy/10.2.1 Azure-Storage/0.7 (go1.12; linux)]
X-Ms-Blob-Cache-Control: []
X-Ms-Blob-Content-Disposition: []
X-Ms-Blob-Content-Encoding: []
X-Ms-Blob-Content-Language: []
X-Ms-Blob-Content-Md5: []
X-Ms-Blob-Content-Type: [text/plain; charset=utf-8]
X-Ms-Blob-Type: [BlockBlob]
X-Ms-Client-Request-Id: [XXXXXXXXXXXXXXXXXXXXxx]
X-Ms-Version: [2018-03-28]

RESPONSE Status: 404 The specified resource does not exist.
Content-Length: [223]
Content-Type: [application/xml]
Date: [Mon, 08 Jul 2019 15:29:43 GMT]
Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Error-Code: [ResourceNotFound]
X-Ms-Request-Id: [XXXXXXXXXXXXXXXXXXXXx]
X-Ms-Version: [2018-03-28]

2019/07/08 15:29:44 ERR: [P#0-T#0] UPLOADFAILED: 1.txt : 404 : 404 The specified resource does not exist.. When Uploading blob. X-Ms-Request-Id: XXXXXXXXXXXXXXXxx

Dst: https://XXXXXXXXXXXXXxx.blob.core.windows.net/XXXXXXXXXXXXxx/1.txt?sv=2018-03-28
2019/07/08 15:29:44 ==> OUTGOING REQUEST (Try=1)
DELETE https://XXXXXXXXXXXxxxx.blob.core.windows.net/XXXXXXXXXXXXxxx/1.txt?sv=2018-03-28&timeout=30
User-Agent: [AzCopy/10.2.1 Azure-Storage/0.7 (go1.12; linux)]
X-Ms-Client-Request-Id: [XXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
X-Ms-Version: [2018-11-09]

2019/07/08 15:29:44 ==> REQUEST/RESPONSE (Try=1/63.061052ms, OpTime=63.105944ms) -- RESPONSE SUCCESSFULLY RECEIVED
DELETE https://XXXXXXXXXXXXxx.blob.core.windows.net/XXXXXXXXXXXxxx/1.txt?sv=2018-03-28&timeout=30
User-Agent: [AzCopy/10.2.1 Azure-Storage/0.7 (go1.12; linux)]
X-Ms-Client-Request-Id: [XXXXXXXXXXXXXXXXXXxx]
X-Ms-Version: [2018-11-09]

2019/07/08 15:29:44 JobID=XXXXXXXXXXXXXXXXXXXXx, Part#=0, TransfersDone=1 of 1
2019/07/08 15:29:44 all parts of Job XXXXXXXXXXXXXXXXXXXx successfully completed, cancelled or paused
2019/07/08 15:29:45 PERF: primary performance constraint is Unknown. States: R: 0, D: 0, W: 0, F: 0, B: 0, T: 0
2019/07/08 15:29:45

Job XXXXXXXXXXx summary
Elapsed Time (Minutes): 0.0342
Total Number Of Transfers: 1
Number of Transfers Completed: 0
Number of Transfers Failed: 1
Number of Transfers Skipped: 0
TotalBytesTransferred: 0
Final Job Status: Failed

Source

dvdobrovolskiy

Most helpful comment

I am a bit afraid I am doing something simple but stupid thing here. But again it fresh version, pretty clean command line.

Good point. I wonder if its some kind of issue with characters somewhere that are special to the Linux shell. To see if that might be the case, I'd suggest you try this:

Copy the exact command line that works on Windows... but before you run it on Linux ...
Put quotes around the url. Probably single quotes, since they are the strictest form of quoting on Linux. That should fix any special-character issues. (BTW, on Windows I also use quotes, but I used double quotes there)

So you'd end up with this for Linux (note the single quotes around the URL.)

azcopy cp 1.txt 'https://XXXXXXXXXXXXXxx.blob.core.windows.net/XXXXXXXXX?st=2019-07-08T15%3A15%3A04Z&se=2119-07-09T15%3A15%3A00Z&sp=racwl&sv=2018-03-28&sr=c&sig=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxxx'

If that doesn't work, maybe the Storage Firewall possibility is worth looking at. That would only affect you if the Windows and Linux machines have different public IP addresses (i.e. they are not behind the same NAT). Docs for Storage Firewalls are here: https://docs.microsoft.com/en-us/azure/storage/common/storage-network-security

JohnRusk on 10 Jul 2019

👍5

All 9 comments

I'm unable to reproduce the issue at hand, sorry.

If I'm understanding what you're doing correctly, you're trying to copy a single file to blob storage using a container SAS token on Linux, yes?

adreed-msft on 8 Jul 2019

I'm unable to reproduce the issue at hand, sorry.

If I'm understanding what you're doing correctly, you're trying to copy a single file to blob storage using a container SAS token on Linux, yes?

Yes. one little test file (10 bytes). Command line same as on windows (same fresh version).
Might be a can to something to collect more info?

dvdobrovolskiy on 9 Jul 2019

Which Linux distro are you running?

Is there any chance that hierarchical namespace is enabled on the blob account that you are targeting? If so, currently you need to use the "dfs" endpoint, not the "blob" end point. E.g. https://account.dfs.core.windows.net/containername

My other suggestion would be to try it with different container name, just in case there is something odd about the name you have. E.g. make a simple test container, with a simple one-word name all in ASCII characters. E.g. "test"

In any case, the error looks most likely to be a container name issue or a dfs/blob issue.

JohnRusk on 9 Jul 2019

Linux main 4.15.0-50-generic #54-Ubuntu SMP Mon May 6 18:46:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Here enpoiunt from azure portal

BlobEndpoint=https://XXXXXX.blob.core.windows.net/;TableEndpoint=https://XXXXX.table.core.windows.net/;SharedAccessSignature=sv=2018-03-28&ss=b&srt=sco&sp=rwdlac&se=2019-07-09T18:26:56Z&st=2019-07-09T10:26:56Z&spr=https&sig=XXXXXXXX

container name "linuxbackup"

container name "backup"also failed. (azcopy for windows got no probs saving to this containers)

dvdobrovolskiy on 9 Jul 2019

Wow, that is a weird error. As noted, we can't reproduce it here on the AzCopy team, and no-one else has reported it.

Is there any chance that your Storage Account has the storage firewall enabled, with a configuration that would have allowed your Windows machine through but rejected your Linux one? E.g. if the two machines had different public IP addresses, or if one was routed over an Express Route and the other was not. I'm not sure if that would produce this type of error, but maybe it would.

JohnRusk on 10 Jul 2019

sorry, never heard about such a feature. Please advise where to check it.

I issued SAS with out ip restrictions (on windows machine works fine with ip filter).

Plus in logs lots of reports of successful connection with azure servers

I am a bit afraid I am doing something simple but stupid thing here. But again it fresh version, pretty clean command line.

dvdobrovolskiy on 10 Jul 2019

I am a bit afraid I am doing something simple but stupid thing here. But again it fresh version, pretty clean command line.

Good point. I wonder if its some kind of issue with characters somewhere that are special to the Linux shell. To see if that might be the case, I'd suggest you try this:

Copy the exact command line that works on Windows... but before you run it on Linux ...
Put quotes around the url. Probably single quotes, since they are the strictest form of quoting on Linux. That should fix any special-character issues. (BTW, on Windows I also use quotes, but I used double quotes there)

So you'd end up with this for Linux (note the single quotes around the URL.)

azcopy cp 1.txt 'https://XXXXXXXXXXXXXxx.blob.core.windows.net/XXXXXXXXX?st=2019-07-08T15%3A15%3A04Z&se=2119-07-09T15%3A15%3A00Z&sp=racwl&sv=2018-03-28&sr=c&sig=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxxx'

JohnRusk on 10 Jul 2019

👍5

Thanks John, second option did the thing. I was a pretty stupid simple thing.

Sorry for wasting your time.

special-character issues was in sig=.
XXXXXXXXXXXXXxx was very simple single English words

dvdobrovolskiy on 11 Jul 2019

No problem, glad it's fixed for you!

JohnRusk on 11 Jul 2019

Was this page helpful?

0 / 5 - 0 ratings