Azure-storage-azcopy: SAS only give 403

[Jaurocha]

I'm trying to use AzCopy V10 to copy a file from blobstorage to local disk on Windows server 2012R2 using sas as authentication. It does not work I only get 403

RESPONSE Status: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

So I try to list because it is a command with fewer parameters.
azcopy.exe list "https://name.blob.core.windows.net/dbdaily?sv=2018-03-28&ss=bfqt&srt=sco&sp=rwdlacup&se=2050-03-07T22:06:19Z&st=2019-03-06T14:06:19Z&spr=https&sig=removed"

Still the same problem.

I have tried sas generated in AzurePortal and Azure storage explorer. I have tried to regenerate the sas in several ways.

Is there a problem with using sas or is do I have to create the token in a special way?
I have seen other that have the same problem with sas but have found no answer.

[zezha-msft]

Hi @Jaurocha, thanks for reaching out!

Your SAS maybe invalid, have you tried generating a container SAS instead? (In Storage Explorer, right-click on a container and select "Get Shared Access Signature")

[Jaurocha]

Hi. Have tried both file and container.

I did another test where I generated an url from AzureExplorer by right click on a file and click Get Shared Access Signature. I copied the url with sas and pasted it into Edge and got a file and started to download no problems.

Then I took the same url with sas and tried it in azcopy like this
azcopy.exe cp "https://name.blob.core.windows.net/dbdaily/backup_2019_03_05_000501_1084268.bak?st=2019-03-07T07%3A43%3A00Z&se=2019-03-09T07%3A43%3A39Z&sp=rl&sv=2018-03-28&sr=b&sig=removed" "h:\DbBkpDownload" --recursive

The result is 403
_RESPONSE Status: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature._

I also think I get another problem because the --recursive should not be needed sense I download one single file. But if I leave it out I get the error message

_failed to perform copy command due to error: cannot start job due to error: cannot download the enitre container / virtual directory. Please use --recursive flag._

But I assume that this is a false error message due to the 403 being misinterpreted.

[zezha-msft]

Hi @Jaurocha, thanks for getting back to us!

I couldn't reproduce your issue unfortunately:

.\azcopy.exe cp "https://zemaintest.blob.core.windows.net/test3/Review-doc/test.bak?s
t=2019-03-11T21%3A30%3A00Z&se=2019-03-12T21%3A30%3A00Z&sp=rl&sv=2018-03-28&sr=b&sig=removed" C:\Use
rs\zezha\Downloads\
Scanning...

Job f343a942-06ba-894a-7c5e-ca2df8dc603e has started
Log file is located at: C:\Users\zezha/.azcopy/f343a942-06ba-894a-7c5e-ca2df8dc603e.log

0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total,


Job f343a942-06ba-894a-7c5e-ca2df8dc603e summary
Elapsed Time (Minutes): 0.0334
Total Number Of Transfers: 1
Number of Transfers Completed: 1
Number of Transfers Failed: 0
Number of Transfers Skipped: 0
TotalBytesTransferred: 15537152
Final Job Status: Completed

Which version of the tool are you using? Please try the latest version if you are not using that one:
.\azcopy.exe --version azcopy version 10.0.8-Preview

[Jaurocha]

Hi @zezha-msft.
I'm using 10.0.8

Is it possible to get a test URL from somewhere so that I can test with a file that you know to work?

[zezha-msft]

@Jaurocha, sure. I simply clicked on the blob in Storage Explorer and generated a SAS with the default permissions and extended the validity to a month.

./azcopy cp "https://zemaintest.blob.core.windows.net/demo1/test_1kb_blob_sync.txt?st=2019-03-29T00%3A21%3A25Z&se=2019-04-30T00%3A21%3A00Z&sp=rl&sv=2018-03-28&sr=b&sig=wl1vewck5Pq%2B1%2B69%2BbG9hRvypmYKZyqBCXKXNFfca8A%3D" ~/work/test

[zezha-msft]

@Jaurocha please let me know if you have any other question.

[Jaurocha]

@zezha-msft
Hi I downloaded the latest version and tried to download your file but got the same errors.
I include them below.

First try
azcopy.exe cp "https://zemaintest.blob.core.windows.net/demo1/test_1kb_blob_sync.txt?st=2019-03-29T00A21A25Z&se=2019-04-30T00A21A00Z&sp=rl&sv=2018-03-28&sr=b&sig=wl1vewck5PqB1B69BbG9hRvypmYKZyqBCXKXNFfca8AD" "h:\DbBkpDownload"
INFO: Scanning...
INFO: source is not validated as a single file: AuthenticationFailed, please check if SAS or OAuth is used properly, or source is a public blob
INFO: trying to copy the source as container/directory/list of files

failed to perform copy command due to error: cannot start job due to error: cannot download the enitre container / virtual directory. Please use --recursive flag.

Second try, added recursive
azcopy.exe cp "https://zemaintest.blob.core.windows.net/demo1/test_1kb_blob_sync.txt?st=2019-03-29T00A21A25Z&se=2019-04-30T00A21A00Z&sp=rl&sv=2018-03-28&sr=b&sig=wl1vewck5PqB1B69BbG9hRvypmYKZyqBCXKXNFfca8AD" "h:\DbBkpDownload" --recursive
INFO: Scanning...
INFO: source is not validated as a single file: AuthenticationFailed, please check if SAS or OAuth is used properly, or source is a public blob
INFO: trying to copy the source as container/directory/list of files
failed to perform copy command due to error: cannot start job due to error: cannot list blobs for download. Failed with error -> github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/azblob.NewResponseError, /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/azblob/zz_generated_response_error.go:29
===== RESPONSE ERROR (ServiceCode=AuthenticationFailed) =====
Description=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:fb55e6ec-901e-0046-26ae-ee11bd000000
Time:2019-04-09T08:28:05.1966585Z, Details:
AuthenticationErrorDetail: Signature size is invalid
Code: AuthenticationFailed
GET https://zemaintest.blob.core.windows.net/demo1?comp=list&include=metadata&prefix=test_1kb_blob_sync.txt%2F&restype=container&sig=REDACTED&sp=rl&sr=b&sv=2018-03-28&timeout=901
User-Agent: [AzCopy/10.0.9-Preview Azure-Storage/0.6 (go1.10.3; Windows_NT)]
X-Ms-Client-Request-Id: [d1bb1252-cc9f-4ffb-5f0a-d4b73bbcd1a5]
X-Ms-Version: [2018-03-28]

RESPONSE Status: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
Content-Length: [400]
Content-Type: [application/xml]
Date: [Tue, 09 Apr 2019 08:28:04 GMT]
Server: [Microsoft-HTTPAPI/2.0]
Vary: [Origin]
X-Ms-Error-Code: [AuthenticationFailed]
X-Ms-Request-Id: [fb55e6ec-901e-0046-26ae-ee11bd000000]

[zezha-msft]

Sorry @Jaurocha, I accidentally deleted the blob.

./azcopy cp "https://zemaintest.blob.core.windows.net/demo1/test_1kb_blob_sync.txt?st=2019-04-09T18%3A08%3A52Z&se=2019-05-10T18%3A08%3A00Z&sp=rwdl&sv=2018-03-28&sr=b&sig=gLJ8ZIekS3Q9DD7t5bQDUrFbqiSf8LOdGRQCplsgqU0%3D" ~/work/test
INFO: Scanning...

Job cc78b1fb-b77c-ea4a-7030-a8111987ab90 has started
Log file is located at: /Users/zed/.azcopy/cc78b1fb-b77c-ea4a-7030-a8111987ab90.log

0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total,


Job cc78b1fb-b77c-ea4a-7030-a8111987ab90 summary
Elapsed Time (Minutes): 0.0334
Total Number Of Transfers: 1
Number of Transfers Completed: 1
Number of Transfers Failed: 0
Number of Transfers Skipped: 0
TotalBytesTransferred: 1024
Final Job Status: Completed

[Jaurocha]

@zezha-msft
Same result with the new file.

First try
H:>azcopy.exe cp "https://zemaintest.blob.core.windows.net/demo1/test_1kb_blob_sync.txt?st=2019-04-09T18A08A52Z&se=2019-05-10T18A08A00Z&sp=rwdl&sv=2018-03-28&sr=b&sig=gLJ8ZIekS3Q9DD7t5bQDUrFbqiSf8LOdGRQCplsgqU0D" "h:\DbBkpDownload"
INFO: Scanning...
INFO: source is not validated as a single file: AuthenticationFailed, please check if SAS or OAuth is used properly, or source is a public blob
INFO: trying to copy the source as container/directory/list of files

failed to perform copy command due to error: cannot start job due to error: cann
ot download the enitre container / virtual directory. Please use --recursive fla
g.

Second try, added recursive
H:>azcopy.exe cp "https://zemaintest.blob.core.windows.net/demo1/test_1kb_blob_sync.txt?st=2019-04-09T18A08A52Z&se=2019-05-10T18A08A00Z&sp=rwdl&sv=2018-03-28&sr=b&sig=gLJ8ZIekS3Q9DD7t5bQDUrFbqiSf8LOdGRQCplsgqU0D" "h:\DbBkpDownload" --recursive
INFO: Scanning...
INFO: source is not validated as a single file: AuthenticationFailed, please check if SAS or OAuth is used properly, or source is a public blob
INFO: trying to copy the source as container/directory/list of files

failed to perform copy command due to error: cannot start job due to error: cann
ot list blobs for download. Failed with error -> github.com/Azure/azure-storage-
azcopy/vendor/github.com/Azure/azure-storage-blob-go/azblob.NewResponseError, /g
o/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storag
e-blob-go/azblob/zz_generated_response_error.go:29
===== RESPONSE ERROR (ServiceCode=AuthenticationFailed) =====
Description=Server failed to authenticate the request. Make sure the value of Au
thorization header is formed correctly including the signature.
RequestId:21779942-601e-0058-7872-f0cb50000000
Time:2019-04-11T14:26:33.8387172Z, Details:
AuthenticationErrorDetail: Signature size is invalid
Code: AuthenticationFailed
GET https://zemaintest.blob.core.windows.net/demo1?comp=list&include=metadata
&prefix=test_1kb_blob_sync.txt%2F&restype=container&sig=REDACTED&sp=rwdl&sr=b&sv
=2018-03-28&timeout=901
User-Agent: [AzCopy/10.0.9-Preview Azure-Storage/0.6 (go1.10.3; Windows_NT)]
X-Ms-Client-Request-Id: [67675175-0850-4ea7-47fc-0b32aa70744b]
X-Ms-Version: [2018-03-28]

RESPONSE Status: 403 Server failed to authenticate the request. Make sure the
value of Authorization header is formed correctly including the signature.
Content-Length: [400]
Content-Type: [application/xml]
Date: [Thu, 11 Apr 2019 14:26:33 GMT]
Server: [Microsoft-HTTPAPI/2.0]
Vary: [Origin]
X-Ms-Error-Code: [AuthenticationFailed]
X-Ms-Request-Id: [21779942-601e-0058-7872-f0cb50000000]

[zezha-msft]

Hi @Jaurocha, I just noticed your copy-pasting is somehow changing the SAS.

Original:
"https://zemaintest.blob.core.windows.net/demo1/test_1kb_blob_sync.txt?st=2019-04-09T18%3A08%3A52Z&se=2019-05-10T18%3A08%3A00Z&sp=rwdl&sv=2018-03-28&sr=b&sig=gLJ8ZIekS3Q9DD7t5bQDUrFbqiSf8LOdGRQCplsgqU0%3D"

Yours:
"https://zemaintest.blob.core.windows.net/demo1/test_1kb_blob_sync.txt?st=2019-04-09T18A08A52Z&se=2019-05-10T18A08A00Z&sp=rwdl&sv=2018-03-28&sr=b&sig=gLJ8ZIekS3Q9DD7t5bQDUrFbqiSf8LOdGRQCplsgqU0D"

The "%"s all got messed up. The original SAS is most definitely correct, I just tried it again.

[Jaurocha]

Hi.
I had neglected to escape the % sign my bat file... Awkward...

Thanks for the assistance.