Azure-sdk-for-python: generate_shared_access_signature for blob service client is broken

[LazzariPaolo]

I am using

azure-storage-blob==12.0.0

I am trying to copy one vhd from one storage account to another.
In the past, with the preview, the following has worked fine:

from azure.storage.blob import BlobServiceClient
from azure.storage.blob import ResourceTypes, AccountPermissions

source_client = BlobServiceClient.from_connection_string(src_conn_string)
dest_client = BlobServiceClient.from_connection_string(dest_conn_string)

sas_token = source_client.generate_shared_access_signature(
    resource_types = ResourceTypes.OBJECT
    permission = AccountPermission.READ
    expiry = datetime.utcnow() + timedelta(hours = 24)
    )

source_blob = (
   f'https://{src_storage_account}'
   f'.blob.core.windows.net/{src_container_name}/'
   f'{vhd_name}?{sas_token}'
   )

dest_blob = dest_client.get_blob_client('Microsoft.Compute/Images', 'copy.vhd')

dest_blob.start_copy_from_url(source_blob)

I can see some stuff has been moved around so I updated my imports:

from azure.storage.blob import BlobClient, ContainerClient, BlobServiceClient
from azure.storage.blob import ResourceTypes
from azure.storage.common import AccountPermissions

However, the above now fails all the time.

azure.core.exceptions.ResourceNotFoundError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
ErrorCode:CannotVerifyCopySource
Error:None

Any ideas?
As far as I can tell, the issue is that the SAS seems to not be valid for whatever reason.

This only fails from the sdk. If generating the SAS from the portal or storage explorer, everything works fine.

[annatisch]

Hi @LazzariPaolo - thanks for the report.
There seems to be a version mismatch here... in v12.0.0 the generate SAS operations are no longer on the clients, rather it would be something like this:

from azure.storage.blob import generate_account_sas

sas_token = generate_account_sas(
    account_name = service_client.account_name,
    account_key,
    resource_types = ResourceTypes.OBJECT
    permission = AccountPermission.READ
    expiry = datetime.utcnow() + timedelta(hours = 24)
)

The API you're using appears to still be one of the beta versions?

[LazzariPaolo]

Hi @annatisch

Thank you for your response.
Unfortunately, I still get the same error with the following:

sas_token = generate_account_sas(
    account_name = service_client.account_name,
    account_key,
    resource_types = 'OBJECT'
    permission = 'READ'
    expiry = datetime.utcnow() + timedelta(hours = 24)
)

Has the API changed for copying blobs across storage accounts?

[LazzariPaolo]

The service_client in your example corresponds to source_client in my example right?

[annatisch]

Thanks @LazzariPaolo - the code above is generating a SAS with READ permissions - so I would expect that to be used to create the URL that is being _copied from_, i.e. the source account as you say. This doesn't need to be a client at all - you can just pass in the account name string.

To make sure the URL formatting is correct, you could do something like this:

source_client = BlobServiceClient.from_connection_string(src_conn_string)
sas_token = generate_account_sas(
    account_name=source_client.account_name,
    account_key="sourceaccountkey",
    resource_types = 'OBJECT'
    permission = 'READ'
    expiry = datetime.utcnow() + timedelta(hours = 24)
)
source_blob = BlobClient(
            source_client.url,
            container_name="container_name",
            blob_name="blob_name",
            credential=sas_token)

copy_url = source_blob.url
print(copy_url)

The next thing to consider is whether a SAS with WRITE permissions is required for the destination blob. Do you know whether this is the case? Or is the destination_client already fully authenticated with the necessary permissions?

[LazzariPaolo]

@annatisch
The destination client is created with a connection string.

For the previous version of the SDK I did not have to specify WRITE permissions for the SAS.
Nevertheless, I tried specifying 'RW' for the sas_token permission but the issue still occurs.

[annatisch]

Thanks @LazzariPaolo - could you please paste the output of the generate SAS token in your code? (Please use a fake account key, or other restrictions such as limited expiry for security)

This will help us visually verify the output of the generate SAS function to see whether a) the correct elements are present and b) those elements that depend on specific ordering are ordered correctly.

Adding @xiafu-msft from the Storage team - do you have insights on what might cause this?
It seems that the generated SAS token might be malformed.

[LazzariPaolo]

@annatisch

Thanks for your help.
Here's an example with a fake key:

import string
import random
from datetime import datetime, timedelta
from azure.storage.blob import generate_account_sas

fake_key = ''.join(random.choice(string.ascii_letters) for i in range(88))
fake_sas_token = generate_account_sas(
    account_name = 'test', 
    account_key = fake_key,
    resource_types = 'OBJECT',
    permission = 'RW', 
    expiry = datetime.utcnow() + timedelta(minutes = 1))

The resulting false token looks something like this:

se=2019-11-07T10%3A38%3A37Z&sp=RW&sv=2019-02-02&ss=b&srt=OBJECT&sig=IVkqdhymQWE5xOZcWuxTZSyALCIMlAq68gFd1pMa5tE%3D

[annatisch]

Thanks @LazzariPaolo - I see the issue now.
The value of resource types is incorrect, try the following:

from azure.storage.blob import generate_account_sas, ResourceTypes, AccountSasPermissions

sas_token = generate_account_sas(
    account_name="account_name",
    account_key="account_key",
        resource_types=ResourceTypes(object=True),
    permission=AccountSasPermissions(read=True),
    expiry=datetime.utcnow() + timedelta(minutes=1)
)

[LazzariPaolo]

Thanks @annatisch ,

I can confirm that that fixes it! All working good now.

[msftbot[bot]]

Thanks for working with Microsoft on GitHub! Tell us how you feel about your experience using the reactions on this comment.