Azure-sdk-for-python: Creating queues in Azure Stack fails

[charliejllewellyn]

I have a very simple piece of code to create a queue in Azure Stack.

from azure.storage.queue import QueueService

queue_service = QueueService(connection_string='DefaultEndpointsProtocol=https;AccountName=testaccount002;AccountKey=iVtx5TePJg+72Q/wQ/z73cV4tpw+U5j8ZEC5cjlWa5pcc3VXJFBaDtKkKSttpkIXhJENrlBi58Nr9js1hivQYQ==;EndpointSuffix=local.azurestack.external')

queue_service.create_queue('taskqueue')

This fails with the following error:

[root@98ec966063e1 /]# python /root/.azure/testQueues.py
Traceback (most recent call last):
  File "/root/.azure/testQueues.py", line 6, in <module>
    queue_service.create_queue('taskqueue')
  File "/usr/lib/python2.7/site-packages/azure/storage/queue/queueservice.py", line 529, in create_queue
    _dont_fail_on_exist(ex)
  File "/usr/lib/python2.7/site-packages/azure/storage/_error.py", line 97, in _dont_fail_on_exist
    raise error
azure.common.AzureHttpError: The value for one of the HTTP headers is not in the correct format.
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidHeaderValue</Code><Message>The value for one of the HTTP headers is not in the correct format.
RequestId:d95d9903-7856-073d-7e34-f6e431cfdaa4
Time:2017-12-18T10:24:32.9138903Z</Message><HeaderName>x-ms-version</HeaderName><HeaderValue>2016-05-31</HeaderValue></Error>

and also

from azure.storage.table import TableService, Entity

table_service = TableService(connection_string='DefaultEndpointsProtocol=https;AccountName=testaccount002;AccountKey=iVtx5TePJg+72Q/wQ/z73cV4tpw+U5j8ZEC5cjlWa5pcc3VXJFBaDtKkKSttpkIXhJENrlBi58Nr9js1hivQYQ==;EndpointSuffix=local.azurestack.external')
table_service.create_table('tasktable')

Fails with the same error:

[root@98ec966063e1 /]# python /root/.azure/testQueues.py
Traceback (most recent call last):
  File "/root/.azure/testQueues.py", line 10, in <module>
    table_service.create_table('tasktable')
  File "/usr/lib/python2.7/site-packages/azure/storage/table/tableservice.py", line 529, in create_table
    _dont_fail_on_exist(ex)
  File "/usr/lib/python2.7/site-packages/azure/storage/_error.py", line 97, in _dont_fail_on_exist
    raise error
azure.common.AzureHttpError: The value for one of the HTTP headers is not in the correct format.
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
  <code>InvalidHeaderValue</code>
  <message xml:lang="en-US">The value for one of the HTTP headers is not in the correct format.
RequestId:137f5353-81c9-0a53-4748-6258ce84b0a3
Time:2017-12-18T11:13:30.8829744Z</message>
</error>

This same code works in Azure.

[lmazuel]

Hi @charliejllewellyn

Note that the Storage repo (for issues) is here (the Storage team supports the client themselves, not the central SDK team where I am):
https://github.com/Azure/azure-storage-python

But I have context on this one, so let me give you what I know.

As ARM, Storage has constraints in term of ApiVersion for AzureStack support. Recommended ApiVersion for Azure Stack and Storage Data Plane is 2015-04-05. Being that ApiVersion is hardcoded into Storage Data client, this means you need to downgrade your SDK to 0.30.0, which is the latest to support this version. As far as I know, this was the Storage team solution for this problem. But since I'm not part of this team, I strongly encourage you to open an issue on their tracker, asking for Azure Stack story.

For CLI, since it's supports changing profile on the fly, downgrading package was not an option. We then decided to ship a new package as a mix-up of different ApiVersion/SDK version:
https://github.com/Azure/azure-multiapi-storage-python

This is NOT an official package, and official solution is the one from the Storage team. I cannot guarantee stability on this package, but this is how we solved this for Azure Storage + CLI + Stack.

FYI @bganapa @derekbekoe @zezha-msft

[bganapa]

Thanks for flagging me , We will take this up with the storage team

[zezha-msft]

Thanks @lmazuel!

@charliejllewellyn, for more information on the differences between Azure Storage and Azure Stack Storage, please refer to this link.

[charliejllewellyn]

Hey both,

Firstly @lmazuel thanks for the detail in your response and also for your help over the last few weeks.

I thought this might be an API version issue again, I guess my concern is that since Azure Stack is marketed as "An extension to Azure" and the concept is that you just re-target deployment code and apps to Azure Stack, the complexity of actually getting this working really concerns me.

FYI this is not the only SDK I am having issues with, we have also to use an old version of the .NET SDK as again the version is hardcoded, as well as the same problems we've had with authentication also being prevalent in Terraform.

As a service provider looking to provide Azure Stack services it is important we can articulate how to get tools working successfully with Azure Stack. However, since the problems here are common across SDK's and devOps tools it seems that a common approach and methodology should be defined by MS to handle Azure Stack. Do either of you know where it would be best to target this request?

[lmazuel]

Hi @charliejllewellyn
I'm glad if I was able to help you :)
@bganapa tagged here is from the Azure Stack team.

You can send me an email, and I can forward to at least @bganapa, and he will know the best list to handle the discussion.

[zhusijia26]

Close this issue due to idle time. Please reopen if there is still question.