Azure-docs: Edge Blob Storage - Storage Explorer - Invalid x-ms-version

[str1ke-cw]

Hello,

I am trying to access a local blob storage on an IoT Edge device. However, I get the following error in the Storage Explorer when trying to view the contents of a container:

<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidHeaderValue</Code><Message>The value for one of the HTTP headers is not in the correct format. RequestId:19eac3da-68e9-498c-8830-335d3dab5266 Time:2020-06-23T15:27:46.7769193Z</Message><HeaderName>x-ms-version</HeaderName><HeaderValue>2019-07-07</HeaderValue><ExceptionDetails><ExceptionMessage>The value 2019-07-07 provided for request header x-ms-version is invalid.</ExceptionMessage><StackTrace>Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.InvalidHeaderProtocolException: The value 2019-07-07 provided for request header x-ms-version is invalid. at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.RunVersionCheck() in /app/xfe/common/Protocols/REST/src/BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 756 at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.ProcessImpl(AsyncIteratorContext`1 async)+MoveNext() in /app/xfe/common/Protocols/REST/src/BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 1386</StackTrace></ExceptionDetails></Error>

The Edge Module Log says:

[2020-06-23 15:21:30.513] [info    ] [tid 32] Info: Accepting {0} request for '{1}'. RemoteEndPoint='{2}', Via='{3}', Expect='{4}', Connection='{5}', Version='{6}', ClientRequestId='{7}', UserAgent='{8}' DurationStayedInQueueInMs='{9}', RemainingRequestsInQueueAtEnqueue='{10}', RemainingRequestsInQueueAtDequeue='{11}', WasInHighPriorityQueue='{12}', ToSetKeepAliveToFalse='{13}', ActiveWorkerThreads='{14}', ActiveCompletionThreads='{15}' PassedInActivityId={16}, EnqueueTime={17}, DequeueTime={18}, RequestTraceActvityId = '{19}', p0="GET", p1="http://192.168.0.194:11002/localsharkstore/test?delimiter=%2F&maxresults=1000&include=metadata&restype=container&comp=list", p2="192.168.0.85:63668", p3="", p4="", p5="keep-alive", p6="2019-07-07", p7="7c1aefec-c745-40ac-a862-6c58903af2e6", p8="MicrosoftAzure Storage Explorer,", p9="0.2467", p10="0", p11="0", p12="False", p13="False", p14="0", p15="0", p16="null", p17="2020-06-23T15:21:30.5125432Z", p18="2020-06-23T15:21:30.5126713Z", p19="264b0d98-25e3-4564-91bb-7a1fcf9b49a5"
[2020-06-23 15:21:30.529] [info    ] [tid 12] Info: Processing exception: {0}, p0="Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.InvalidHeaderProtocolException: The value 2019-07-07 provided for request header x-ms-version is invalid.  at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.RunVersionCheck() in /app/xfe/common/Protocols/REST/src/BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 756   at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.ProcessImpl(AsyncIteratorContext`1 async)+MoveNext() in /app/xfe/common/Protocols/REST/src/BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 1386"
[2020-06-23 15:21:30.563] [info    ] [tid 12] Info_debug: Setting status description as "{0}", p0="The value for one of the HTTP headers is not in the correct format."
[2020-06-23 15:21:30.584] [error   ] [tid 12] Error: Billing: EmitQoSEventAndCollectBillingDataForPreAuthFailures Exception : {0}, p0="System.NullReferenceException: Object reference not set to an instance of an object.    at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.EmitQoSEventAndCollectBillingDataForPreAuthFailures(MeasurementEventStatus measurementStatus, Boolean skipBillingAndMetrics) in /app/xfe/common/Protocols/REST/src/BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 6112"

I cannot spot the mistake. Can you please help?

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 07a44a28-a19c-f68e-167c-5e8ad91605e0
• Version Independent ID: 33eacdff-4f0a-c543-d8f9-e7b208bab015
• Content: Store block blobs on devices - Azure IoT Edge
• Content Source: articles/iot-edge/how-to-store-data-blob.md
• Service: iot-edge
• GitHub Login: @kgremban
• Microsoft Alias: kgremban

[BharathNimmala-MSFT]

@str1ke-cw Thank you for reaching out to us. Please note that this channel (the Azure Docs repository) is only for driving improvements related to the Azure documentation. Can you please share the documentation link which are referring and facing this issue ? so that we can assist accordingly.

[str1ke-cw]

@BharathNimmala-MSFT Thank you for your quick reply. I used this documentation for setting up the Blob Storage module:

• https://docs.microsoft.com/de-de/azure/iot-edge/how-to-store-data-blob
• https://docs.microsoft.com/de-de/azure/iot-edge/how-to-deploy-blob

I noticed, that it says on the first page: "Blob storage modules on IoT Edge use the Azure Storage SDKs, and are consistent with the 2017-04-17 version of the Azure Storage API for block blob endpoints". Is this still up to date?

I found documented here that 2019-07-07 should be the current version and I found no way to change it in Storage Explorer or the Azure Python SDK.

Furthermore, I was missing any further documentation on the error I encountered and how to resolve it.

I had it working before and the only major change that comes to my mind is that previously I was running the module on armv7, while now I am running it on armv8. I noticed, that it is says in this blog post from 2019, that armv8 is in beta. Is that still so?

Thank you for your help.

[BharathNimmala-MSFT]

@str1ke-cw Thank you for sharing the documentation links . I have assigned this to our team mate, who is SME in this area and they will work with you on this.

[SatishBoddu-MSFT]

@str1ke-cw Thanks for reaching out to us! Could you please let us know the OS your are using ? Under which category your system is mentioned , Tier 1 or Tier 2 ?

[str1ke-cw]

@SatishBoddu-MSFT I am using Ubuntu Server 18.04 on ARM64. It is listed in Tier 1 as _public preview_. Ubuntu 18.04 ARM64 (I presume that ist Ubuntu 18.04 Desktop ?) is also listed under Tier 2 as compatible.

[SatishBoddu-MSFT]

@str1ke-cw Thank you for the confirmation, I am working on it.

[SatishBoddu-MSFT]

@str1ke-cw Could you please confirm

1. The version of the Azure blob storage SDK you are using ?.
2. How your module is accessing the local blob endpoint(url) ?
3. Have you tried from any back-end application to access the blobstoragemodule?

Please mask sensitive info in your replies.

I was able to repro this issue when using Azure.Storage.Blobs 12.4.4; C# [Ubuntu 18.04 64 ]

I was success in uploading the blob from backend app with Microsoft.WindowsAzure.Storage; C#

[SatishBoddu-MSFT]

@str1ke-cw I have assigned the issue to the content author for further review and update the doc as appropriate. If there are further questions regarding this matter, please tag me in your reply.

[str1ke-cw]

@SatishBoddu-MSFT thank you for your reply. Please, see below for my answers and further questions.

@str1ke-cw Could you please confirm

1. The version of the Azure blob storage SDK you are using ?.

According to the logs: "Starting Azure Blob Storage on IoT Edge, version 1.2.0.0."
The docker image is tagged as "latest".

2. How your module is accessing the local blob endpoint(url) ?

http;BlobEndpoint=http://AzureBlobStorageonIoTEdge:11002/<name>;AccountName=<name>;AccountKey=<key>

3. Have you tried from any back-end application to access the blobstoragemodule?

I used the Azure Python SDK and it failed with the same error. I haven't tried using the C# SDK just to access the Local Blob Storage and it is not really an option for us.

Should the Local Blob Storage use API version 2019-07-07, as it is the newest or 2017-04-17 how it is documented?

Where / how can I find out, which is the newest docker image for Local Blob Storage?

When will it be possible to use the Local Blob Storage on arm64 with the Python SDK and Azure Storage Explorer?

Looking forward to your reply. Thank you very much.

[ms32035]

https://mcrflowprodcentralus.data.mcr.microsoft.com/mcrprod/azure-blob-storage?P1=1593699711&P2=1&P3=1&P4=Wet5Hau761fQ6UorxuYiqfVnrqpmuTkHWvHSrBNBaKc%3D&se=2020-07-02T14%3A21%3A51Z&sig=W9KNrhj%2BMI1Rw%2FqAvo6WqGXcxTHMD1paw%2Bst3Bs7pnw%3D&sp=r&sr=b&sv=2015-02-21

You can try 1.3-linux-arm32v7 which seems to work, only different file permissions

[str1ke-cw]

@ms32035 Thank you for the reply. I will do that. For the link I get a: ERROR 403: Time-Limited URL validation failed

[ms32035]

https://mcr.microsoft.com/v2/azure-blob-storage/tags/list

[kgremban]

@mbialecka could you take a look at this issue?

[str1ke-cw]

I just want to let you know, that the module is working as expected when 1.3-linux-arm32v7 is used (thank you @ms32035).

Please let me know, when the arm64v8 is updated as well. Also from all that I got so far it seems that the documentation here should be updated to say that API version 2019-07-07 is used (at least for arm32v7) - or is there just some version negotiation that is missing in the arm64v8 version?

[mbialecka]

There is no 1.3 module for arm64, we only have this one preview version, and some things may not work as expected. At this time I don't have an arm64 to investigate this issue. All versions should be compatible with 2017-04-17 api as specified in our docs. New api versions should be backwards compatible and using newer api should still work with the module. However, it's best to use 2017-04-17 if possible (sometimes SDKs may not handle the differences well).

[str1ke-cw]

@mbialecka Which version do you mean with this _one preview version_? The "1.2-linux-arm64v8"? Is a version 1.3 planned for arm64? Could you please provide a roadmap? This would be very interesting also for IoT Edge on arm64 in general.

As a user, I would hope for basic things to work without me having to worry about API versions, as long as I use the newest version of a Azure Blob Storage on IoT Edge on a platform (e.g. arm64) and the newest version of a supported SDK (e.g. Azure Python SDK), even if arm64 support is in _Public Preview_ for the Blob Storage and in general. Basically, I understood the idea as such: use the Blob Storage on the edge just like in the cloud.

[str1ke-cw]

@kgremban @mbialecka Could you please give further information on this? My worry now is that _1.2-linux-arm64v8_ is the one and only preview version, you referred to before, and that is no plans for future versions for arm64v8. My hope is that an architecture supported as _preview_ will ultimately transistion into a fully supported architecture.

[arduppal]

@str1ke-cw:- Thank you for your feedback, will add it in our backlog items and will to do it when we get to it.
Currently, there are no plans on future versions or any update on the current versions of this module. For the blob storage module to go GA, IoT Edge ARM64 should go GA which is currently in public preview, see here: https://docs.microsoft.com/en-us/azure/iot-edge/support#tier-1

[ms32035]

Now the Edge Blob Storage doesn't even work with Azure Storage Explorer

The value 2019-12-12 provided for request header x-ms-version is invalid.

Would you mind reprioritising your backlog?

[arduppal]

@ms32035 :- Please try using v1.14.2 or older, till we investigate it. Thanks for reporting it.

[jasonjalufka]

@ms32035 I was having issues connecting to local blob storage on an IoTEdge Device through Azure Storage Explorer. Downgrading to Version 1.13.1 somehow resolved the issue for me.

Now my issue is that anytime I try to do anything beyond instantiating a BlobServiceClient to local blob storage, i.e. create a container or upload a blob, I see the following error: Using Python azure-storage-blob package

azure.core.exceptions.HttpResponseError: The value for one of the HTTP headers is not in the correct format.
RequestId:1dce6b7a-af79-4a12-ab20-6ccd5fa0b0f9
Time:2020-09-22T01:57:59.8807923Z
ErrorCode:InvalidHeaderValue
Error:None
HeaderName:x-ms-version
HeaderValue:2019-12-12
ExceptionDetails:None
ExceptionMessage:The value 2019-12-12 provided for request header x-ms-version is invalid.

I'm using the following package versions:

azure-core          1.8.1
azure-iot-device    2.2.0
azure-storage-blob  12.5.0

Is there a way I could change the value of x-ms-version headers using the Python SDK?
Any insight would be greatly appreciated. I've been trying to resolve this for days.

[mbialecka]

@jasonjalufka Blob Client has api_version parameter. Please use '2017-04-17'.

[jasonjalufka]

@mbialecka Wow, thank you! Can't believe I missed that! 2017-04-17 didn't work, but it did give me this error message:

Unexpected Error: Unsupported API version '2017-04-17'. Please select from:
2019-02-02
2019-07-07
2019-10-10
2019-12-12

I tried 2019-10-10 which gave the same HttpResponseError as before, but changing the api_version to 2019-02-02 resolved the error! 👍

[maecky]

@mbialecka I am having the same issue with the Python SDK as @jasonjalufka . Can't use version 2017-04-17:

Unexpected Error: Unsupported API version '2017-04-17'. Please select from:
2019-02-02
2019-07-07
2019-10-10
2019-12-12

[keecm]

api version 2019-07-07 solved my problem.

[iannovic]

I'm having the same issue with the Python SDK

[SatishBoddu-MSFT]

I'm having the same issue with the Python SDK

@iannovic did you tried api version 2019-07-07 ?

[iannovic]

I'm having the same issue with the Python SDK

@iannovic did you tried api version 2019-07-07 ?

@SatishBoddu-MSFT
Yea i tried all the api_version possibilities including the 2019-07-07

Here are the dependencies i am using:

IMAGE:
"image":"mcr.microsoft.com/azure-blob-storage:1.2-linux-arm64v8"
requirements.txt file:

azure-iot-device~=2.2.0
azure-storage-blob~=12.5.0
azure-cognitiveservices-vision-computervision~=0.6.0

Running on Aarch64 Ubuntu 18.04 Iotedge device..

Here is the specific error message with 2019-07-07:
Traceback (most recent call last): File "main.py", line 107, in <module> loop.run_until_complete(main()) File "/usr/local/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete return future.result() File "main.py", line 59, in main blob.upload_blob(byte_im) File "/usr/local/lib/python3.7/site-packages/azure/core/tracing/decorator.py", line 83, in wrapper_use_tracer return func(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/azure/storage/blob/_blob_client.py", line 539, in upload_blob return upload_block_blob(**options) File "/usr/local/lib/python3.7/site-packages/azure/storage/blob/_upload_helpers.py", line 156, in upload_block_blob process_storage_error(error) File "/usr/local/lib/python3.7/site-packages/azure/storage/blob/_shared/response_handlers.py", line 147, in process_storage_error raise error azure.core.exceptions.HttpResponseError: The value for one of the HTTP headers is not in the correct format. RequestId:117501cb-15cf-4603-b16d-00bb9568bd12 Time:2020-10-06T15:29:12.4941516Z ErrorCode:InvalidHeaderValue Error:None HeaderName:x-ms-version HeaderValue:2019-07-07 ExceptionDetails:None ExceptionMessage:The value 2019-07-07 provided for request header x-ms-version is invalid. StackTrace:Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.InvalidHeaderProtocolException: The value 2019-07-07 provided for request header x-ms-version is invalid. at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.RunVersionCheck() in /app/xfe/common/Protocols/REST/src/BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 756 at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.ProcessImpl(AsyncIteratorContext`1 async)+MoveNext() in /app/xfe/common/Protocols/REST/src/BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 3231

[alevincenzi]

Same problem here.

alessandro@ubuntu:~$ iotedge --version
iotedge 1.0.9.5

alessandro@ubuntu:~$ iotedge list
NAME             STATUS           DESCRIPTION      CONFIG
edgeAgent        running          Up 3 minutes     mcr.microsoft.com/azureiotedge-agent:1.0
edgeContainer    running          Up 3 minutes     mcr.microsoft.com/azure-blob-storage:1.3-linux-amd64
edgeHub          running          Up 2 minutes     mcr.microsoft.com/azureiotedge-hub:1.0

alessandro@ubuntu:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:    18.04
Codename:   bionic

alessandro@ubuntu:~$ uname -a
Linux ubuntu 5.4.0-42-generic #46~18.04.1-Ubuntu SMP Fri Jul 10 07:21:24 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

alessandro@ubuntu:~$

Using Azure storage explorer on ubuntu

After connecting with connection string (iotedge and storage explorer on the same host) i get

Unable to retrieve child resources.

Details:
["RestError:<?xml version=\"1.0\" encoding=\"utf-8\"?>
<Error>
  <Code>InvalidHeaderValue</Code>
  <Message>The value for one of the HTTP headers is not in the correct format.\n
                     RequestId:2b7757dc-8ef9-4c9b-8ccd-901bd69258cd\n
                     Time:2020-10-13T11:06:37.4997063Z</Message>
  <HeaderName>x-ms-version</HeaderName>
  <HeaderValue>2019-12-12</HeaderValue>
  <ExceptionDetails>
    <ExceptionMessage>The value 2019-12-12 provided for request header x-ms-version is invalid.</ExceptionMessage>
    <StackTrace>Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.InvalidHeaderProtocolException: The value 2019-12-12 provided for request header x-ms-version is invalid.\n
         at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.RunVersionCheck() in F:\\Storage-XStore\\src\\XFE\\common\\Protocols\\REST\\src\\BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 788\n
         at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.ProcessImpl(AsyncIteratorContext`1 async)+MoveNext() in F:\\Storage-XStore\\src\\XFE\\common\\Protocols\\REST\\src\\BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 1538</StackTrace>
  </ExceptionDetails>
</Error>"]

The module logs:

[2020-10-13 11:06:37.384] [info    ] [tid 43] Info: Accepting {0} request for '{1}'. RemoteEndPoint='{2}', Via='{3}', Expect='{4}', Connection='{5}', Version='{6}', ClientRequestId='{7}', UserAgent='{8}' DurationStayedInQueueInMs='{9}', RemainingRequestsInQueueAtEnqueue='{10}', RemainingRequestsInQueueAtDequeue='{11}', WasInHighPriorityQueue='{12}', ToSetKeepAliveToFalse='{13}', ActiveWorkerThreads='{14}', ActiveCompletionThreads='{15}' PassedInActivityId={16}, EnqueueTime={17}, DequeueTime={18}, RequestTraceActvityId = '{19}', p0="GET", p1="http://192.168.211.137:11002/iotedgevm/?include=metadata&comp=list", p2="192.168.211.137:43518", p3="", p4="", p5="keep-alive", p6="2019-12-12", p7="a4389a72-7815-4ee6-a051-fda2e5de3e2e", p8="MicrosoftAzure Storage Explorer,", p9="2.0657", p10="0", p11="0", p12="False", p13="False", p14="0", p15="0", p16="null", p17="2020-10-13T11:06:37.3729560Z", p18="2020-10-13T11:06:37.3731978Z", p19="2b7757dc-8ef9-4c9b-8ccd-901bd69258cd"
[2020-10-13 11:06:37.466] [info    ] [tid 64] Info: Processing exception: {0}, p0="Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.InvalidHeaderProtocolException: The value 2019-12-12 provided for request header x-ms-version is invalid.
   at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.RunVersionCheck() in F:\Storage-XStore\src\XFE\common\Protocols\REST\src\BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 788
   at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.ProcessImpl(AsyncIteratorContext`1 async)+MoveNext() in F:\Storage-XStore\src\XFE\common\Protocols\REST\src\BasicHttpProcessorWithAuthAndAccountContainerBase.cs:line 1538"
[2020-10-13 11:06:37.510] [info    ] [tid 64] Info_debug: Setting status description as "{0}", p0="The value for one of the HTTP headers is not in the correct format."
[2020-10-13 11:06:37.574] [info    ] [tid 46] Info: Accepting {0} request for '{1}'. RemoteEndPoint='{2}', Via='{3}', Expect='{4}', Connection='{5}', Version='{6}', ClientRequestId='{7}', UserAgent='{8}' DurationStayedInQueueInMs='{9}', RemainingRequestsInQueueAtEnqueue='{10}', RemainingRequestsInQueueAtDequeue='{11}', WasInHighPriorityQueue='{12}', ToSetKeepAliveToFalse='{13}', ActiveWorkerThreads='{14}', ActiveCompletionThreads='{15}' PassedInActivityId={16}, EnqueueTime={17}, DequeueTime={18}, RequestTraceActvityId = '{19}', p0="GET", p1="http://192.168.211.137:11002/iotedgevm/?comp=properties&restype=service&_=1602587197566", p2="192.168.211.137:43520", p3="", p4="", p5="keep-alive", p6="2018-03-28", p7="29d38f80-0d44-11eb-9105-45e3a6016302", p8="Microsoft Azure Storage Explorer", p9="0.0673", p10="0", p11="0", p12="False", p13="False", p14="2", p15="0", p16="null", p17="2020-10-13T11:06:37.5741587Z", p18="2020-10-13T11:06:37.5741971Z", p19="c4c3a0bb-7e9f-4caf-83ad-eb80ac994059"
[2020-10-13 11:06:37.615] [info    ] [tid 17] WossProvider : Initializing new cache : {0}, p0="Account"
[2020-10-13 11:06:37.616] [info    ] [tid 17] WossProvider : {0} cache >> Setting up auto-refresh..., p0="Account"
[2020-10-13 11:06:37.619] [info    ] [tid 17] WossProvider : Initializing new cache : {0}, p0="NegativeAccount"
[2020-10-13 11:06:37.653] [info    ] [tid 64] Perf: PerfCounters: Account={0} Operation={1} on Container={2} with Status={3} RequestHeaderSize={4} RequestSize={5} ResponseHeaderSize={6} ResponseSize={7} ErrorResponseByte={8} TimeInMs={9} ProcessingTimeInMs={10} UserTimeoutInMs={11} OperationTimeoutInMs={12} MaxAllowedTimeoutInMs={13} SlaUsedInMs={14} ReadLatencyInMs={15} WriteLatencyInMs={16} ClientIP={17} UserAgent='{18}' RequestVersion='{19}' ProcessorVersionUsed='{20}' RequestUrl='{21}' ClientRequestId='{22}' MeasurementStatus={23} HttpStatusCode={24} TotalFeTimeInMs={25} TotalTableServerTimeInMs={26} TotalTableServerRoundTripCount={27} TotalPartitionWaitTimeInMs={28} TotalXStreamTimeInMs={29} TotalXStreamRoundTripCount={30} SmbOpLockBreakLatency={31} LastTableServerInstanceName={32} LastTableServerErrorCode={33} TotalAccountCacheWaitTimeInMs={34} TotalContainerCacheWaitTimeInMs={35} InternalStatus={36} RequestContentType='{37}' ResponseContentType='{38}' PartitionKey='{39}' ItemsReturned='{40}' BatchOperationCount='{41}' LastXStreamErrorCode={42} AuthenticationType='{43}' AccountConcurrentReq='{44}' OverallConcurrentReq='{45}' Range='{46}' EntityType='{47}' {48} LastTSPartition={49} TotalXCacheTimeInMs={50} TotalXCacheRoundTripCount={51} TotalTableServerOnlyTimeInMicroSeconds={52} TotalAuthTimeInMs={53} AuthenticationTimeInMs={54} AuthorizationTimeInMs={55} GetAuthenticationDataTimeInMs={56} GetAuthorizationDataTimeInMs={57} VnetPolicyCheckTimeInMs={58}, p0="null", p1="PreAuthenticationFailure", p2="null", p3="PreauthenticationFailure", p4="0", p5="0", p6="0", p7="0", p8="0", p9="NA", p10="NA", p11="NA", p12="NA", p13="NA", p14="NA", p15="NA", p16="NA", p17="192.168.211.137:43518", p18="MicrosoftAzure Storage Explorer, 1.15.1, linux,  azsdk-js-storageblob/12.2.0-preview.1 (NODE-VERSION v12.13.0; Linux 5.4.0-42-generic)", p19="2019-12-12", p20="2019-12-12", p21="http://192.168.211.137:11002/iotedgevm/?include=metadata&comp=list", p22="a4389a72-7815-4ee6-a051-fda2e5de3e2e", p23="PreauthenticationFailure", p24="400", p25="NA", p26="NA", p27="NA", p28="NA", p29="NA", p30="NA", p31="NA", p32="NA", p33="NA", p34="0", p35="0", p36="ClientOtherError", p37="", p38="application/xml", p39="", p40="0", p41="0", p42="NA", p43="NA", p44="", p45="1", p46="NA", p47="", p48="", p49="NA", p50="NA", p51="NA", p52="0", p53="0", p54="0", p55="0", p56="0", p57="0", p58="NA"
[2020-10-13 11:06:37.796] [info    ] [tid 17] Perf: PerfCounters: Account={0} Operation={1} on Container={2} with Status={3} RequestHeaderSize={4} RequestSize={5} ResponseHeaderSize={6} ResponseSize={7} ErrorResponseByte={8} TimeInMs={9} ProcessingTimeInMs={10} UserTimeoutInMs={11} OperationTimeoutInMs={12} MaxAllowedTimeoutInMs={13} SlaUsedInMs={14} ReadLatencyInMs={15} WriteLatencyInMs={16} ClientIP={17} UserAgent='{18}' RequestVersion='{19}' ProcessorVersionUsed='{20}' RequestUrl='{21}' ClientRequestId='{22}' MeasurementStatus={23} HttpStatusCode={24} TotalFeTimeInMs={25} TotalTableServerTimeInMs={26} TotalTableServerRoundTripCount={27} TotalPartitionWaitTimeInMs={28} TotalXStreamTimeInMs={29} TotalXStreamRoundTripCount={30} SmbOpLockBreakLatency={31} LastTableServerInstanceName={32} LastTableServerErrorCode={33} TotalAccountCacheWaitTimeInMs={34} TotalContainerCacheWaitTimeInMs={35} InternalStatus={36} RequestContentType='{37}' ResponseContentType='{38}' PartitionKey='{39}' ItemsReturned='{40}' BatchOperationCount='{41}' LastXStreamErrorCode={42} AuthenticationType='{43}' AccountConcurrentReq='{44}' OverallConcurrentReq='{45}' Range='{46}' EntityType='{47}' {48} LastTSPartition={49} TotalXCacheTimeInMs={50} TotalXCacheRoundTripCount={51} TotalTableServerOnlyTimeInMicroSeconds={52} TotalAuthTimeInMs={53} AuthenticationTimeInMs={54} AuthorizationTimeInMs={55} GetAuthenticationDataTimeInMs={56} GetAuthorizationDataTimeInMs={57} VnetPolicyCheckTimeInMs={58}, p0="iotedgevm", p1="GetBlobServiceProperties", p2="", p3="Success", p4="545", p5="0", p6="185", p7="601", p8="0", p9="131.6481", p10="130.552", p11="922337203685477", p12="30000", p13="30000", p14="0", p15="0", p16="1.0961", p17="192.168.211.137:43520", p18="Microsoft Azure Storage Explorer, 1.15.1, linux, Azure-Storage/2.10.3 (NODE-VERSION v12.13.0; Linux 5.4.0-42-generic)", p19="2018-03-28", p20="2018-03-28", p21="http://192.168.211.137:11002/iotedgevm/?comp=properties&restype=service&_=1602587197566", p22="29d38f80-0d44-11eb-9105-45e3a6016302", p23="Success", p24="200", p25="130.551", p26="0", p27="0", p28="0", p29="0", p30="0", p31="0", p32="", p33="0", p34="0", p35="0", p36="ExpectedSuccess", p37="", p38="application/xml", p39="", p40="0", p41="0", p42="0", p43="NA", p44="", p45="2", p46="", p47="", p48="", p49="", p50="0", p51="0", p52="0", p53="51.169", p54="38.291", p55="12.878", p56="0", p57="0", p58="0"
[2020-10-13 11:06:37.860] [info    ] [tid 17] [BlobInterface.cc:1490] [ListBlobsInOrder] ListBlobsInOrder received. Container:source-local BlobNameStart:null MaxBlobNames:1 OrderType:1 Flags:1

Any help is appreciated :)

[alevincenzi]

Yes ... no choice bu to downgrade.

Finally, Storage Explorer 1.14.2 1 + AzCopy 10.5.0 (proposed by @arduppal above) can connect with mcr.microsoft.com/azure-blob-storage:1.3-linux-amd64

[kgalic]

@alevincenzi Can you please explain how did you downgrade the Storage Explorer?

UPDATE:
Please disregard my question: I downloaded it from the Github, unpacked it, and started it via the Console.

[alevincenzi]

Hello @kgalic.
I had to move to Windows. There you can uninstall the app and reinstall the 1.14 from this link.

On Ubuntu, i installed the latest from the snap store. At the moment i did not downgrade there. Sorry.

[SatishBoddu-MSFT]

Hello @alevincenzi , Just checking, Did you get unblocked from this issue?

[alevincenzi]

Hello @SatishBoddu-MSFT.

The issue with Azure Storage Explorer is solved. Had to downgrade.

The problem persists when i run the C# example provided here:

using System;
using System.IO;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;

namespace Demo
{
    class Program
    {
        static void Main(string[] args)
        {
            string StorageAccount = "...";
            string StorageAccessKey = "...";
            string IoTEdgeDeviceIP = "127.0.0.1";
            string connectionString =
                "DefaultEndpointsProtocol=https;"+
                "BlobEndpoint=http://"+ IoTEdgeDeviceIP + ":11002/"+ StorageAccount + ";"+
                "AccountName="+ StorageAccount + ";"+
                "AccountKey="+ StorageAccessKey;

            // Create a BlobServiceClient object which will be used to create a container client
            BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);

            //Create a unique name for the container
            string containerName = "quickstartblobs" + Guid.NewGuid().ToString();

            // Create the container and return a container client object
            BlobContainerClient containerClient = blobServiceClient.CreateBlobContainer(containerName);

            // Create a local file in the ./data/ directory for uploading and downloading
            string localPath = "./data/";
            string fileName = "quickstart" + Guid.NewGuid().ToString() + ".txt";
            string localFilePath = Path.Combine(localPath, fileName);

            // Write text to the file
            File.WriteAllText(localFilePath, "Hello, World!");

            // Get a reference to a blob
            BlobClient blobClient = containerClient.GetBlobClient(fileName);

            Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);

            // Open the file and upload its data
            using (var uploadFileStream = File.OpenRead(localFilePath))
            {
                blobClient.Upload(uploadFileStream, true);
                uploadFileStream.Close();
            }

            Console.WriteLine("Listing blobs...");

            // List all blobs in the container
            foreach (BlobItem blobItem in containerClient.GetBlobs())
            {
                Console.WriteLine("\t" + blobItem.Name);
            }

            // Download the blob to a local file
            string downloadFilePath = localFilePath.Replace(".txt", "DOWNLOADED.txt");

            Console.WriteLine("\nDownloading blob to\n\t{0}\n", downloadFilePath);

            // Download the blob's contents and save it to a file
            BlobDownloadInfo download = blobClient.Download();

            using (FileStream downloadFileStream = File.OpenWrite(downloadFilePath))
            {
                download.Content.CopyTo(downloadFileStream);
                downloadFileStream.Close();
            }

            // Clean up
            Console.Write("Press any key to begin clean up");

            Console.ReadLine();

            Console.WriteLine("Deleting blob container...");
            containerClient.DeleteAsync();

            Console.WriteLine("Deleting the local source and downloaded files...");
            File.Delete(localFilePath);
            File.Delete(downloadFilePath);

            Console.WriteLine("Done");
        }
    }
}

The code is taken from this page: https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-dotnet

You can setup with:

dotnet new console --name Demo
cd Demo
dotnet add package Azure.Storage.Blobs --version 12.6.0

I am running the module mcr.microsoft.com/azure-blob-storage:1.3-linux-amd64.

And I get:

alessandro@ubuntu:~/Demo$ dotnet run

Unhandled Exception: Azure.RequestFailedException: The value for one of the HTTP headers is not in the correct format.
RequestId:14b6c017-5a1f-4e71-80cb-7a7908782c84
Time:2020-10-23T18:18:20.6340052Z
Status: 400 (The value for one of the HTTP headers is not in the correct format.)
ErrorCode: InvalidHeaderValue

Headers:
x-ms-request-id: 14b6c017-5a1f-4e71-80cb-7a7908782c84
Server: Microsoft-NetCore/2.0
Date: Fri, 23 Oct 2020 18:18:20 GMT
Connection: close
Content-Type: application/xml
Content-Length: 1175

   at Azure.Storage.Blobs.BlobRestClient.Container.CreateAsync_CreateResponse(ClientDiagnostics clientDiagnostics, Response response)
   at Azure.Storage.Blobs.BlobRestClient.Container.CreateAsync(ClientDiagnostics clientDiagnostics, HttpPipeline pipeline, Uri resourceUri, PublicAccessType access, String version, Nullable`1 timeout, IDictionary`2 metadata, String requestId, String defaultEncryptionScope, Nullable`1 preventEncryptionScopeOverride, Boolean async, String operationName, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobContainerClient.CreateInternal(PublicAccessType publicAccessType, IDictionary`2 metadata, BlobContainerEncryptionScopeOptions encryptionScopeOptions, Boolean async, CancellationToken cancellationToken, String operationName)
   at Azure.Storage.Blobs.BlobContainerClient.Create(PublicAccessType publicAccessType, IDictionary`2 metadata, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobServiceClient.CreateBlobContainer(String blobContainerName, PublicAccessType publicAccessType, IDictionary`2 metadata, CancellationToken cancellationToken)
   at Demo.Program.Main(String[] args) in /home/alessandro/Demo/Program.cs:line 29

Using the same connection string, Azure Storage Explorer 1.14.2 connects fine while the client application fails with (i guess) the same error that i was having with the latest version 1.15.1.

PS: I had to rename the account name. local-blob is not supported ;) Use localblob instead.

[alevincenzi]

An update.

I had to downgrade to Azure.Storage.Blobs @ 12.4.4 to see it working.

12.16.0 and 12.5.1 were both failing with the mentioned error.

Nevertheless, now the listing of blobs fails:

Exception has occurred: CLR/System.FormatException
An unhandled exception of type 'System.FormatException' occurred in System.Private.CoreLib.dll: 'String '' was not recognized as a valid DateTime.'
   at System.DateTimeParse.Parse(ReadOnlySpan`1 s, DateTimeFormatInfo dtfi, DateTimeStyles styles, TimeSpan& offset)
   at System.DateTimeOffset.Parse(String input, IFormatProvider formatProvider, DateTimeStyles styles)
   at System.DateTimeOffset.Parse(String input, IFormatProvider formatProvider)
   at Azure.Storage.Blobs.Models.BlobItemProperties.FromXml(XElement element)
   at Azure.Storage.Blobs.Models.BlobItem.FromXml(XElement element)
   at Azure.Storage.Blobs.Models.BlobsFlatSegment.<>c.<FromXml>b__30_0(XElement e)
   at System.Linq.Enumerable.SelectEnumerableIterator`2.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Azure.Storage.Blobs.Models.BlobsFlatSegment.FromXml(XElement element)
   at Azure.Storage.Blobs.BlobRestClient.Container.ListBlobsFlatSegmentAsync_CreateResponse(ClientDiagnostics clientDiagnostics, Response response)
   at Azure.Storage.Blobs.BlobRestClient.Container.<ListBlobsFlatSegmentAsync>d__33.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Storage.Blobs.BlobContainerClient.<GetBlobsInternal>d__71.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Azure.Storage.Blobs.Models.GetBlobsAsyncCollection.<GetNextPageAsync>d__5.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
   at Azure.Storage.StorageCollectionEnumerator`1.StoragePageable.<GetEnumerator>d__5.MoveNext()
   at BlobDemo.Program.Main(String[] args) in C:\DEV\BlobDemo\Program.cs:line 54

Line 54 is

            foreach (BlobItem blobItem in containerClient.GetBlobs())

[alevincenzi]

FYI, the example with SDKV11 (legacy) works out of the box:

https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-dotnet-legacy

Here the full code, for those who need it:

using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Blob;

namespace BLobDemoV11
{
    class Program
    {
        public static void Main()
        {
            Console.WriteLine("Azure Blob Storage - .NET quickstart sample\n");


            string StorageAccount = "...";
            string StorageAccessKey = "...";
            string IoTEdgeDeviceIP = "...";

            string connectionString =
                "DefaultEndpointsProtocol=https;"+
                "BlobEndpoint=http://"+ IoTEdgeDeviceIP + ":11002/"+ StorageAccount + ";"+
                "AccountName="+ StorageAccount + ";"+
                "AccountKey="+ StorageAccessKey;

            // Check whether the connection string can be parsed.
            CloudStorageAccount storageAccount;
            if (CloudStorageAccount.TryParse(connectionString, out storageAccount))
            {
                // Create the CloudBlobClient that represents the 
                // Blob storage endpoint for the storage account.
                CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();

                // Create a container called 'quickstartblobs' and 
                // append a GUID value to it to make the name unique.
                CloudBlobContainer cloudBlobContainer =
                    cloudBlobClient.GetContainerReference("quickstartblobs-" + Guid.NewGuid().ToString());
                cloudBlobContainer.Create();

                // Set the permissions so the blobs are public.
                BlobContainerPermissions permissions = new BlobContainerPermissions
                {
                    PublicAccess = BlobContainerPublicAccessType.Blob
                };
                cloudBlobContainer.SetPermissions(permissions);

                // Create a file in your local MyDocuments folder to upload to a blob.
                string localPath = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments);
                string localFileName = "QuickStart_" + Guid.NewGuid().ToString() + ".txt";
                string sourceFile = Path.Combine(localPath, localFileName);
                // Write text to the file.
                File.WriteAllText(sourceFile, "Hello, World!");

                Console.WriteLine("Temp file = {0}", sourceFile);
                Console.WriteLine("Uploading to Blob storage as blob '{0}'", localFileName);

                // Get a reference to the blob address, then upload the file to the blob.
                // Use the value of localFileName for the blob name.
                CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(localFileName);
                cloudBlockBlob.UploadFromFile(sourceFile);

                // List the blobs in the container.
                Console.WriteLine("List blobs in container.");
                BlobContinuationToken blobContinuationToken = null;
                do
                {
                    var results = cloudBlobContainer.ListBlobsSegmented(null, blobContinuationToken);
                    // Get the value of the continuation token returned by the listing call.
                    blobContinuationToken = results.ContinuationToken;
                    foreach (IListBlobItem item in results.Results)
                    {
                        Console.WriteLine(item.Uri);
                    }
                } while (blobContinuationToken != null); // Loop while the continuation token is not null.

                // Download the blob to a local file, using the reference created earlier.
                // Append the string "_DOWNLOADED" before the .txt extension so that you 
                // can see both files in MyDocuments.
                string destinationFile = sourceFile.Replace(".txt", "_DOWNLOADED.txt");
                Console.WriteLine("Downloading blob to {0}", destinationFile);
                cloudBlockBlob.DownloadToFile(destinationFile, FileMode.Create);

                Console.WriteLine("Press the 'Enter' key to delete the example files, " +
                    "example container, and exit the application.");
                Console.ReadLine();
                // Clean up resources. This includes the container and the two temp files.
                Console.WriteLine("Deleting the container");
                if (cloudBlobContainer != null)
                {
                    cloudBlobContainer.DeleteIfExists();
                }
                Console.WriteLine("Deleting the source, and downloaded files");
                File.Delete(sourceFile);
                File.Delete(destinationFile);
            }
            else

            {
                // Otherwise, let the user know that they need to define the environment variable.
                Console.WriteLine(
                    "A connection string has not been defined in the system environment variables. " +
                    "Add an environment variable named 'AZURE_STORAGE_CONNECTION_STRING' with your storage " +
                    "connection string as a value.");
                Console.WriteLine("Press any key to exit the application.");
                Console.ReadLine();
            }

            Console.WriteLine("Press any key to exit the sample application.");
            Console.ReadLine();
        }
    }
}

And here the package:

  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.Storage.Blob" Version="11.2.2" />
  </ItemGroup>

[ma3goul]

Hi All ,

I have the same problem with the blob on the edge .
It works for me on the host using the python legacy SDK but when I try to dockerise it , it fail :/

Any updates on this topic ?

Thank you