Azure-sdk-for-net: [BUG] Special Character Causes Queue Message to Fail

[speedy-ms]

Describe the bug
When you try to place a message on a queue with a "�" character it causes an exception with the message " XML specified is not syntactically valid."

Expected behavior
The character would be escaped and therefore queued and dequeued successfully.

Actual behavior (include Exception or Stack Trace)
The call to place the message onto the queue causes a RequestFailedException to be thrown. Here is the stack trace:

Azure.RequestFailedException : XML specified is not syntactically valid.
RequestId:cd13cfd1-3a38-4156-9756-75d7d2f1a1e6
Time:2020-04-16T02:00:31.5292245Z
Status: 400 (XML specified is not syntactically valid.)
ErrorCode: InvalidXmlDocument

Additional Information:
LineNumber: 1
LinePosition: 78
Reason: Error parsing Xml content

Headers:
Server: Windows-Azure-Queue/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id: cd13cfd1-3a38-4156-9756-75d7d2f1a1e6
x-ms-version: 2018-11-09
x-ms-error-code: InvalidXmlDocument
Date: Thu, 16 Apr 2020 02:00:31 GMT
Content-Length: 327
Content-Type: application/xml

   at Azure.Storage.Queues.QueueRestClient.Messages.EnqueueAsync_CreateResponse(ClientDiagnostics clientDiagnostics, Response response)
   at Azure.Storage.Queues.QueueRestClient.Messages.EnqueueAsync(ClientDiagnostics clientDiagnostics, HttpPipeline pipeline, Uri resourceUri, QueueSendMessage message, String version, Nullable`1 visibilitytimeout, Nullable`1 messageTimeToLive, Nullable`1 timeout, String requestId, Boolean async, String operationName, CancellationToken cancellationToken)
   at Azure.Storage.Queues.QueueClient.SendMessageInternal(String messageText, Nullable`1 visibilityTimeout, Nullable`1 timeToLive, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.Queues.QueueClient.SendMessageAsync(String messageText, Nullable`1 visibilityTimeout, Nullable`1 timeToLive, CancellationToken cancellationToken)
   at Azure.Storage.Queues.QueueClient.SendMessageAsync(String messageText)
   at MessageQueueProviderTests.invalid_message()

To Reproduce
The following xunit test will trigger the exception (assuming you have the queue "test-queue" in Azure storage emulator):

        [Fact]
        public async Task invalid_message()
        {
            var messageText = "String with � character";
            var queueClient = new QueueClient("UseDevelopmentStorage=true;", "test-queue");

            await queueClient.SendMessageAsync(messageText);
        }

Environment:

• Azure.Storage.Queues 12.3.0
  
  
  
  • .NET Core SDK 3.1.101
  
  

[msftbot[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

[amishra-dev]

Hi @speedy-ms ,
This is by design. We expect the message to be a valid xml body message. A quick workaround for you would be to base64 encode the string. Please ack if this works for you.

Thanks

[speedy-ms]

Hi @amishra-dev,

I will base64 encode our messages going forward, I think that is one issue that we have faced when changing from Microsoft.Azure.Storage.Queue to Azure.Storage.Queues - that, in the former, all messages are automatically encoded to and decoded from base64 whereas the latter it does not appear to happen.

[sanisoclem]

Hi @speedy-ms ,
This is by design. We expect the message to be a valid xml body message. A quick workaround for you would be to base64 encode the string. Please ack if this works for you.

Thanks

Would be helpful if this was written in the docs somewhere - neither the function signature nor the docs convey this information.

Also from a consumer point of view, wouldn't it be better if QueueClient did the unescaping/escaping? I imagine that a user of this API won't really care about the underlying format (xml). What's important is whatever string we pass to SendMessage will be equal to the string that is returned by ReceiveMessage.

I encountered this error today when I sent a message with a &. It was puzzling because there was no indication in the API that you need an xml escaped string (and the docs says The message content must be a UTF-8 encoded string that is up to 64KB in size.) but the exception says its invalid xml. It's not very intuitive.

[kasobol-msft]

This has been addressed in https://github.com/Azure/azure-sdk-for-net/pull/16689 along with base64 option and available in https://www.nuget.org/packages/Azure.Storage.Queues/12.5.0