Aws-sdk-net: Synchronous operations are disallowed. Call ReadAsync or set AllowSynchronousIO to true instead.

[NinoFloris]

Aws sdk is doing synchronous reads on async calls

Full stack trace:

System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.InvalidOperationException: Synchronous operations are disallowed. Call ReadAsync or set AllowSynchronousIO to true instead.
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.Read(Span`1 buffer)
   at Amazon.Runtime.Internal.Util.WrapperStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at Amazon.Runtime.Internal.Util.PartialReadOnlyWrapperStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at Amazon.Runtime.Internal.Util.WrapperStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at Amazon.Runtime.Internal.Util.HashStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at Amazon.Runtime.Internal.Util.ChunkedUploadWrapperStream.FillInputBuffer()
   at Amazon.Runtime.Internal.Util.ChunkedUploadWrapperStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at Amazon.Runtime.Internal.Util.WrapperStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.<>c.<BeginReadInternal>b__43_0(Object <p0>)
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__274_0(Object obj)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location where exception was thrown ---
   at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
   at System.Net.Http.StreamToStreamCopy.<CopyAsync>g__DisposeSourceAsync|0_0(Task copyTask, Stream source)
   at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
   at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendRequestContentWithExpect100ContinueAsync(HttpRequestMessage request, Task`1 allowExpect100ToContinueTask, HttpContentWriteStream stream, Timer expect100Timer, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)

Repro (configure _client first):

public class Foo {
    private class File
    {
       public long? Filesize { get;set; }   
    }

    AmazonS3Client _client;
   string _bucket;

    public async Task Upload(File file, Stream stream)
    {
        long effectiveLength = 0;
        if (file.Filesize != null) 
        {
            if (!stream.CanSeek)
                throw new ArgumentException("Unseekable streams have to pass in a file size"); 
            effectiveLength = stream.Length - stream.Position;
        }
        else 
        {
            effectiveLength = file.Filesize.Value;
        }

        var request = 
            new PutObjectRequest(
                AutoResetStreamPosition = stream.CanSeek,
                Key = "bar",
                BucketName = _bucket,
                InputStream = stream
            )
        request.Headers.ContentLength = effectiveLength;

        await _client.PutObjectAsync(request)
    }
}

Call Upload with a Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream e.g. by uploading a file via http to a controller action that will call Upload, anything from AspNetCore 3.0 and up will do.

This is happening because none of the streams in Amazon.Runtime.Internal.Util actually override any of the Async methods (ReadAsync, BeginRead); those will call the sync versions by default.

Related:
https://github.com/aws/aws-sdk-net/issues/675
[Announcement] AllowSynchronousIO disabled in all servers

[NinoFloris]

@normj if you're serious about what's in your bio this should get some prio.

[normj]

@NinoFloris we are aware of the issue. This is also critical to get done for WASM Blazor support which is single threaded. We are working on a PR to update all of stream implementations to have an async implementation. I hope to give you an update on a release status soon.

[btecu]

Is there an update here? It looks like #1416 was adding ReadAsync, which is needed.

[oliverjanik]

How's this tracking? Stream.ReadAsync has existed since .NET 4.5 (released 8 years ago) and async is very much the norm these days. In fact I'd happily forgo sync versions if you only gave me async.

I'd really appreciate if you could elevate this library to be compatible with modern dotnet 👍