Aspnetcore: Kestrel timeots problem with large requests

[vnikopol]

Azure VM
ASP.NET Core 2.2
IIS, Kestrel
RequestBody - json, about 10mb

We are getting a problem with Kestrel timeouts when making large requests to a .NET Core API. We use IAuthorizationFilter before controller, after filter we have automatic redirect to AccessDenied page. But if request body is large , we get a 502 error instead 'ForbitResult'. As I understand it, if the request body is not handle in 5 sec, Kestrel will cut off the TCP connection, so we get 502.

How can we fix it?

2019-01-24 14:36:20.0437|3|INFO|Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker|url: https://localhost/api/test|action: Put|Authorization failed for the request at filter ClaimRequirementFilter'. 
2019-01-24 14:36:20.0437|1|INFO|Microsoft.AspNetCore.Mvc.ForbidResult|url: https://localhost/api/test|action: Put|Executing ForbidResult with authentication schemes (). 
...
2019-01-24 14:36:20.0719|32|INFO|Microsoft.AspNetCore.Server.Kestrel|url: |action: |Connection id "0HLK1U0KE50PM", Request id "0HLK1U0KE50PM:00000001": the application completed without reading the entire request body. 
2019-01-24 14:36:25.6520|33|INFO|Microsoft.AspNetCore.Server.Kestrel|url: |action: |Connection id "0HLK1U0KE50PM", Request id "0HLK1U0KE50PM:00000001": automatic draining of the request body timed out after taking over 5 seconds. 
2019-01-24 14:36:25.6520||WARN|Microsoft.AspNetCore.Server.Kestrel|url: |action: |Connection processing ended abnormally. Microsoft.AspNetCore.Connections.ConnectionAbortedException: The connection was timed out by the server.
   at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
   at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
   at System.IO.Pipelines.Pipe.ReadAsync(CancellationToken token)
   at System.IO.Pipelines.Pipe.DefaultPipeReader.ReadAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.BeginRead(ValueTask`1& awaitable)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequestsAsync[TContext](IHttpApplication`1 application)

[muratg]

How large is the request here?

[vnikopol]

@muratg

How large is the request here?

RequestBody is about 10mb json, but user doesn't have good Internet. Locally, I use proxy (Charles) with speed throttling to emulate this situation.

[muratg]

@Tratcher @halter73 Is there a setting to increase in this case?

[Tratcher]

See MinRequestBodyDataRate on KestrelServerOptions.Limits. 5 seconds is the grace period, not the limit. After that the request must upload at least 240 bytes per second or else the connection will be closed. The size of the body should not matter up until the 30mb RequestBodySizeLimit.

[vnikopol]

@Tratcher
Thanks, but I think it is not. Because if speed is least 240 bytes per second, i will get error: "Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate."

[halter73]

automatic draining of the request body timed out after taking over 5 seconds.

This appears to be the key part of the log. If the application doesn't consume the entire request body, Kestrel will attempt to drain the request body after the app completes in an attempt to reuse the connection.

However, if this attempt to drain the request body lasts over 5 seconds, Kestrel will give up and kill the connection. This can happen even if the upload is well over 240 bytes/second if the unconsumed request body is large enough.

[vnikopol]

@halter73 Yes, it looks like my situation. Are there any best practices to solve this?

[halter73]

@vnikopol You could drain the request body yourself in middleware.

[vnikopol]

@halter73

You could drain the request body yourself in middleware.

Thanks. I use context.Request.Body.CopyTo(Stream.Null); and it is work fine. But can I drain the request body in Filter Pipline (AutorizationFilter), not middleware?

[Tratcher]

Dedicating resources to an unauthorized user is a security risk. Only so much effort should be put into draining a request body for them. I suggest letting it fail as is. If this is a common scenario your users run into then make sure to verify their credentials on some prior GET action/page where the failure can more safely and easily be reported.

On another note, automatic redirects are not API friendly and imply you're using the wrong kind of auth (cookies?). For JWT bearer auth or similar you'd respond directly with a 403.

[vnikopol]

@Tratcher Thanks again.