Kestrelhttpserver: ASP.NET Core throughput while await on non cpu bound task

[bnayae]

I was publishing the same issue to the ASP Hosting branch but it seem that it may be

When I compare ASP.NET and ASP.NET Core Web API (over IIS Express).
It's seem that even those I'm awaiting Task.Delay it doesn't handle the next request and behave like
having Thread Pool starvation.

I'm using the same HttpClient instance with MaxConnectionsPerServer = 100,
I was also testing it on Docker deployment (with VS 2017 docker support) and It was behaving the same.

the following code work great on ASP.NET and poor on ASP.NET Core
C# public async Task<string> Get(int id) { Debug.Write("."); await Task.Delay(TimeSpan.FromSeconds(2)); return $"#{id:00}"; }
I was testing it with the following code (using 50 concurrent request):

    private static async Task CallAsync(int times)
    {
        var sw = Stopwatch.StartNew();
        using (var http = new HttpClient(new HttpClientHandler() { MaxConnectionsPerServer = 100 }))
        {
            var tasks = from i in Enumerable.Range(0, times)
                        let url = $"{BASE_URL}/{i}"
                        select InvokeAsync(http, url);
            string[] responses = await Task.WhenAll(tasks);
        }
        sw.Stop();
        Console.WriteLine($"\r\n\r\nTOTAL DURATION: {sw.Elapsed.TotalSeconds:N3} seconds");
    }

    private static async Task<string> InvokeAsync(HttpClient http, string url)
    {
        var latency = Stopwatch.StartNew();
        Console.Write("-");
        string response = await http.GetStringAsync(url);
        latency.Stop();
        Console.Write($"{latency.Elapsed.TotalSeconds:0.0}, ");
        return response;
    }

[davidfowl]

@bnayae I mentioned this on the other issue but you haven't changed the code. Can you use a single http client instead of creating one per request?

Also what OS are you running on?

[benaadams]

If you change from max open connections to 10? MaxConnectionsPerServer = 10 as mentioned previously?

[bnayae]

I did it, I was running with single instance and MaxConnectionsPerServer = 100 but it was behaving more of the same (far from the expectation)

[davidfowl]

@bnayae What OS are you running on? Windows? Can you provide your entire application?

[bnayae]

I'm working on Win 10 Enterprise (Surface pro 3).
the code for reproduce available here

[davidfowl]

@bnayae Can you run that on a windows server machine and see if it repros?

[bnayae]

I don't have Win Server installed on any of my machines but I will try to deploy it to Azure and see how it behave.

[bnayae]

I was having problem with the Azure deployment from VS 2017, you may try to run the source code and see the problem

[davidfowl]

@bnayae What are the repro steps? Can you provide your client program as well?

[bnayae]

With help of my colleague Ido Flatow we found the issue.
The difference was that ASP.NET wasn't removed from Iis Express when I stop debugging while ASP.NET Core did. issue happens only before Iis Express becoming fully functional.

[halter73]

I don't think I completely understand what the problem/fix was. Is there something ASP.NET Core should be doing differently so other people don't run into this issue? Either way, I'm glad things are now performing well for you.

[kevin-meyer-aarons]

@bnayae Are you saying there is an issue, but only when the asp.net core application is hit under load while starting up? If so, I'm experiencing the same behavior.

I hacked up some middleware that works around the issue:

public class ThrottledStartupMiddleware
{
    private const int NOT_INITIALIZED = 0;
    private const int INITIALIZING = 1;
    private const int INITIALIZED = 2;

    private readonly RequestDelegate _next;
    private static int _state = NOT_INITIALIZED;

    public ThrottledStartupMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task Invoke(HttpContext context)
    {
        while (INITIALIZING == Interlocked.CompareExchange(ref _state, INITIALIZING, NOT_INITIALIZED))
        {
            await Task.Delay(TimeSpan.FromMilliseconds(100));
        }

        try
        {
            await _next(context);
        }
        finally { 
            Interlocked.CompareExchange(ref _state, INITIALIZED, INITIALIZING);
        }
    }
}

[Tratcher]

@pan-wang ?

[pan-wang]

@bnayae could you please give a summary about the issue here? I am not sure about what you meant " ASP.NET wasn't removed from Iis Express" and "before Iis Express becoming fully functional".

[muratg]

Closing this one as the discussion seem to have stopped. @kevin-meyer-aarons / @bnayae feel free to file another issue to continue the discussion.

[jonasrdm]

I think this topic needs more attention. Our system randomly stopped to respond requests when starting up, and we noticed that it only happened when a lot of requests where being made while it was starting. I don't understand exactly what is really happening but @kevin-meyer-aarons solution saved me. Thanks dude! =D

[pan-wang]

@jonasrdm So the workaround is to do some throttle control at the back-end?