Runtime: HttpClient Performance Slow Compared to .NET 4.7

[narciero]

I have conducted a basic performance test of HttpClient in .NET Core 2.0 versus HttpClient in .NET Framework 4.7 and have noticed a gap in performance. The .NET Framework 4.7 HttpClient/WebClient outperforms .NET Core's HttpClient by ~1.5-2x in terms of how long it takes to complete a batch of _n_ requests.

Testing

The test is a console app (run in release mode on Windows 7 16GB 3.5GHz) with identical code for .NET Core/Framework that follows this sequence:

1. Create a single, shared HttpClient instance with a maximum of _n_ (for testing _n_=10,100) connections.

   // .NET Core 2.0
   var httpClient = new HttpClient(new HttpClientHandler { MaxConnectionsPerServer = 100 });
   
   // .NET Framework 4.7
   ServicePointManager.DefaultConnectionLimit = 100;
   var httpClient = new HttpClient();
   
   // .NET Framework 4.7 - WebClient instance is created ONCE PER REQUEST
   ServicePointManager.DefaultConnectionLimit = 100;
   var webClient = new WebClient();
   

2. Start 10,000 simulataneous requests and time how long each request takes to complete + how long all take to complete. Here is code demonstrating how requests are made / timed.

private void RunTest(int count)
{
    // requests is a list of Request data structures used to store individual request latency / responses
    var requests = Enumerable.Range(0, count).Select(j => CreateRequest(j)).ToList();

    // this stopwatch is for duration of all requests
    var stopwatch = Stopwatch.StartNew();

    // start all requests and wait for completion
    var requestTasks = requests.Select(MakeRequest).ToArray();
    Task.WaitAll(requestTasks);

    stopwatch.Stop();
    // total run time = stopwatch.ElapsedMilliseconds
}

private Task MakeRequest(Request request)
{
    var stopwatch = Stopwatch.StartNew();
    var response = await httpClient.GetStringAsync(request.Url);
    stopwatch.Stop();

    // save request duration and response
    request.DurationMs = stopwatch.ElapsedMilliseconds;
    request.ResponseId = ParseResponse(response);
}

I am testing against a basic python server that is well under capacity to rule out any server side bottlenecks. The server returns a simple JSON response containing an id field which is used to validate the HttpClient responses.

{ "id": "some_identifier" }

JSON deserialization / response validation is NOT included in performance stats.

Results

These are average statistics (in milliseconds) of 5 separate runs of 10,000 requests each. It is worth mentioning that the actual time spent on each request was much less with .NET Core's HttpClient, it's just that the batch of requests takes longer to complete as a whole vs .NET Framework.

| framework | total run time (ms) | avg req time (ms) | median req time (ms) | total time spent on requests (s) |
|---------------|---------------------|-------------------|----------------------|----------------------------------|
| .NET Core 2.0 | 5614 | 216 | 282 | 777 |
| .NET 4.7 | 3055 | 1355 | 1339 | 10,585 |

[MarinAtanasov]

Frankly, I am surprised that you could do 10000 requests with this code.

I've tried something similar and got the same error as this person.

Going back to performance, I've tried testing AspNet Core 2.0's REST performance using a DotNet Core 2.0 console application (yes, I know that there are tools for that) and the client ended up being the bottleneck for my little test. I was expecting to be able to create more requests than the server can respond to. If I create too many requests at the same time, I get the linked error on the client. If I create them sequentially in several threads, the client's process has a much higher CPU usage than the server.

[narciero]

@MarinAtanasov I was initially receiving that error as well. Setting MaxConnectionsPerServer fixed the problem for me, but yes the client was still the bottleneck.

[MarinAtanasov]

Considering that there are tools which can measure upwards of a million requests per second (written in other languages), what are the max requests per second which can be achieved through a .Net Core 2.0 client and how much CPU/memory would it consume?

@narciero, from your test it would seem that you cannot do more than two thousand requests / second. Am I missing something obvious here?

[narciero]

@MarinAtanasov its important to note that i tested using MaxConnectionsPerServer set to 100, but yes under those conditions 2k req/s is what i observed.

[Priya91]

cc @karelz @wfurt Performance of HttpClient, is this something that can be measured in the new lab setup.

[wfurt]

The performance will depend on may factors - like using keep-alive, DNS, etc.
It is interesting that average request time for 2.0 is much shorter but overall duration is longer.
That may be because the time is spent in underlying native call. But that is hard to know for sure without more details.

I don't have any benchmarks for .net framework yet but it is on TODO list.

[karelz]

Yes, this is very much on our list for ManagedHandler. We plan to compare Windows vs. Linux vs. Desktop vs. other platforms vs. old HttpClientHandler.
We will need to measure more characteristics than just this - different payload sizes, number of concurrent requests, measure CPU and network utilization, try scale up (1 vs. 2 vs. 4 vs. 8 cores), etc.

Given that it is on our backlog and the data above is not complete and will need to be re-measured anyway, I don't think we need this issue opened. Closing (let me know if you think it is wrong).

[TechnikEmpire]

Do all similar objects suffer from this or is it just HttpClient?

[karelz]

This particular issue lacks details and deeper analysis. The results are a bit suspicious and were not attempted to reproduce by anyone else. There is a suspicion that the compared platforms don't use the same settings under the hood (e.g. MaxConnectionsPerServer is set differently on each platform). Also, other perf comparisons typically show better performance on .NET Core than on .NET Framework.
It is premature to claim that HttpClient "suffers from this" as we do not know at all what "this" is. At this moment I consider it a test issue, until proven otherwise.

[MarinAtanasov]

I am not sure how it is compared to .NET Framework, but we definitely know that HttpClient in .NET Core is incredibly slow and heavy. That is proven by the fact that we can expect performance improvements up to 1000% (x10) in .NET Core 2.1. Source: .NET Core 2.1 Roadmap.

@TechnikEmpire, if you are using microservices, expect this to be a bottleneck. We had a framework performance comparison for one of our microservices where .NET Core had about 50% of the performance of Node or Go. We should get our hands on .NET Core 2.1 sometime during the summer. Until then, I do not know if there are any fast alternatives to using HttpClient.

[TechnikEmpire]

@MarinAtanasov I'm using the client to handle the upstream side of a local transparent filtering proxy and yeah it shows. I guess I'll just target .net full for now to avoid the issue.

[karelz]

@MarinAtanasov do you have any evidence that HttpClient in .NET Core is incredibly slow and heavy? Are you measuring Windows or Linux?
While we were able to speed up specific scenarios (esp. on Linux) in 2.1 (up to 10x), I would not use it as a "proof" that HttpClient in 2.0 is slow in all scenarios.

@TechnikEmpire if you have a specific code which shows performance difference between .NET Framework and .NET Core - especially on Windows, we would be very interested in getting our hands on it. Is it something you could share as a perf test?

[TechnikEmpire]

@karelz I'll see if I can't whip up a test case. I test my proxy code by running an external Apache tool to measure latency and throughput so I'm wondering if you'll accept such a test case.

[wfurt]

That may be ok @TechnikEmpire as well as the original post simply measured total time to completion.
In ideal case we would get simple console app + instructions how to run it with external dependencies.
It should be full app one can look at and run, not just code fragments. Ability to reproduce and verify claims easily will greatly improve chance this will be addressed.

[karelz]

@TechnikEmpire ideally I would like to see an app with workload which we can measure with our tools on our side. If you have recommendations for existing tools, we're all ears of course. It would be just unfortunate if the workload is tied to them.

[TechnikEmpire]

Ok thanks I'll see what I can do. I've written a couple of projects heavily relying on http client (one a crawler and one a proxy) and the performance of both wasn't stellar but that's anecdotal of course. Will let you know if I can out something together I just have a few things on my plate atm.

[soualid]

We are migrating a .net application (4.5 under windows) to dotnet core and we also noticed that HttpClient is ~20 times slower using dotnet core (2.0.5) under linux.

Our application is using a lot of microservices, we'll have to wait for this issue to be addressed (hopefully in dot net core 2.1) before moving forward.

[karelz]

@soualid you can try 2.1 right now. It has the new networking stack mentioned above, which is faster, esp. on Linux.

• Try .NET Core 2.1 Preview1 - it is opt-in there (see details):
  
  
  
  • Environment variable: COMPlus_UseManagedHttpClientHandler=true
  
  
  • AppContext: System.Net.Http.UseManagedHttpClientHandler=true
  
  
• Or try our recent daily builds - it is default networking stack there.

[soualid]

We just tried and we can confirm that .NET Core 2.1 Preview 1 is way more performant than 2.0 under linux using docker containers, it now takes around 90ms for HttpClient to call our microservices, where it tooks ~400ms using 2.0.5.

Congrats, we are now eagerly waiting for the 2.1 GA version. :-)

[BKlippel]

It can't come too soon, in my experience so far, .net core performance is unacceptable for production use, compared even to .net framework, Let's not bother comparing it to Node, Java, Python, etc, where it gets it's ars soundly handed to itself repeatedly.

[svick]

@BKlippel Can you share the data you're basing that on? Have you tried using .Net Core 2.1 RC1 to see if it's an improvement?

[k2ly2n]

request.ResponseId = ParseResponse(response);

not sure what you have in there but this might be the reason for increased total run time. this is the only part not adding up to individual request processing times.

[sgf]

@BKlippel Can you share the data you're basing that on? Have you tried using .Net Core 2.1 RC1 to see if it's an improvement?

HttpClient is very slow, some open source projects based on the underlying httpClient are being criticized.

such as the project https://github.com/ThreeMammals/Ocelot

its performance is much lower than project https://github.com/Kong/kong

In order to get performance as well as some other reason people have to give up using .Net core.

[stephentoub]

@sgf, you've been going around commenting on a variety of issues, making broad negative claims about performance, with nothing to substantiate those claims. If you're hitting a performance issue, please open a new issue that includes a repro of the problem and specifics about what the exact problem is.

[TechnikEmpire]

Ohhh snap got called out sup dog

[sgf]

@sgf, you've been going around commenting on a variety of issues, making broad negative claims about performance, with nothing to substantiate those claims. If you're hitting a performance issue, please open a new issue that includes a repro of the problem and specifics about what the exact problem is.

Thank you for your reminder.
The work of the .net core team is very good and very amazing. It is advisable to be appreciated and encouraged.

But isn't the issue used to give feedback?
Do you want people to post any appreciation and praise in the channels that were originally used for feedback?

The purpose of replying to some issues is simply to let the community administrator not easily ignore some of the user's problems. At the same time, I have encountered similar problems.
I've seen community administrators turn off user feedback relentlessly, and although some issues have been raised for years, they have not been effectively addressed.

People often spend a lot of time discussing in a issue, but it is easily closed by community administrators.

[sgf]

@sgf, you've been going around commenting on a variety of issues, making broad negative claims about performance, with nothing to substantiate those claims. If you're hitting a performance issue, please open a new issue that includes a repro of the problem and specifics about what the exact problem is.

Some words are not very good, but there are such things happening around me (for example, the people around me have been transferred from .net to java/go/rust in recent years. There are many reasons for this because the ecology of .net is not good enough, or Performance is not high enough). As an ordinary .net developer, I am a little anxious, and I hope that .net is getting better and better.

[TechnikEmpire]

As someone who lives in .net land both at work and at home, if you're getting generally bad performance with .net, its your programming skills, not the platform.

[sgf]

As someone who lives in .net land both at work and at home, if you're getting generally bad performance with .net, its your programming skills, not the platform.

I agree that the skill is indeed one aspect.
But Clr performance can't be ignored.

A friend of mine submitted his project BeetleX to TechEmpower .
the result No.23 is not good enough (A far cry from the No.1 -- actix a rust project).

He thinks it may be mainly a performance issue with CLR, SQLDriver and Socket.

[TechnikEmpire]

As someone who lives in .net land both at work and at home, if you're getting generally bad performance with .net, its your programming skills, not the platform.

I agree that the skill is indeed one aspect.
But Clr performance can't be ignored.

A friend of mine submitted his project BeetleX to TechEmpower .
the result No.23 is not good enough (A far cry from the No.1 -- actix a rust project).

Means nothing. I'd like to see the results of a performance profiler hooked directly up to tests against the project in question. As is, the stuff you've linked means nothing. Unless you're going to submit a PR or learn the basics of programming, please stop making noise on repositories that thousands of people are following.

[sgf]

As someone who lives in .net land both at work and at home, if you're getting generally bad performance with .net, its your programming skills, not the platform.

I agree that the skill is indeed one aspect.
But Clr performance can't be ignored.
A friend of mine submitted his project BeetleX to TechEmpower .
the result No.23 is not good enough (A far cry from the No.1 -- actix a rust project).

Means nothing. I'd like to see the results of a performance profiler hooked directly up to tests against the project in question. As is, the stuff you've linked means nothing. Unless you're going to submit a PR or learn the basics of programming, please stop making noise on repositories that thousands of people are following.

got it

[bugproof]

I'm using Refit in WPF app. I ported it to .NET Core (3.1 now) and HTTP requests cause major UI slowdowns/freezes ( I send them inside WPF commands). Refit uses HttpClient under the hood so this might be related... Initially, I thought it's WPF problem but after removing HTTP call it came back to normal.

[stephentoub]

Refit uses HttpClient under the hood so this might be related

It's possible it's using it "incorrectly". If it's creating and disposing a new HttpClient() instance on every access, that is going to be more expensive on .NET Core. In .NET Framework, HttpClient is a thin wrapper around HttpWebRequest, with the connection pool managed separately, so disposing of the HttpClient instance won't tear down connections in the connection pool. In .NET Core, the handler owned by HttpClient owns the connection pool, and if you do new HttpClient(), it's creating a new handler that it owns. It's possible the slow down you're seeing is because, due to the way HttpClient is being used, it's creating a new connection on every access in .NET Core and it's not in .NET Framework.

[bugproof]

@stephentoub That was it! I was creating and disposing my API client (which wraps Refit) on every access. I didn't know it makes any difference in .NET Core 😮 . Thanks a lot 😄 it works fine now just like it did before.

[stephentoub]

@bugproof, great, glad you got it sorted out.