warp 0.2 is slow in benchmarks

[xfix]

In Framework Benchmarks, warp 0.2 is very very slow (check for entry called warp-rust): https://www.techempower.com/benchmarks/#section=test&runid=7f65c127-fad2-4a88-a6cb-5333c68362ef&hw=ph&test=plaintext

This isn't the issue with warp 0.1 (again, check for entry called warp-rust): https://www.techempower.com/benchmarks/#section=test&runid=d13756ac-4638-4c62-b1b4-cde1fd47a72a&hw=ph&test=plaintext

250,000 performance score seems much worse than 1,800,000. PHP running with php-fpm and nginx is faster than warp 0.2. It may be worth looking into why, because that's really unexpected.

Benchmark code: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Rust/warp-rust/src/main.rs

[seanmonstar]

Ah, try adding the unstable_pipeline() option, which enables HTTP/1 pipelining used by the plaintext benchmark.

warp::serve(routes)
    .unstable_pipeline()
    .run(..)
    .await;

[thomvil]

interesting. is this documented? i can't seem to find it. Maybe i am looking in the wrong place?

[seanmonstar]

It's purposefully marked with #[doc(hidden)] because it's an option that is only useful for those benchmarks. It shouldn't be used in the real world, but pipelined plaintext benchmarks aren't real world. 🤷‍♂

[xfix]

That seems reasonable, okay.

I find it surprising FrameworkBenchmarks uses HTTP/1.1 pipelining, considering no real HTTP clients support that making the benchmark hardly realistic, but oh well.

[danieljl]

That seems reasonable, okay.

I find it surprising FrameworkBenchmarks uses HTTP/1.1 pipelining, considering no real HTTP clients support that making the benchmark hardly realistic, but oh well.

Don't all major web browsers support HTTP/1.1 pipelining?

[leo-lb]

@danieljl @xfix All major browsers support HTTP/1.1 pipelining. And most HTTP libraries do too, because without that it means opening a new TCP connection for every request, which is obviously slow.

[seanmonstar]

Well, HTTP/1 pipelining usually refers to sending multiple requests before receiving their responses on the same connection. In that case, browsers eventually disabled the "feature", because it caused more problems than it solved. So in practice, the feature is rarely used in the real world.

[leo-lb]

@seanmonstar Alright, I confused Keep-Alive with pipelining then. Buggy servers is unfortunate, otherwise it is very useful. However it makes little sense to test HTTP/1.1 when HTTP/2 exists and is well supported by browsers.

[leo-lb]

@seanmonstar Question, can warp (hyper) do HTTP/2 without TLS? My platform isnt supported by rustls (only openssl) Thanks in advance

[xfix]

Hyper supports HTTP/2 without TLS as long as the client assumes the server supports HTTP/2. Note however that hyper doesn't support HTTP/1 cleartext negotation (Upgrade: h2c).

[phungleson]

Hi, I understand the reasons why unstable_pipeline is undocumented and it is fine but perhaps we should do something to update the benchmark score? At least it is helpful for PR sake and help add more adopters. For a relatively big project, choosing something that performs only 3% vs 80% or 90% sounds quite a bad decision, even no one knows what does it mean in reality.