Rocket: Web Framework Benchmarks

[hasanOryx]

Hello,
Why rocket is not appearing here, how it can benchmarked with other web frameworks, mainly with other rust based frameworks?

[SergioBenitez]

There are three reasons:

1. I personally believe that developers tend to misplace trust on benchmarks. As such, it might be a good thing that Rocket isn't on there. Perhaps then the decision of which framework to choose can be based on what's really important: ergonomics, productivity, security, and correctness.
2. Rocket doesn't and won't ever cater to benchmarks. That is, we won't make design decisions or changes to Rocket _just_ to be faster at a particular benchmark without further justification.
3. No one's written/submitted to TechEmpower.

This is also a duplicate of #296.

[marcbarbosa]

@SergioBenitez I totally agree with you about your points to not add Rocket to TechEmpower benchmark. But, I did implemented as a Rust/Rocket learning process.
It would be great if you could check it out and give some advices. Thanks!
https://github.com/TechEmpower/FrameworkBenchmarks/pull/3996

[DevotionGeo]

Would you guys please have a look at this thread at DevTalk, especially OvermindDL1's comment.

[jebrosen]

@DevotionGeo

Second of all, last I checked the rocket.rs code it’s using is forcing each request to come as a new tcp connection, which will significantly make it ‘look’ slower than most other libraries (the fact its even still as fast as it is, is quite impressive though). To simulate more ‘actual’ web usage all testing should be done with new tcp connections each time though.

This sounds to me like the change in https://github.com/TechEmpower/FrameworkBenchmarks/pull/5296 to disable keep_alive. The implementation of keep_alive in either hyper 0.10 (the underlying http library), or rocket 0.4, or both, seems to be buggy, but it is not a problem for all benchmarks and deployment setups. I don't know if this change is reflected in the latest TechEmpower results, but I do remember that before that change there was a very high error rate in the benchmark for rocket. I expect the async migration (#1065), coming in rocket 0.5, to fix all of these and several other issues at the connection level because it includes an update to hyper 0.13 - which is mentioned in that thread as well:

Third, the rocket.rs version they are using is still using the old version of hyper pre-async, which though still fast is not anywhere near as fast as modern hyper (which is used by the nightly version of rocket, which is the one most people use of rocket anyway until the next big release).

---

Finishing and testing async and other 0.5 work is currently my top priority for Rocket. I am interested updating the benchmark to use rocket 0.5 once released, if nobody beats me to it. At the same time, I plan to make any changes if necessary to ensure that it looks like an "idiomatic" rocket application and doesn't have any undue performance issues stemming from the particular implementation.

[DevotionGeo]

@jebrosen
Thank you for the great work you do!