Deno: Deno http server serves around 25-40% less req/sec than that of Node.js

Created on 13 Jul 2020 · 9Comments · Source: denoland/deno

While doing performance comparison between Node.js and Deno, I noticed that Deno ~~performance gets affected when number of connections are increased~~ http server serves around 25-40% less req/sec than that of Node.js.

GitHub repo: https://github.com/trivikr/node-deno-http-benchmarks
UPDATE: The updated load testing benchmarks are posted in https://github.com/denoland/deno/issues/6722#issuecomment-658771957

System Information

~~Click to expand~~

$ sw_vers                           
ProductName:    Mac OS X
ProductVersion: 10.14.6
BuildVersion:   18G3020

$ sysctl -n machdep.cpu.brand_string
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz

$ deno --version
deno 1.1.3
v8 8.5.216
typescript 3.9.2

$ node --version
v14.5.0

$ autocannon --version
autocannon v5.0.1
node v14.5.0

with 100 connections

~~Node.js v14.5.0~~

node-100

~~Deno v1.1.3~~

deno-100

with 1000 connections

~~Node.js v14.5.0~~

node-1000

~~Deno v1.1.3~~

deno-1000

Source

trivikr

Most helpful comment

@narwy We have not started this work, but we likely will undertake this in the coming weeks/months. I have a specific plan of how to do it. ETA is 3 months.

ry on 25 Jul 2020

❤5 👍2

All 9 comments

So does Node.js. It just appears that performance drops off a bit quicker with Deno. We had hoped that a JS/TS based HTTP server was going to give us the performance we wanted, but that hasn't been achievable and it is on the roadmap to port it to Rust. (It is also the easiest way to get HTTP/2 and QUIC as well).

There are some HTTP benchmarks (https://deno.land/benchmarks) that are part of CI pipeline. A load test like this might be a good addition.

kitsonk on 13 Jul 2020

is on the roadmap to port HTTP server to Rust

Is there a link to this specific request on roadmap?
On quick search, I could find https://github.com/denoland/deno/issues/5625 which is referred from https://github.com/denoland/deno/issues/2473

trivikr on 13 Jul 2020

Updated benchmarks on [email protected] with [email protected]
Also, the headers were updated to return the headers in deno in https://github.com/trivikr/node-deno-http-benchmarks/commit/43e181039a7f7dbc7ef960fc826dbdf33ef2e55f

System Information

Click to expand

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04 LTS
Release:        20.04
Codename:       focal

$ lscpu | grep "Model name:" | sed -r 's/Model name:\s{1,}//g'
Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz

$ deno --version
deno 1.2.0
v8 8.5.216
typescript 3.9.2

$ node -v
v14.5.0

$ autocannon -v
autocannon v5.0.1
node v14.5.0

with 10 connections

Node.js 14.5.0

$ autocannon -c10 localhost:3000
Running 10s test @ http://localhost:3000
10 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────────┤
│ Latency │ 0 ms │ 0 ms │ 0 ms  │ 0 ms │ 0.01 ms │ 0.08 ms │ 10.99 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 32367   │ 32367   │ 44223   │ 45503   │ 43404   │ 3756.58 │ 32366   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.59 MB │ 3.59 MB │ 4.91 MB │ 5.05 MB │ 4.82 MB │ 417 kB  │ 3.59 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

434k requests in 10.06s, 48.2 MB read

Deno 1.2.0

$ autocannon -c10 localhost:3000
Running 10s test @ http://localhost:3000
10 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────────┤
│ Latency │ 0 ms │ 0 ms │ 0 ms  │ 1 ms │ 0.02 ms │ 0.15 ms │ 10.95 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬─────────┬────────┬──────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%  │ Avg      │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼────────┼──────────┼─────────┼─────────┤
│ Req/Sec   │ 19391   │ 19391   │ 27199   │ 30639  │ 26365.82 │ 3429.79 │ 19389   │
├───────────┼─────────┼─────────┼─────────┼────────┼──────────┼─────────┼─────────┤
│ Bytes/Sec │ 2.15 MB │ 2.15 MB │ 3.02 MB │ 3.4 MB │ 2.93 MB  │ 381 kB  │ 2.15 MB │
└───────────┴─────────┴─────────┴─────────┴────────┴──────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

290k requests in 11.06s, 32.2 MB read

with 100 connections

Node.js v14.5.0

autocannon -c100 localhost:3000
Running 10s test @ http://localhost:3000
100 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────────┤
│ Latency │ 1 ms │ 2 ms │ 3 ms  │ 4 ms │ 1.99 ms │ 0.75 ms │ 30.14 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 34463   │ 34463   │ 46719   │ 47295   │ 45481.6 │ 3724.53 │ 34460   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.83 MB │ 3.83 MB │ 5.19 MB │ 5.25 MB │ 5.05 MB │ 413 kB  │ 3.83 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

455k requests in 10.08s, 50.5 MB read

Deno v1.2.0

$ autocannon -c100 localhost:3000
Running 10s test @ http://localhost:3000
100 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬───────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max       │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼───────────┤
│ Latency │ 2 ms │ 2 ms │ 4 ms  │ 4 ms │ 2.28 ms │ 1.14 ms │ 104.55 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴───────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 29887   │ 29887   │ 35839   │ 36319   │ 35258.4 │ 1814.32 │ 29881   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.32 MB │ 3.32 MB │ 3.98 MB │ 4.03 MB │ 3.91 MB │ 201 kB  │ 3.32 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

353k requests in 10.08s, 39.1 MB read

with 1000 connections

Node.js v14.5.0

$ autocannon -c1000 localhost:3000
Running 10s test @ http://localhost:3000
1000 connections

┌─────────┬───────┬───────┬───────┬───────┬──────────┬──────────┬───────────┐
│ Stat    │ 2.5%  │ 50%   │ 97.5% │ 99%   │ Avg      │ Stdev    │ Max       │
├─────────┼───────┼───────┼───────┼───────┼──────────┼──────────┼───────────┤
│ Latency │ 22 ms │ 24 ms │ 31 ms │ 59 ms │ 25.54 ms │ 14.58 ms │ 311.06 ms │
└─────────┴───────┴───────┴───────┴───────┴──────────┴──────────┴───────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 22079   │ 22079   │ 40831   │ 41151   │ 38663.2 │ 5595.15 │ 22065   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 2.45 MB │ 2.45 MB │ 4.53 MB │ 4.57 MB │ 4.29 MB │ 621 kB  │ 2.45 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

387k requests in 10.22s, 42.9 MB read

Deno v1.2.0

$ autocannon -c1000 localhost:3000
Running 10s test @ http://localhost:3000
1000 connections

┌─────────┬───────┬───────┬───────┬───────┬──────────┬───────────┬─────────────┐
│ Stat    │ 2.5%  │ 50%   │ 97.5% │ 99%   │ Avg      │ Stdev     │ Max         │
├─────────┼───────┼───────┼───────┼───────┼──────────┼───────────┼─────────────┤
│ Latency │ 14 ms │ 31 ms │ 42 ms │ 43 ms │ 39.78 ms │ 239.53 ms │ 10013.98 ms │
└─────────┴───────┴───────┴───────┴───────┴──────────┴───────────┴─────────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev  │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Req/Sec   │ 19151   │ 19151   │ 23839   │ 24703   │ 23235.2 │ 1612.2 │ 19139   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Bytes/Sec │ 2.13 MB │ 2.13 MB │ 2.65 MB │ 2.74 MB │ 2.58 MB │ 179 kB │ 2.12 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴────────┴─────────┘

Req/Bytes counts sampled once per second.

232k requests in 10.23s, 25.8 MB read
69 errors (69 timeouts)

trivikr on 15 Jul 2020

Closing as:

This performance difference was known and called out in v1 announcement blog post under HTTP Server Performance https://deno.land/v1#http-server-performance
Deno is tracking this in benchmarks under "HTTP Server throughput" for realistic 10 keep-alive connections https://deno.land/benchmarks
- load tests can be added for 100 or 1000 connections, but v1 blog post recommends using some different programming language for higher throughput
As per comment by @kitsonk porting http server to rust is in the roadmap https://github.com/denoland/deno/issues/6722#issuecomment-657294324

trivikr on 16 Jul 2020

👍1

@trivikr @kitsonk We got any timelines/roadmap for http to be ported to Rust, can we also think about http2 when the port is due to happen, this seems to have been forgotten about also.

uncodable on 25 Jul 2020

@narwy We have not started this work, but we likely will undertake this in the coming weeks/months. I have a specific plan of how to do it. ETA is 3 months.

ry on 25 Jul 2020

❤5 👍2

@ry thanks for the update can't wait 👍 I think also the http3 draft has been released this month or something too, could be worth a read also.

uncodable on 25 Jul 2020

I think also the http3 draft has been released this month or something too, could be worth a read also.

QUIC is in draft-29 and it's not standardized yet.
Details https://datatracker.ietf.org/doc/draft-ietf-quic-http/

Support for HTTP/2, HTTP/3 and QUIC is being tracked in https://github.com/denoland/deno/issues/3995

trivikr on 26 Jul 2020

@narwy We have not started this work, but we likely will undertake this in the coming weeks/months. I have a specific plan of how to do it. ETA is 3 months.

@ry Is there an update on this ?