Fasthttp: Nearly same results as net/http using wrk tool

Created on 1 Apr 2016 · 7Comments · Source: valyala/fasthttp

I was just tried to run a benchmark using wrk tool and found the following results:

fasthttp/main.go

package main

import "github.com/valyala/fasthttp"

func main() {
    fasthttp.ListenAndServe(":8080", func(ctx *fasthttp.RequestCtx) {
        ctx.SetContentType("text/plain; charset=utf-8")
        ctx.WriteString("Hello, World!")
    })
}

Results

❯ wrk -t20 -c50 -d20s http://127.0.0.1:8080
Running 20s test @ http://127.0.0.1:8080
  20 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   393.99us   57.68us  12.97ms   77.78%
    Req/Sec     5.06k   326.95     5.77k    67.75%
  2023259 requests in 20.10s, 285.57MB read
Requests/sec: 100664.12
Transfer/sec:     14.21MB

standard/main.go

package main

import (
    "io"
    "net/http"
)

func main() {
    http.ListenAndServe(":8080", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        io.WriteString(w, "Hello, World!")
    }))
}

Results:

❯ wrk -t20 -c50 -d20s http://127.0.0.1:8080
Running 20s test @ http://127.0.0.1:8080
  20 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   426.75us  100.82us   4.83ms   94.29%
    Req/Sec     4.69k   202.12     5.71k    69.50%
  1877157 requests in 20.10s, 232.73MB read
Requests/sec:  93390.63
Transfer/sec:     11.58MB

Env:
Go 1.6
GOMAXPROCS default

Am I a missing something, or is it the expected result.

question

Source

vishr

Most helpful comment

FYI, I get the following results on an ancient laptop:

fasthttp without pipelining

$ ./wrk -t2 -c512 -d10s http://localhost:8080 --latency
Running 10s test @ http://localhost:8080
  2 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.02ms    3.20ms  35.89ms   75.01%
    Req/Sec    37.24k     6.88k   51.51k    60.10%
  Latency Distribution
     50%    5.34ms
     75%    7.51ms
     90%   10.20ms
     99%   16.59ms
  741662 requests in 10.04s, 104.68MB read
Requests/sec:  73871.59
Transfer/sec:     10.43MB

net/http without pipelining:

$ ./wrk -t2 -c512 -d10s http://localhost:8080 --latency
Running 10s test @ http://localhost:8080
  2 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9.71ms    5.43ms  73.07ms   73.85%
    Req/Sec    24.38k     2.23k   32.23k    71.72%
  Latency Distribution
     50%    8.92ms
     75%   11.89ms
     90%   16.57ms
     99%   27.91ms
  486695 requests in 10.07s, 60.34MB read
Requests/sec:  48344.19
Transfer/sec:      5.99MB

fasthttp with 16 pipelined requests:

$ ./wrk -t2 -c128 -d10s http://localhost:8080 -s pipeline.lua --latency -- / 16
Running 10s test @ http://localhost:8080
  2 threads and 128 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.24ms    1.89ms  32.82ms   81.02%
    Req/Sec   324.85k    32.85k  421.28k    65.50%
  Latency Distribution
     50%    1.74ms
     75%    3.08ms
     90%    4.57ms
     99%    9.04ms
  6485840 requests in 10.05s, 0.89GB read
Requests/sec: 645568.23
Transfer/sec:     91.12MB

net/http with 16 pipelined requests:

$ ./wrk -t2 -c128 -d10s http://localhost:8080 -s pipeline.lua --latency -- / 16
Running 10s test @ http://localhost:8080
  2 threads and 128 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    28.08ms   27.28ms 323.61ms   87.56%
    Req/Sec    35.22k     3.08k   46.56k    68.50%
  Latency Distribution
     50%   19.84ms
     75%   36.39ms
     90%   61.49ms
     99%  134.22ms
  706581 requests in 10.07s, 87.60MB read
Requests/sec:  70166.42
Transfer/sec:      8.70MB

The contents of pipeline.lua:

init = function(args)
   request_uri = args[1]
   depth = tonumber(args[2]) or 1

   local r = {}
   for i=1,depth do
     r[i] = wrk.format(nil, request_uri)
   end
   req = table.concat(r)
end

request = function()
   return req
end

valyala on 3 Apr 2016

👍2

All 7 comments

Try setting -t parameter in wrk to the number of CPU cores in your system. This should minimize context switch overhead in wrk.
Increase -c parameter to 1000 or higher. This should load all the available CPU resources.
Use requests' pipelining in order to overcome operating system's limit on the number of context switches between wrk and webserver.

These suggestions should significantly improve fasthttp performance numbers. See plaintext test results in TechEmpower benchmarks round 12 for the reference.

valyala on 2 Apr 2016