fasthttp is not recognizing the request as chunked if 'Transfer-Encoding' header is in lowercase and header normalization is disabled

Created on 8 Sep 2018 · 2Comments · Source: valyala/fasthttp

When header normalization is disabled fasthttp cannot recognize the fact that request's transfer encoding is chunked and treats it as a usual request. This leads to i/o timeout, since fasthttp thinks that there is more to come and request is finished.

Here is a client for repro.
client.go:

package main

import (
    "fmt"
    "log"
    "os"
    "time"

    "github.com/valyala/fasthttp"
)

func main() {
    url := os.Args[1]

    client := fasthttp.Client{
        ReadTimeout:                   2 * time.Second,
        DisableHeaderNamesNormalizing: true,
    }
    req := fasthttp.AcquireRequest()
    resp := fasthttp.AcquireResponse()

    defer fasthttp.ReleaseRequest(req)
    defer fasthttp.ReleaseResponse(resp)

    req.SetRequestURI(url)

    err := client.Do(req, resp)
    if err != nil {
        log.Fatal(err)
    } else {
        fmt.Print(resp.Header.String())
        fmt.Print(string(resp.Body()))
    }
}

And here is a simplest possible server.
bugger_server.go:

package main

import (
    "flag"
    "log"
    "net"
)

func main() {
    bug := flag.Bool(
        "bug",
        false,
        "controls whether to cause bug with fasthttp client",
    )
    address := flag.String(
        "addr",
        "localhost:8080",
        "specifies which address to use for the server",
    )
    flag.Parse()

    l, err := net.Listen("tcp", *address)
    if err != nil {
        log.Fatal(err)
    }

    log.Print("server started at " + *address)
    for {
        c, err := l.Accept()
        if err != nil {
            log.Print(err)
            continue
        }

        if *bug {
            c.Write([]byte("HTTP/1.1 200 OK\r\n" +
                "content-type: text/plain\r\n" +
                "transfer-encoding: chunked\r\n\r\n"))
        } else {
            c.Write([]byte("HTTP/1.1 200 OK\r\n" +
                "Content-Type: text/plain\r\n" +
                "Transfer-Encoding: chunked\r\n\r\n"))
        }

        chunks := [][]byte{
            []byte("24\r\nThis is the data in the first chunk \r\n"),
            []byte("1B\r\nand this is the second one \r\n"),
            []byte("3\r\ncon\r\n"),
            []byte("8\r\nsequence\r\n"),
            []byte("0\r\n\r\n"),
        }

        for _, ch := range chunks {
            c.Write(ch)
        }
    }
}

You run them as:

go run bugged_server.go --bug

and

go run client.go http://localhost:8080

For another example of a real world server with same behaviour see here.
Now, I'm not quite sure where RFCs stand on this, but I think I read somewhere that the clients ought to treat headers in a case-insensitive manner, also, other clients like curl or the one from Go's net/http have no problem with this kind of servers.

bug pendindevelopment

Source