Fasthttp: if using fasthttp.RequestCtx as context.Context, it will panic during server shutdown

[peczenyj]

If I use *fasthttp.RequestCtx as context.Context

during a server.Shutdown()

It will panic because we expect the context.Err() will be different than nil when the context id Done.

More information:
https://github.com/valyala/fasthttp/commit/51532b9517b3a01b442e0d58bcfd7fffbf183c68#commitcomment-32317617

related to issue #459

[peczenyj]

example of panic by composing the context with another context.
I trap signal 15 to perform a server.Shutdown() so to emulate this we must start an http request ( via curl )

func handler(ctx *fasthttp.RequestCtx) {
        logrus.Infof("start timeout countdown")
        c, cancel := context.WithTimeout(ctx, 5*time.Second)
        defer cancel()
        <-c.Done()
        logrus.Infof("start timeout done: %v", c.Err())
}

[peczenyj]

Of course, I forget update TestShutdownDone to check if ctx.Err() is not nil...

[Arnold1]

hi i have a few quesitons:
1) is it allowed to use ctx in context.WithTimeout and can i write to ctx after c.Done?
1) does my timeout handling work fine here, i want to make sure the go routine will be stopped in case of an timeout and it does not read from fasthttp.RequestCtx while a new request comes in?
1) is there an overhead of using context?
1) since fastHTTPHandler can be called concurrently (correct?), what will i need to put a mutex lock around?

type ResultData struct {
    data []int
}

func fastHTTPHandler(ctx *fasthttp.RequestCtx) {
    c, cancel := context.WithTimeout(ctx, 40 * time.Millisecond)
    defer cancel()

        resp := make(chan ResultData)

    go process(ctx, resp)

    select {
    case <-ctx.Done():
        fmt.Printf("error: processing timeout - %d\n", id)
        break
    case res := <-resp:
        fmt.Printf("finished processing\n")
    }

        // this does not make sense here, but lets say you put the res.data in a json...
    response = make([]int, 0, 100)
        for data := range res.data {
          response = append(response, data)
        }
        ctx.Write(response)
}

func process(ctx context.Context, respChan chan<- ResultData) {
    fmt.Printf("processing %d started...\n", id)

    sleepTime := time.Duration(rng.Int31n(80) * time.Millisecond)
    time.Sleep(sleepTime)

    fmt.Printf("%d ended after %s\n", id, sleepTime)

       // just demonstrates to return some data
       a := []int{1, 2, 3, 4, 5, 6, 7, 8}
       rand.Seed(time.Now().UnixNano())
       rand.Shuffle(len(a), func(i, j int) { a[i], a[j] = a[j], a[i] })

        respChan <- ResultData{a}
}

[erikdubbelboer]

1) You can write to ctx as long as the handler that the ctx came from hasn't returned yet.
1) Your code looks fine yes.
1) There is always a slight overhead yes. Contexts tend to generate garbage which is also means longer garbage collection pauses.
1) You don't need to use any mutex in the example you gave.

[Arnold1]

how to avoid using context.WithTimeout to get less gc?

[erikdubbelboer]

You can't avoid garbage when you use context. Then you need to build your own timeout mechanism.

[Arnold1]

@erikdubbelboer thx for info.

"Then you need to build your own timeout mechanism." ... any exampe for that? i still would like to return ctx.Write in case of an timeout...

my process actually talks to another server and waits for response.... and when the outside detects an timeout i want the sending to be stopped... how to achieve that efficiently?

func process(respChan chan<- ResultData) {
    httpReq := fasthttp.AcquireRequest()
    fhttpReq.Header.Add("Content-Type", "application/json")
    fhttpReq.Header.SetMethod("POST")
    fhttpReq.SetRequestURI(url + "/test")
    fhttpReq.SetBody(marshalledReq)
    fresp := fasthttp.AcquireResponse()
    err = fclient.DoTimeout(fhttpReq, fresp, 20*time.Second)

    // return fresp.Body() to the respChan
    a := parseResponse(fresp.Body())
    respChan <- ResultData{a}
}

[erikdubbelboer]

I would do the same as your code but instead of using context.WithTimeout I would use fasthttp.AcquireTimer.

client.DoTimeout is the best option then yes.

[Arnold1]

@erikdubbelboer
ok, try to use fasthttp.AcquireTimer.

• lets say there is a timeout how can i stop the go routine and the sending/receiving in func process(respChan chan<- ResultData) {?

• what does the 20*time.Second in fclient.DoTimeout?

• also does it make sense to manually call:

                fasthttp.ReleaseRequest(fhttpReq)
                fasthttp.ReleaseResponse(fresp)

after the packet is received to improve performance?

[erikdubbelboer]

func process(stop *int64, resp chan<- ResultData) {
    // process...
    // You can check if you should stop using
    if atomic.LoadInt64(&stop) == 1 {
        return
    }

    resp <- ResultData{}
}

func fastHTTPHandler(ctx *fasthttp.RequestCtx) {
    stop := int64(0)
    resp := make(chan ResultData, 2)

    go process(&stop, resp)
    go process(&stop, resp)

    responses := make([]ResultData, 0, 2)

    t := fasthttp.AcquireTimer()
    defer fasthttp.ReleaseTimer()

readLoop:
    for {
        select {
        case <-t.C:
            atomic.StoreInt64(&stop, 1)
            break readLoop
        case r <- resp:
            responses = append(responses, r)

            if len(responses) == 2 {
                break readLoop
            }
        }
    }

    // ctx.Write etc...
}

Now to get stop and resp into pools that's quite a bit harder. You'll need to make sure all process functions are done before you can put it in the pool. There are ways, but it's best if you try to find out yourself first.

[Arnold1]

@erikdubbelboer

• go process(&stop, resp) is twice, copy past error?

• right now i create a new go-routine every time fastHTTPHandler is called. is there anything i can optimize to avoid creating a single go-routine every time? creating a go-routine is also costly i guess?

[erikdubbelboer]

No, I thought you maybe wanted to do some parallel processing in multiple goroutines. Creating a goroutine is very cheap these days. But if it’s a bottleneck you could always use a goroutine pool.

[Arnold1]

@erikdubbelboer
i see that between 100- 1200 go routines run at the same time. but it could drop from 1200 -> 100 in 30 seconds or increase from 100 - 1200.

should i use https://github.com/hnlq715/goroutine-pool or https://github.com/Jeffail/tunny

[erikdubbelboer]

In the past I would have said yes. But with the recent improvements I wouldn't do this. Starting a new goroutine is really cheap in the latest Go. The overhead you're seeing is probably only a couple of microseconds (not milli, micro).