Got: Support stream retries

[dmathieu]

While standard requests allow specifying a retries number or function, it seems got.stream doesn't handle it.
Yet a streaming request can fail just like any other one. Handling those retries seems quite useful.

Unfortunately, my node-fu isn't good enough to implement this while making sure it does work (I've tried doing so in my error handler, but it appears the second request never starts, as if duplexer3 couldn't handle multiple requests).

[sindresorhus]

I've tried doing so in my error handler, but it appears the second request never starts, as if duplexer3 couldn't handle multiple requests

Yes, you can't reuse a stream.

[sindresorhus]

I agree. We should support this.

[floatdrop]

got.stream should support retries, because it reuses requestAsEventEmmitter code. Yet after body content is piped into stream no retries will be applied on stream errors.

@sindresorhus should we cache streamed body to allow retries?

[sindresorhus]

should we cache streamed body to allow retries?

You mean so it can continue where it left off in case the connection is broken in the middle of a request? Is that even possible? Or am I misunderstanding?

[floatdrop]

I mean, that content will be resent from beginning, if something goes wrong. In case of input streams it mesns we should save what we have sent somewhere.

Отправлено с iPhone

27 мая 2017 г., в 17:49, Sindre Sorhus [e-mail] написал(а):

should we cache streamed body to allow retries?

You mean so it can continue where it left off in case the connection is broken in the middle of a request? Is that even possible? Or am I misunderstanding?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[sindresorhus]

@floatdrop Agreed. We might be able to build that upon what's being added in https://github.com/sindresorhus/got/pull/284.

[overlookmotel]

I've just published a module that uses got and resumes after network errors: https://www.npmjs.com/package/got-resume

If a request fails, the next request picks up from where the last one left off, using HTTP range headers.

It's designed for downloading large files where network errors are common.

[szmarczak]

Current workaround looks something like:

function make(tries = 0) {
    try {
        got.stream(URL)...
    } catch (err) {
        if ((err.code === 'ESOCKETTIMEDOUT' || err.code === 'ETIMEDOUT') && tries < 2) {
            make(tries + 1);
        } else {
            throw err;
        }
    }
}

make(); // throws if tried 2 times and got timed out error

[silentroach]

Why not retry requests if no input streams used? For example I use got to stream the response from server to parser.

[szmarczak]

@silentroach The thing is that if it errors while downloading you could get duplicated response.

[silentroach]

@szmarczak what about "bad" response status/codes? it will throw before I will get any data.

[szmarczak]

@silentroach Not necessarily. If the response body is short, then everything will be sent in one packet, thus the data event will be emitted immediately.

[silentroach]

sad :{

thank you for explanation