Node-fs-extra: Promisify Await WriteStream ReadStream Streams

Created on 26 May 2017 · 6Comments · Source: jprichardson/node-fs-extra

Hi @jprichardson @RyanZim I want to await the end of a stream. Are there any plans to add that? Would you be interested in a PR? If yes, should the api be to return the same old e.g. WriteStream with an added .promise() that can be awaited?

Edit - maybe the promises should be the names of the events? E.g. await fse.createWriteStream(path).end()?

promises wont-fix

Source

ubershmekel

Most helpful comment

Hi @RyanZim,

The nodejs s3 api only gave me a stream. I wanted to download a file to disk so I can process it in ffmpeg. So I needed to know when the file was done downloading.

Here's what I ended up with:

function streamPromise(stream) {
    return new Promise((resolve, reject) => {
        stream.on('end', () => {
            resolve('end');
        });
        stream.on('finish', () => {
            resolve('finish');
        });
        stream.on('error', (error) => {
            reject(error);
        });
    });
}

async function s3Download(srcBucket, srcKey, outputPath) {
    var objReq = s3.getObject({
        Bucket: srcBucket,
        Key: srcKey
    });
    let out = fse.createWriteStream(outputPath);
    objReq.createReadStream().pipe(out);
    return streamPromise(out);
}

I didn't do the due diligence to figure out the difference between finish and end but it worked so I moved on.

ubershmekel on 21 Jun 2017

👍9 ❤4

All 6 comments

@ubershmekel Sorry for the slow response. Why would you be using streams if you're waiting for all the data?

RyanZim on 21 Jun 2017

Hi @RyanZim,

The nodejs s3 api only gave me a stream. I wanted to download a file to disk so I can process it in ffmpeg. So I needed to know when the file was done downloading.

Here's what I ended up with:

function streamPromise(stream) {
    return new Promise((resolve, reject) => {
        stream.on('end', () => {
            resolve('end');
        });
        stream.on('finish', () => {
            resolve('finish');
        });
        stream.on('error', (error) => {
            reject(error);
        });
    });
}

async function s3Download(srcBucket, srcKey, outputPath) {
    var objReq = s3.getObject({
        Bucket: srcBucket,
        Key: srcKey
    });
    let out = fse.createWriteStream(outputPath);
    objReq.createReadStream().pipe(out);
    return streamPromise(out);
}

I didn't do the due diligence to figure out the difference between finish and end but it worked so I moved on.

ubershmekel on 21 Jun 2017

👍9 ❤4

I'm inclined to call this an edge case not worth supporting, though I totally get why you'd want this in a few cases.

Since this is such an edge case, I'd consider it better suited for a separate library.

Voting to close, attn @jprichardson & @manidlou for consensus.

RyanZim on 21 Jun 2017

👍1

I'd also say that's an edge case that is not fitting well here.

manidlou on 22 Jun 2017

👍1

Cool. Thanks for the response.

ubershmekel on 22 Jun 2017

Just to add another data point: I was also looking for this functionality.

I have a stream.Readable from a http response and want to write it to a file. I would think in a Node.js environment it is common to work with streams, so having native support in this library would be useful. (I would be worried that not everybody will write their own streamPromise function like above, but rather read the whole stream into a Buffer to then be able to use the standard fs-extra functions..)