React-snap: Cannot write to stream after nil

[behnammodi]

version 1.17.0 work

[bdefore]

i believe this issue predates 1.17, but i was able to workaround it by adding a next tick setTimeout to the queue.end() call in react-snap. not ideal, but i didn't want to dig into the queue management logic, which i think is failing because the queue is being destroyed before it's finished (or the converse). i've forked and branched with this and a few other changes here: https://github.com/bdefore/react-snap/tree/fix/error-handling-extra Edit: Easier to read from the PR diff: https://github.com/stereobooster/react-snap/pull/263

specific change is here: https://github.com/bdefore/react-snap/blob/fix/error-handling-extra/src/puppeteer_utils.js#L223

[stereobooster]

Yes, I guess @bdefore is right. It was there. I saw it from time to time

[stereobooster]

@uxitten can you please try to install react-snap from branch and say if it helps or not

"react-snap": "https://github.com/stereobooster/react-snap.git#maybe-fix-for-248"

[behnammodi]

@stereobooster yes sure

[behnammodi]

@stereobooster does not work right. no error, but not crawl all url

version 1.17.0 crawl all url

[stereobooster]

Interesting. I don't know what is happening. Does anybody have a reproducible example?

[behnammodi]

i can't reproducible example

[matthinz]

It looks like this issue can occur when queue ends with files that will be skipped because they already exist on the file system.

I was able to work around it for our use case. I can try and get a better repro later this week.

[matthinz]

Ok, I am able to reproduce this issue. I pushed up an example repro here: https://github.com/matthinz/react-snap-issue-248-repro

The key seems to be setting "concurrency": 1 in package.json and having the last elements in queue be URLS that will be skipped because the file already exists on disk.