Puppeteer-extra: [puppeteer-extra-plugin-block-resources] Error: Request is already handled!

[ItzRabbs]

This is similar to issue https://github.com/berstend/puppeteer-extra/issues/90 and https://github.com/berstend/puppeteer-extra/issues/91 but is occured sporadically with this being the only plugin enabled and no other request interception exists in the code.

Unfortunately, I'm not allowed to post the code that's running but it's scraping a terribly designed web page with a few dozen iframes that continuously change as the code loops through them. The iframes also use the same resources.

I could be wrong, but I believe this error is occuring because the onRequest function is called multiple times for such "duplicate" resources even though they were handled once before. Regardless, I'm thinking that a solution would be to check if the request has been handled before request.continue() or request.abort() is called, as puppeteer does in the code below.

https://github.com/puppeteer/puppeteer/blob/c2651c2b5cc888ebd0ca6be87063d9f98b9eb59c/lib/NetworkManager.js#L456

assert(!this._interceptionHandled, 'Request is already handled!');

[ItzRabbs]

Pull request: https://github.com/berstend/puppeteer-extra/pull/157

[liamcharmer]

Is this really solved?

[ItzRabbs]

I did this manually and it solved it for me. Doesn't seem to be merged in with the master.

[liamcharmer]

What can i do then to avoid this?? 🌊

[benallfree]

FYI I have a Puppeteer PR in to solve this https://github.com/puppeteer/puppeteer/pull/6733

[seth100]

FYI I have a Puppeteer PR in to solve this puppeteer/puppeteer#6733

I hope they will accept it as soon as possible!

[benallfree]

@seth100 Try the package I just published https://www.npmjs.com/package/enchant-puppeteer

[seth100]

@seth100 Try the package I just published https://www.npmjs.com/package/enchant-puppeteer

thanks @benallfree, but trying to install that I get 404

[benallfree]

@seth100 My fault, fixed in 0.0.2. Please install again: npm i enchant-puppeteer@latest.

[seth100]

@benallfree it works great! No error no more! Thank you

[benallfree]

@seth100 Awesome, please head over to my repo if you have any other issues. When @berstend sees this maybe I can talk to him about integrating it into puppeteer-extra 👍

[seth100]

@benallfree thanks, yes I hope it will be integrated into puppeteer-extra too!

[berstend]

Seems like the puppeteer maintainers are open to merge this in: https://github.com/puppeteer/puppeteer/pull/6735#issuecomment-759336801

In case they end up not patching this upstream we can incorporate the changes in puppeteer-extra as well.

[seth100]

Seems like the puppeteer maintainers are open to merge this in: puppeteer/puppeteer#6735 (comment)

In case they end up not patching this upstream we can incorporate the changes in puppeteer-extra as well.

great news, I'll stay tuned for that!