Lighthouse: pages the modify fetch mess with gathering

[parmalarm]

Provide the steps to reproduce

1. Run LH on
2. Go to https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fwww.tiffany.com and the page speed score does not load.

What is the current behavior?

I'm trying to pull lighthouse scores for www.tiffany.com but every time i do i receive a "backend error" message. This issue started around Sep 4th, prior to that date, i was able to pull lighthouse KPI's however the SEO score has never been able to populate.

Today i get the following message: Lighthouse returned error: generic::internal: Extension result empty. Lighthouse calling Page.navigate with https://www.tiffany.com/ Uncaught (in promise) Error: Extension timed out. No call to exit() before render timeout. at wrs.Extension.onTimeout_ (http://ghost-rider/devtools_script_api.js:5525:13) at http://ghost-rider/devtools_script_api.js:708:9 at Map.forEach () at chromium.DevTools.Connection.dispatchMessage (http://ghost-rider/devtools_script_api.js:707:76) at chromium.DevTools.Connection.onJsonMessage_ (http://ghost-rider/devtools_script_api.js:689:65) at :1:40 Dismiss

What is the expected behavior?

I expect to see a desktop and mobile speed scores pulled i from lighthouse.

Environment Information

• Affected Channels: All channels
• Lighthouse version:
• Chrome version:
• Node.js version:
• Operating System: windows & mac

Related issues

[exterkamp]

This is failing in PSI, DevTools, and CLI because of

Runtime error encountered: Protocol error (Runtime.evaluate): Promise was collected

I'll look more into this, but might be a duplicate of: #5973

[exterkamp]

Seems like https://www.tiffanys.com/ is consuming our fetch() functions and erroring on them?

See:

function a(e, t) {
    return new Promise(function(n, r) {
        var i = e.clone(); // death here with: TypeError: e.clone is not a function
        "POST" === e.method ? e.json().then(function(r) {
            ...

This is killing the Lighthouse run on the robots.txt gatherer when we try to fetch the robots.txt page from what I can see.

[connorjclark]

Yup, this page is intercepting the fetch command. Since this page is minified and does not have source maps, we can't investigate further.

My first inclination is that we should cache the native fetch (just like we do for other things that developers like to override ... see https://github.com/GoogleChrome/lighthouse/blob/648248e270deeef33e1b16b3420b43cc97230a97/lighthouse-core/gather/driver.js#L1472 ). https://www.tiffanys.com/ keeps timing out so I can't confirm this will fix it right now.

[patrickhulce]

Seems like the better fix is to have the robots txt gatherer useIsolation instead. At some point we should probably make that the default and force gatherers to opt-in to page context if they absolutely need it.

[exterkamp]

This is happening in lighthouse/lighthouse-core/lib/stack-collector.js as well, and similarly needs to run in isolation. I added a comment in #10130 and opened #10133 to fix this.

This is due to the workbox library detector running a fetch command I think.

Applying that line gives a result 🎉

[gsuberland]

If an additional data point helps, I am also running into this bug on https://poly.nomial.co.uk/

I believe it may be related to content security policy.

[philkrie]

@patrickhulce testing in Chrome Canary 85, I'm still getting this issue for tiffany.com (confirmed using LH v6.0.0), so it looks like #10133 didn't completely fix it

This is the message provided in Dev Tools; it's a different message so perhaps a different issue

PROTOCOL_TIMEOUT

Channel: DevTools
Initial URL: https://www.tiffany.com/
Chrome Version: 85.0.4157.0
Stack Trace: LHError: PROTOCOL_TIMEOUT
    at eval (devtools://devtools/remote/serve_file/@a4f2f880e7da1a083b75ee48b665dc2e2aa22840/lighthouse_worker/lighthouse_worker_module.js:1453:229)

[patrickhulce]

@philkrie that is #6512, very unrelated but even more annoying because we can't really do anything about it :)

[connorjclark]

We should modify the stack collector code we inject to use a window.___LH_FETCH native global.