Lighthouse: web.dev consistently errors on Network.getResponseBody for main content

[brendankenny]

• run node lighthouse-cli/index.js https://web.dev --only-categories best-practices --view
• get Properly defines charset – Error!

(also happens in DevTools and PSI)

Example report

The error is

Required MainDocumentContent gatherer encountered an error: Protocol error (Network.getResponseBody): No resource with given identifier found

and occurs here in driver when called from main-document-content.

It seems unusual that the main document content wouldn't be available for the backend. @robdodson does mention some unusual things done with the service worker and "an app shell that pulls in an index.json of the page content when you go to other pages. We essentially replace main content area with this new content" that could be causing trouble (maybe the request really didn't have any content?).

[rpong]

We are experiencing the same issue with the lighthouse npm package. leaving a comment to follow this thread.

[robdodson]

btw I noticed that our service worker was being too aggressive (https://github.com/GoogleChrome/web.dev/issues/3089). @brendankenny I tried to use canary to audit our staging server (https://web-dev-staging.appspot.com/) since it doesn't install the sw, but got an error

[connorjclark]

@robdodson https://github.com/GoogleChrome/lighthouse/issues/10883

should get fixed for next chromium canary

[Vadorequest]

Running into this too for the first time today. I got scored 0 for "Best Practices" when analyzing this website: https://nrn-v2-mst-aptd-at-lcz-sty-c1-ah8bmgx1e.vercel.app/en

LH Report: https://lighthouse-dot-webdotdevsite.appspot.com//lh/html?url=https%3A%2F%2Fnrn-v2-mst-aptd-at-lcz-sty-c1-ah8bmgx1e.vercel.app%2Fen#best-practices

[thepassle]

We're also running into this problem at modern-web.dev:

It seems like disabling the service worker 'fixes' the problem. We dont do anything fancy with our service worker — we either load an index.html either from network or from cache, we dont do any transformations on the html or anything like that. I really fail to see the connection between the service worker and defining a charset in the html ☹️

[patrickhulce]

I really fail to see the connection between the service worker and defining a charset in the html ☹️

Certain service workers mean that Lighthouse cannot fetch the HTML document to check for a charset. Error! means a Lighthouse bug, not something the page needs to fix (see https://github.com/GoogleChrome/lighthouse/issues/9968)

[thepassle]

We debugged a little bit more, we use a code snippet to reload the page when a new service worker has taken over:

if("serviceWorker"in navigator){let e;navigator.serviceWorker.addEventListener("controllerchange",()=>{e||(e=!0,window.location.reload())})}

❌ With that piece of code:

✅ Without that piece of code:

It seems that the reload is causing the best practices to error, hope that helps 🙂

EDIT:

In case anybody wants to workaround this, this seems to fix it. Also skips an unnecessary reload the first time a user visits your app (even though its so fast you might not have noticed)

        async function handleUpdate() {
          if ("serviceWorker"in navigator) {
            let refreshing;

            const oldSw = (await navigator.serviceWorker.getRegistration())?.active?.state;

            navigator.serviceWorker.addEventListener('controllerchange', async () => {
              if (refreshing) return;

              const newSw = (await navigator.serviceWorker.getRegistration())?.active?.state;

              if(oldSw === 'activated' && newSw === 'activating') {
                refreshing = true;
                window.location.reload();
              }
            });
          }
        }

        handleUpdate();

[patrickhulce]

Ah thanks @thepassle that helps a lot! I think this is happening because Chrome might only keep the network resources from the current page load in the memory cache which is what Lighthouse relies on to get the response body of the root document. We can do a bit more digging but this is a great head start 👍

related #8984 #11027

[robdodson]

We use a similar snippet on web.dev