Workbox: fetchDidFail with StaleWhileRevalidate is not called

[Lucide]

Library Affected:
workbox v5.1.3 npm

Browser & Platform:
Google Chrome v85

Issue or Feature Request Description:
I don't seem to get how to run some code when a fetch in a StaleWhileRevalidate strategy fails. I'd like to use the callback to detect offline usage, but fetchDidFail is not called when I tick "offline" and reload the pahe. (fetchDidSucceed works tho). Is this expected? I didn't find any mention of that in the documentation.

Here's an example:

const messageIfFail: WorkboxPlugin = {
    fetchDidFail: async function () {
        // No return expected.
        // NOTE: `originalRequest` is the browser's request, `request` is the
        // request after being passed through plugins with
        // `requestWillFetch` callbacks, and `error` is the exception that caused
        // the underlying `fetch()` to fail.
        while (true) {
            console.log("called");
        }
        // send message to client
    }
}

registerRoute(
    ({url}) => {
        return url.origin == "https://fonts.googleapis.com"
            || url.origin == "https://cdn.jsdelivr.net"
    },
    new StaleWhileRevalidate({
        cacheName: 'cdn',
        plugins: [
            messageIfFail
        ]
    })
);

[TheForsakenSpirit]

@Lucide try to use in this way.
But I am not sure that fetchDidFail must run on StaleWhileRevalidate. Because original request get cached response and was closed. Copy of original request return response to cache.
fetchDidFail run`s only on fail of original request.

[Lucide]

I see, together with fetchDidSucceed, they get only get called when the cache is being populated. I think that even if the code underneath answers the original requests, the callback mentions "fetch", not response. The fetch callbacks should behave like they do in NetworkFirst, because we have the guarantee a network fetch will always be made.\
Especially if they do get called the first run. StaleWhileRevalidate should theoretically behave the same with or without network, shouldn't it? Regardless of implementation optimizations.

I'll probably need to use the snipped you linked anyway, because I'd like to send a targeted message, and Mozilla does it like this:

addEventListener('fetch', event => {
  event.waitUntil(async function() {
    // Exit early if we don't have access to the client.
    // Eg, if it's cross-origin.
    if (!event.clientId) return;

    const client = await clients.get(event.clientId);
    if (!client) return;

    // Send a message to the client.
    client.postMessage({
      msg: "Hey I just got a fetch from you!",
//...

Workbox doesn't expose the event to plugins.

But how would that solve the fetch issue? It's still being done inside the strategy. I can't access it, can I?

[TheForsakenSpirit]

@Lucide I think you must use Broadcast-update strategy. And handle update response inside your app.

But I am not sure, that this will work in safari.

[Lucide]

I've seen that, but I think a targeted message will be neater. But that's off-topic, the issue is to when to send that message.
I'd like for StaleWhileRevalidate to support fetch callbacks.

Edit:\
Perhaps I've misinterpreted what the fetch callback are for. fetchDidFail doesn't work even with NetworkFirst, with "offline" ticked.

[TheForsakenSpirit]

@Lucide it fires only in case, where fetch failed _and_ cache not found. fetchDidFail in most cases used for handle cases where cache don`t have response and network request fails for creating some 'default response';
Or you can build your own 'fetch' handler with custom handlers of failed cache 'request' and real request.

[Lucide]

Alright. I find that to be a bit misleading tho. Because it mentions "fetch" specifically, "request" might be better? Maybe some more explanatory lines in the docs. Thank you!

[Lucide]

I started trying to tweak a StaleWhileRevalidate strategy but got a bit overwhelmed.
Maybe instead of fetchDidFail I could use cacheDidUpdate. If there's no cache writing optimization, a cache write means a network response arrived. (I suppose requestWillFetch is useless as well here).
So, I tried this:

registerRoute(
    ({url}) => {
        return url.origin == "https://fonts.googleapis.com"
            || url.origin == "https://cdn.jsdelivr.net"
    },
    ({event}) => {
        const {request} = event as FetchEvent;
        return new StaleWhileRevalidate({
            cacheName: "cdn",
            plugins: [
                {
                    requestWillFetch: async function () {
                        // i have access to the event to send a targeted message
                       console.log(event as FetchEvent);
                    }
                }
            ]
        }).handle({event, request});
    });

I get:

The FetchEvent for "https://cdn.jsdelivr.net/npm/normalize.css@8/normalize.min.css" resulted in a network error response: the promise was rejected.
Promise.then (async)
(anonymous) @ sw.js:655
index.html:6 GET https://cdn.jsdelivr.net/npm/normalize.css@8/normalize.min.css net::ERR_FAILED
sw.js:265 Uncaught (in promise) no-response: The strategy could not generate a response for 'https://cdn.jsdelivr.net/npm/normalize.css@8/normalize.min.css'. The underlying error is TypeError: Cannot read property 'clone' of undefined.
    at StaleWhileRevalidate.handle (http://localhost:63342/font-atlas-generator/sw.js:3911:23)
WorkboxError @ sw.js:265
handle @ sw.js:3911

sw.js:377 workbox Router is responding to: https://fonts.googleapis.com/css2?family=Open+Sans&display=block
sw.js:377 workbox Using StaleWhileRevalidate to respond to 'https://fonts.googleapis.com/css2?family=Open+Sans&display=block'
The FetchEvent for "https://fonts.googleapis.com/css2?family=Open+Sans&display=block" resulted in a network error response: the promise was rejected.
Promise.then (async)
(anonymous) @ sw.js:655
sw.js:265 Uncaught (in promise) no-response: The strategy could not generate a response for 'https://fonts.googleapis.com/css2?family=Open+Sans&display=block'. The underlying error is TypeError: Cannot read property 'clone' of undefined.
    at StaleWhileRevalidate.handle (http://localhost:63342/font-atlas-generator/sw.js:3911:23)
WorkboxError @ sw.js:265
handle @ sw.js:3911
main.css:1 GET https://fonts.googleapis.com/css2?family=Open+Sans&display=block net::ERR_FAILED

I don't detect any network/CORS error, It used to work fine before the modification.
Am I doing something wrong here?

[TheForsakenSpirit]

@Lucide requestWillFetch must return Request object.
Check this.

[jeffposnick]

fetchDidFail will execute when the underlying fetch() fails when using StaleWhileRevalidate, even if the strategy ends up using a response from the Cache Storage API.

I see from your sample route that you're attempting to use this with subresources loaded from a CDN, and in most cases, responses to those sorts of URLs will use long-lived Cache-Control headers that don't require revalidation. So what my guess is as to what's happening is that even when you're offline, the underlying fetch() request succeeds, rather than fails, since a cache hit in the browser's "normal" HTTP cache is sufficient, and fetch() never ends up going against the network anyway.

You can see this in action at https://glitch.com/edit/#!/upbeat-rebel-octopus?path=sw.js, where the example is for a HTTP response that has a maximum age of 20 seconds. If you go offline and try again within 20 seconds of populating the "normal" HTTP cache with a response, then the fetch() will be fulfilled from the HTTP cache. If you wait longer than 20 seconds, then the response in the HTTP cache will be considered too old to use without revalidation, and the fetch() will fail when it makes the network request to revalidate.

I'm going to close this for now, but if I'm misinterpreting anything, let me know and we can revisit.

[Lucide]

@TheForsakenSpirit So sorry! Changed the code without re-checking.

@jeffposnick Very well, I've seen some responses have 80000+ seconds on max-age, but that's ok, I don't actually need to detect the connection status, just signal the app went to "offline mode". Until an http cached response expires, I can consider it indistinguishable from online work.
Hoping that I'm not forgetting something stupid again, I still have two errors I can't manage to solve.\
Only this route is affected. (I also have the recommended route for Google Fonts font files, but it has never given errors as now. I clear the application data regularly during testing)

The FetchEvent for "https://cdn.jsdelivr.net/npm/normalize.css@8/normalize.min.css" resulted in a network error response: an "opaque" response was used for a request whose type is not no-cors
normalize.min.css:1 Failed to load resource: net::ERR_FAILED
sw.js:377 workbox Precaching is responding to: /font-atlas-generator/build/css/main.css
sw.js:377 workbox Using StaleWhileRevalidate to respond to 'https://fonts.googleapis.com/css2?family=Open+Sans&display=block'
The FetchEvent for "https://fonts.googleapis.com/css2?family=Open+Sans&display=block" resulted in a network error response: an "opaque" response was used for a request whose type is not no-cors
css2:1 Failed to load resource: net::ERR_FAILED

This happens randomly, when opening the browser. If it occurs, it doesn't go away until the app data is cleared. I have no Idea why. Normally cross-origin requests are correctly stored as "cors". (Only if made explicit through html).
This one breaks the sw functionality.

---

workbox Network request for 'https://cdn.jsdelivr.net/npm/normalize.css@8/normalize.min.css' threw an error. TypeError: Failed to fetch
print @ sw.js:377
(anonymous) @ sw.js:390
wrappedFetch @ sw.js:2641
async function (async)
wrappedFetch @ sw.js:2550
_getFromNetwork @ sw.js:3945
handle @ sw.js:3885
(anonymous) @ sw.js:4019
handleRequest @ sw.js:786
(anonymous) @ sw.js:653
sw.js:377 workbox Network request for 'https://fonts.googleapis.com/css2?family=Open+Sans&display=block' threw an error. TypeError: Failed to fetch
print @ sw.js:377
(anonymous) @ sw.js:390
wrappedFetch @ sw.js:2641
async function (async)
wrappedFetch @ sw.js:2550
_getFromNetwork @ sw.js:3945
handle @ sw.js:3885
(anonymous) @ sw.js:4019
handleRequest @ sw.js:786
(anonymous) @ sw.js:653
sw.js:1 Uncaught (in promise) TypeError: Failed to fetch
sw.js:377 workbox Using StaleWhileRevalidate to respond to 'https://fonts.googleapis.com/css2?family=Open+Sans&display=block'
sw.js:1 Uncaught (in promise) TypeError: Failed to fetch

This happens instead when the app is offline and the http cache is disabled ("Disable Cache" ticked). This is the "development" version, the same errors appears in "production" mode too. The same errors show up in a completely generated sw.
Since service workers exist to provide offline support, I'm surely missing something here.

This is the code I'm using:

registerRoute(
    ({url}) => {
        return url.origin == "https://fonts.googleapis.com"
            || url.origin == "https://cdn.jsdelivr.net"
    },
    ({event}) => {
        const {request} = event as FetchEvent;
        return new StaleWhileRevalidate({
            cacheName: "cdn",
            plugins: [
                {
                    fetchDidFail: async function () {
                        //...
                    }
                }
            ]
        }).handle({event, request});
    });

setCatchHandler(async () => { //doesn't get called in my situation
    console.log("TEST");
    return Response.error();
});

[TheForsakenSpirit]

@Lucide recourse called by fetch or by html tag?
If second try add crossorigin="anonymous"

[Lucide]

Both inside <head> of index.html:

<link crossorigin="anonymous" rel="stylesheet" href="https://cdn.jsdelivr.net/npm/normalize.css@8/normalize.min.css">
<link crossorigin="anonymous" rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Open+Sans&display=block">

[jeffposnick]

When attempting to debug this, I would suggest starting fresh from an Incognito window, and additionally, not checking Disable Cache. It sounds like you've made a number of changes to your SW during development, and it would be good to rule out inconsistent cache state (using the Incognito window) while also replicating what users will actually see in production (don't check Disable Cache).

You might see a log message about a failed fetch() made by StaleWhileRevalidate in development, but if the strategy uses a cached response from the Cache Storage API to satisfy the request, the "catch" handler won't run. The "catch" handler only runs when there's no response (from fetch() or Cache Storage API) that can be used by a strategy.

FWIW, your code could be simplified to:

registerRoute(
    ({url}) => {
        return url.origin === "https://fonts.googleapis.com"
            || url.origin === "https://cdn.jsdelivr.net"
    },
    new StaleWhileRevalidate({
      cacheName: 'cdn',
      plugins: [...],
    })
});

setCatchHandler(async () => { //doesn't get called in my situation
    console.log("TEST");
    return Response.error();
});

[Lucide]

Thank you for the clarifications

I keep getting the first error, even when I don't touch the sources. I just need to keep reopening the page and sooner or later it breaks, and I need to wipe the cache storage to get it working again.

I'm not using "disable Cache" to affect the cache storage, it just disables the http cache.\
To clear the cache storage and DBs, I use the dedicated panel in Chrome "Clear Storage". It clears pretty much everything and it's designed to be used when working with service workers.\
I need to disable the http cache to simulate a true offline experience, since the max-age in the cdn headers are extremely high.

I'm aware that the worker currently doesn't clean up the cache on activation (update), but the error shows up randomly just by opening the page, not only during an sw update.

The second problem occurred with generated service workers too. For the same two origins. It's not a log entry, it's an uncaught js exception. I didn't understand how to handle it.

The sw is tiny and the entire source can be posted here:

sw.ts

import {registerRoute, setCatchHandler} from "workbox-routing";
import {CacheableResponsePlugin} from "workbox-cacheable-response";
import {ExpirationPlugin} from "workbox-expiration";
import {precacheAndRoute} from 'workbox-precaching';
import {setCacheNameDetails} from "workbox-core"
import {StaleWhileRevalidate, CacheFirst} from "workbox-strategies";

declare var self: ServiceWorkerGlobalScope;
export {};

setCacheNameDetails({
    prefix: "font-atlas-generator",
})

precacheAndRoute(self.__WB_MANIFEST);

registerRoute(
    ({url}) => {
        return url.origin == "https://fonts.googleapis.com"
            || url.origin == "https://cdn.jsdelivr.net"
    },
    new StaleWhileRevalidate({
        cacheName: "cdn",
        plugins: [
            {
                fetchDidFail: async function ({event}) {
                    if (!event) return;
                    const fetchEvent = event as FetchEvent;
                    if (!fetchEvent.clientId) return;
                    const client = await self.clients.get(fetchEvent.clientId);
                    if (!client) return;
                    client.postMessage({
                        msg: "offline"
                    });
                }
            }
        ]
    })
);

registerRoute(
    ({url}) => url.origin == "https://fonts.gstatic.com",
    new CacheFirst({
        cacheName: 'google-fonts-webfonts',
        plugins: [
            new CacheableResponsePlugin({
                statuses: [0, 200],
            }),
            new ExpirationPlugin({
                maxAgeSeconds: 60 * 60 * 24 * 365,
                maxEntries: 30,
            }),
        ],
    })
);

setCatchHandler(async () => {
    return Response.error();
});

I register the sw with this snippet (the main script is imported at the end of the html file):

if ("serviceWorker" in navigator) {
    const wb = new Workbox("sw.js");

    wb.addEventListener("activated", () => {
        console.log("new service worker activated, reloading to cache everything");
        location.reload();
    });
    wb.register();
    navigator.serviceWorker.addEventListener("message", (event) => {
        if (event.data.msg == "offline") {
            qr.header.classList.add("offline");
        }
    });
}

[Lucide]

debugging the cors error

I opened the page on a different machine where the service worker had never run, and I've seen the error can happen between reloads too (not only start-on-open). The first three times it worked right, the fourth failed.\
No source files were touched and No service worker-powered versions of the app had ever ran on this pc. I'm lost!

Edit:

I noticed it's somehow linked to the chrome developer tools being open and closed.

• observation 1: when reloading the page, only three requests are logged: for the stylesheet, the main js file, and the favicon.\
  No html file. Is this expected? I thought the http cache affected workbox's fetches, not the entire event capturing.

• observation 2:\
  after the developer tools have been opened (Ctrl+shift+i):

  • the first page reload will trigger all the requests again, and the cors error will always occur.
  • the second page reload will trigger all the requests again, no error messages will be print.
  • the subsequent page reloads will show the same three requests mentioned above

  after the developer tools have been closed:

  • the first page reload will trigger all the requests again, no error messages will be print.
  • the subsequent page reloads will show the same three requests mentioned above

Looks like opening the dev tools alters some caching mechanism. It temporarily breaks cors requests too. Not sure why the error persisted between reloads in my previous tests, might have been because I was toggling the dev panel.

Still no idea about the offline error tho.

The deployed app behaves erratically as well.

[Lucide]

I've ran the app in Firefox, to see if I could gather something more. It says

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://cdn.jsdelivr.net/npm/normalize.css@8/normalize.min.css. (Reason: CORS request did not succeed).

GET https://cdn.jsdelivr.net/npm/normalize.css@8/normalize.min.css

Failed to load ‘https://cdn.jsdelivr.net/npm/normalize.css@8/normalize.min.css’. A ServiceWorker passed an opaque Response to FetchEvent.respondWith() while handling a ‘cors’ FetchEvent. Opaque Response objects are only valid when the RequestMode is ‘no-cors’. sw.js:1853:26

GET https://fonts.googleapis.com/css2?family=Open+Sans&display=block

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://fonts.googleapis.com/css2?family=Open+Sans&display=block. (Reason: CORS request did not succeed).

Failed to load ‘https://fonts.googleapis.com/css2?family=Open+Sans&display=block’. A ServiceWorker passed an opaque Response to FetchEvent.respondWith() while handling a ‘cors’ FetchEvent. Opaque Response objects are only valid when the RequestMode is ‘no-cors’.

A Learn More link sends here, where it talks about general network/http errors, but I don't think that's the case. Neither are adblockers, because chrome with external debugger starts blank, and my "Firefox Developer Edition" installation is blank.

Given that, I got the cors error quite often on Firefox before writing this comment, now it's ten minutes I'm fiddling with the page and it hasn't shown up since.

Firefox doesn't give the offline "uncaught exception" error.

Should I open a new issue dedicated to these two, rename this one, or else?