Next.js: Offline First Support

[espoal]

Offline support is very useful, and is crucial for creating a modern PWA. Along with HTML manifests it helps to bridge the gap between a website and a native app.

This feature has two different flavours:

• Offline support: This is the ability to navigate the site even when offline, if the site has been loaded while online. This should be the easiest feature to implement.

• Offline first support: This is the ability to open the site even when offline, even if the site has not been loaded in the browser.

Both should be doable thanks to webpack-offline plugin. Anyhow since I'm learning both React and Next.js at once, I haven't been able to make it work yet.

Steps to make it work:

• Install webpack-offline

npm install offline-plugin --save-dev

• Create a custom next.config.js in your root folder

const OfflinePlugin = require('offline-plugin');

module.exports = {
  webpack: (config, { dev }) => {
        config.plugins = [
            new OfflinePlugin()
        ];
    return config
  }
};

• Initialize webpack-offline

This is the step I'm having problem with. I think you should do it in a component, inside componentDidMount, inside the top level.

This issue is both a reminder for me to keep working on this, and a request for help from someone more knowledgeable.

[sedubois]

I'd absolutely love to see this working too, along with explanations/documentation/example about it.

Note that next/prefetch doesn't yet really allow offline behaviour, because it doesn't prefetch data: https://github.com/zeit/next.js/issues/740

Not directly related to Next.js but I also wonder how much data can actually be kept offline (e.g if a webapp has videos etc -- is there some hard limit in the browser? And what about mobile?), how the user could specifically ask for one thing to be pre-downloaded for later, etc. I also previously mentioned things here: https://github.com/zeit/next.js/issues/24#issuecomment-258804529 (before next/prefetch was a thing).

[msand]

There are varying data limits in different platforms, browsers and versions. You can test the limits in the "Browser storage abuser": https://demo.agektmr.com/storage/

The standardized method which is intended for offline storage and caching is IndexedDB. It is now supported even in iOS Safari (v10), but there it has performance issues. Otherwise it has wide support now: http://caniuse.com/#feat=indexeddb

E.g. PouchDB still uses WebSQL instead of IndexedDB on Safari. https://github.com/pouchdb/pouchdb/issues/5572
Same with localForage: https://github.com/localForage/localForage/issues/604

PouchDB has a nice summary of data limits: https://pouchdb.com/faq.html#data_limits (slightly out of date)
And this even older article also mention how to handle some out of storage errors and other aspects regarding mobile browsers https://www.html5rocks.com/en/tutorials/offline/quota-research/

You can also query for your current quota and request more persistent storage in some browsers: https://jakearchibald.com/2014/offline-cookbook/#cache-persistence

Another way would be to use long cache expiration values and immutable cache control together with service workers. This would easily allow user specified caching, just by making a http request for each selected resource. This would also work quite nicely in old browsers. But to be able to maintain and manually delete various caches to avoid limits would only be possible in browsers supporting service workers.
https://developer.mozilla.org/en-US/docs/Web/API/Cache
https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API/Using_Service_Workers

Just remember, when you run out of space, the browser can evict an entire origin at a time until it is within limits:
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Browser_storage_limits_and_eviction_criteria

[msand]

Another useful article by Addy Osmani: https://medium.com/dev-channel/offline-storage-for-progressive-web-apps-70d52695513c#.9vja3t8gp

Apparently there is also a Storage api being worked on: https://storage.spec.whatwg.org/

This allows actually persistent storage:
"Traditionally, as the user runs out of storage space on their device, the data stored with these APIs gets lost without the user being able to intervene. However, persistent boxes cannot be cleared without consent by the user. This thus brings data guarantees users have enjoyed on native platforms to the web."

[impronunciable]

IMO making a site/app work offline involves a lot of decisions a framework shouldn't take by itself. I'll work on an example on a site working offline with a service worker but there are different techniques for different kind of apps.

[espoal]

thanks @impronunciable . Do you plan to use webpack-offline or something else?

what kind of decisions are you talking about? I think we can divide the problem in two main issues:

1) Static assets caching: Like js, HTML, images, ... this is almost already implemented, at least in the offline flavor and with the exclusion of /static, and given that we are using react it should have one preferred implementation, via webpack-offline and service workers.

2) Data caching: state, queries, volatile data, .... pose more concerns, as it at least requires to presume how the users will load data. Maybe we could show how to preserve state with redux, and then people will put data in redux as they prefer. Or maybe we could use GraphQL/Apollo which should cover such a case by caching queries and mutations.

[impronunciable]

@servermeta it really depends on your case. I'm finishing implementing an aggressive caching strategy without using plugins, just a custom server and a strategy from https://serviceworke.rs/

[ooade]

I have it working here. Battled a lot with Offline-Plugin, had some issues with the .next directory then i switched to sw-precache-webpack-plugin, ignored .next directory and served all assets to the sw

[espoal]

thank you @ooade , well done, you saved me a lot of time.

Anyhow I see that state does not persist between refreshes, reloads. I'll try to think how to add this feature.

[espoal]

thank @ooade . By localhost you mean a local db, like mongodb, or localstorage?

I think we should divide the problem: offline data retrieving should be left to the dev, while we could preserve redux state. Check this:

https://github.com/rt2zz/redux-persist

with this we can store state in localstorage, so it can persist between refreshes, reloads, tabs and sessions.

[addyosmani]

Hey folks. My team at Google work on a few Service Worker libraries (with Webpack plugins) like https://github.com/GoogleChrome/sw-precache and https://github.com/GoogleChrome/sw-toolbox used on React/Webpack heavy sites like Lyft, Housing.com, Flipkart etc.

If Next decides to explore offline support we would be happy to share some pointers. I think there's great opportunity for prescribing patterns like PRPL out of the box, given code-splitting is already being done. Production-only Service Worker caching would be a neat addition.

In addition to giving you instant repeat loads for those pages, it would also opt you early on into V8's code caching so your parse/compile times for repeat visits would be lower than today.

Feel free to holler if any of this is interesting @rauchg :)

[rauchg]

@addyosmani offline support is one of our top priorities post 2.0 stable

[YarivGilad]

@rauchg any estimation regarding 2.0 stable release date?
We're about to kick off a full on production and we would love to use Next.js
I will appreciate any type of estimation, days / weeks / months...
Thank you so much!

[sebastienbarre]

@Ajar-Ajar 2.0.0 was released today.

[egeozcan]

@rauchg is the offline-first support going to be tracked here or are you going to create another issue for it?

[sedubois]

Please also see the newly open-sourced redux-offline by @jevakallio. Would be awesome to have a next + redux-offline example.

[saulflores95]

So do we have any info on how to achieve this, I been trying to do it the next.config.js and installing the offline-plugin, but I could not get it to work. Next is an awesome project but it would be super complete(and cooler) if it had this feature(Offline-first )out of the box!

[AugustinLF]

@saulflores95 Using @ooade's NextSimpleStarter's way did work for me :)

[sedubois]

@AugustinLF NextSimpleStarter doesn't offer offline capabilities. https://github.com/ooade/NextSimpleStarter/issues/23#issuecomment-294310240

[timmywil]

@sedubois For anyone coming in and reading this, that's a bit of an overstatement. To be fair, it does have some offline capabilities with the use of sw-precache and sw-toolbox. My app works offline solely with those two tools, but the initial state of my app doesn't change. If I was trying to be specific, I might say that it does not offer offline solutions for constructing state beyond what was initially sent down the wire.

[freecates]

@timmywil,do you have a GitHub repo of your nextjs offline capable app? Thanks.

[ssured]

I just created a (beta) version of next supporting offline using appcache, which is needed for Safari. Please have a look at http://github.com/ssured/nownextmicro

[Sly777]

Hey all, I added offline support to my boilerplate.
https://github.com/Sly777/ran

It's a little bit buggy. So that's why It's named as "experimental" 😄 Anyway, I hope it helps.

[jabacchetta]

@rauchg Is this feature still a priority?

[prateekrastogi]

@rauchg With next.js 4.0 beta released, are there any chances of offline first support being in the roadmap for that version?

[HelloEdit]

I would ask for news of the feature ^^

[idhard]

Next.js 5.0 has been released (congratulations!) but there is no mention of offline support , is there a new roadmap you would like to share ? thanks for the amazing job done so far

[arunoda]

@idhard actually, we may not going to support offline support by default.
(But things might change)

But we are in the process of making sure Next.js doesn't do any magic. So, you'll be able to use direct webpack plugins and loaders as is.
Next 5 is the step one.

[hanford]

@idhard I think it'd be counter intuitive for blanket offline support, some apps definitely won't want that feature to be enabled.

On my personal website I'm using this https://github.com/zeit/next.js/tree/canary/examples/with-sw-precache and we're also going to be using ^ in production at Eaze once iOS 11.3 is released.

[idhard]

@hanford yeah similar discussion has been done on CRA and end up removing support of webworkers by default (https://github.com/facebook/create-react-app/issues/2554#event-1431558318), however i still think webworkers and PWA are going to be the defacto solution for offline support , so would be good to know if Next's team has some plan of adding an official support , like pre fetched pages.

[hanford]

@idhard yeah, kind of an interesting dilemma for the core team. I've been really happy with the sw-precache plugin i mentioned above.

my personal website is using the sw-precache webpack plugin, along with serving a manifest.json from the static directory. If you're curious, the code is over here .. the commits are a little sloppy, but i added offline support and the manifest json within the last week.

[sedubois]

@hanford what happens in iOS 11.3, will it have service workers? Would you have any reference with more info?

[arunoda]

@hanford @idhard we tried service workers way before CRA and had a lots of issues.
That's why we decided to build a prefetching solution purely with traditional web caching tech.
It works amazing well. A new set of improvements are coming soon.

Yes of course offline is a place we need SW.
But it's very unstable and hard to use API. Things could go wrong and break your site.

So, we may not want to do it ourselves.
But we want to allow users to use things like sw-precache via Next.js plugin (or simply adding a set of webpack loaders and plugins)

[ssured]

@sedubiois see https://developer.apple.com/library/content/releasenotes/General/WhatsNewInSafari/Articles/Safari_11_1.html for Apples plans on iOS Safari. Service Workers are announced

[hanford]

Yep @ssured @sedubois service workers are landing in Mobile Safari in iOS 11.3 which is pretty exciting! I'm on the iOS 11.3 Beta 2 and there are numerous bugs, (service workers don't get recognized properly when the website is added to the homescreen, but I'm confident apple will fix these before the public release)

I think what @arunoda is suggesting is that Next.js current caching strategy (cache-control headers, etags, etc) is much more stable than service workers. Service workers do enable some really neat new functionalities though, especially getting finer grain control over network requests (returning cached content early) .. But what Next.js works everywhere and considerably less overhead .. (unregistering service workers is a total pain)

[hanford]

@ssured @sedubois I made a little plugin that works the same way as the plugins that Zeit launched the other day .. it should alleviate offline next apps and be pretty straight forward plugging in to your existing apps

Let me know if you have any feedback! https://github.com/hanford/next-offline

[prateekrastogi]

@hanford thanks for making our life a bit easier
@arunoda While the plugin support in next.js 5 is awesome, wouldn't it be much more beneficial for community if all the plugins are hosted in main next.js repo plugins folders, just as all the examples are, instead of an auxiliary repo ? Most developers visit main repo, and thus, potential plugin developers would have much more incentive to create pull request, thereby saving community time wastage due to repetition and inevitable plugin ecosystem fragmentation arising as a result of separate repos.

[jabacchetta]

For anyone else still deciding on what to do moving forward, I also made use of the sw-precache webpack plugin with relative ease (example, again).

[designspin]

I was using my own solution but have switched to the solution provided by hanford. I had to make a few modifications in next.config.js to stop the plugin auto registering the service worker, but it seems to be working well.

I now just need to figure out how I can get this working with my custom server. For example I have a route setup as article/:slug. When I visit one of these urls the service worker is trying to dispatch a document for that url. Anybody know how I can stop that and make it serve article instead? This is related to settings in Workbox I guess.

[caribou-code]

Should we still be expecting any integration of service workers or offline support in future NextJS releases? Looks like some people were previously saying it was a priority feature, but this issue has been open for over a year and a half...

[hanford]

@caribou-code I can't speak for the Zeit team on their plans for Next.js, but I wrote this awhile ago: https://github.com/hanford/next-offline that allows you to automatically generate a service worker that will work offline.

I've used it in several applications and it's worked pretty well. Under the hood it's utilizing googles workbox, which is a very exciting project: https://developers.google.com/web/tools/workbox/

Some examples where I use next-offline:

• https://github.com/hanford/trends
• https://github.com/hanford/personal-website

[caribou-code]

@hanford I was just using next-offline before posting on here and it's pretty good! In fact it's the only solution I've managed to implement so far that actually works. Good work!

However, I did really want to get a solution working with sw-precache-webpack-plugin because there's an example of this in the NextJS repo, although I can't work out how to configure it to cache and serve all my Next files through the service worker. That plugin seems to be quite popular as well.

[ooade]

I created NextSimpleStarter a year ago to help resolve this issue. But, it came to my notice that sw-precache alone won't be able to meet most offline requirements so we recently moved to using workbox which solves most of them.

Although, I am yet to update it to current standards(icon sizes and so on), which I will fix in few days. Feel Free to give it a trial. Drop an issue if you don't find it satisfying.

[dancancro]

@hanford This looks great. It runs for me in development mode but there's no service worker in that mode. I can't tell from your readme how to make it work in production mode and with a service worker and without a node server. I also deploy my app to Netlify and I've been using next export. My app is totally static. I have no problem not using next export if that's a problem. I'll do whatever's most performant and doesn't cost anything. It's a hobby app, so I'm flexible.

@ooade This looks great too, but I got an error when starting it. Changing "serverless" to "server" per your instruction fixed that error but then I got the following error

A bad HTTP response code (404) was received when fetching the script.

[hanford]

@dancancro you should definitely be able to use next-offline while also using next-export

Mind opening an issue in next-offline with some steps to reproduce so I can take a deeper look?

[dancancro]

@hanford I can do that if you want but I did nothing special and I'm not suggesting anything's broken about the starter. The steps to reproduce the problem are simply your instructions. The only problem is that I don't know how to run it in production mode. Judging from this condition a service worker is not supposed to be registered in dev mode, so what happened for me is expected behavior. I just need some instructions - how to run it in production mode, and if next export is possible, then how to run static, server-rendered code in production mode with next export.

[hanford]

@dancancro I understand, but that discussion shouldn't be happening here, this is certainly not an issue with Next.js itself.

Please open an issue over here and I'd be happy to take a look / answer questions you might have.

The community doesn't benefit if we have a discussion in an unrelated issue/repo

[shadowwalker]

I recently created an easy to use zero config PWA plugin for Next.js: next-pwa

Checkout the example here