Gatsby: FEATURE: Encourage caching by giving each build of bundle.js a unique name

[KyleAMathews]

Currently someone can visit a new page and if they've cached the old bundle.js in their browser, after loading that, the new page will go blank as the old bundle.js doesn't know yet about it.

[gesposito]

https://github.com/webpack/docs/wiki/long-term-caching

[jeberly]

Is there a timeline on this? Or some hints on what would have to be done to make this work? Thanks!

[KyleAMathews]

No one is working or has signaled plans to work on this yet afaik. You're
welcome of course to research a solution and submit a PR :-)
On Wed, Nov 11, 2015 at 1:00 PM John Eberly [email protected]
wrote:

Is there a timeline on this? Or some hints on what would have to be done
to make this work? Thanks!

—
Reply to this email directly or view it on GitHub
https://github.com/gatsbyjs/gatsby/issues/27#issuecomment-155908263.

[jeberly]

@KyleAMathews thanks ;) I assume we would want this to be a configurable option? Or always on?

[KyleAMathews]

My gut reaction is default on but configurable.
On Wed, Nov 11, 2015 at 6:00 PM John Eberly [email protected]
wrote:

@KyleAMathews https://github.com/KyleAMathews thanks ;) I assume we
would want this to be a configurable option? Or always on?

—
Reply to this email directly or view it on GitHub
https://github.com/gatsbyjs/gatsby/issues/27#issuecomment-155973180.

[KyleAMathews]

One tricky thing about using Webpack's default hash is we can't determine the name until _after_ it's built meaning we wouldn't easily be able pass the bundle.js file path to the HTML template. Which means we should just generate our own unique bundle name(s). Perhaps the easiest way to do this is to use the timestamp when gatsby build starts.

[wyattanderson]

One tricky thing about using Webpack's default hash is we can't determine the name until after it's built meaning we wouldn't easily be able pass the bundle.js file path to the HTML template.

Is it crazy to suggest using some sort of template string as the path to the bundle file then doing a pass after the build to replace the template string with the generated filename?

[KyleAMathews]

@wyattanderson not crazy :) just mildly tricky. Would be a great PR if you want to work on it. Just added styles.css output on prod builds so we're ready for this.

[scottnonnenberg]

This was my solution, from my top-level html.js. First, outside the React component:

const now = new Date();
const buster = now.getTime();

Then in render():

<script src={link('/bundle.js?t=' + buster)} />;

So every build you get one new cache buster value based on the current time.

[qixotic]

https://github.com/webpack/webpack/issues/86 has some related discussion. It looks like if we change webpack-config.js production to output filename: '[chunkhash].bundle.js',, one could do something like the following in a gatsby-node.js:

const Path = require('path');
const FileSystem = require('fs');

function RevisionPlugin() {}
RevisionPlugin.prototype.apply = function(compiler) {
  compiler.plugin('done', (statsData) => {
    const stats = statsData.toJson();
    // FileSystem.writeFileSync(Path.join(__dirname, 'stats.json'), JSON.stringify(stats));

    if (!stats.errors.length) {
      const htmlFileName = 'index.html';
      const outfile = Path.join(__dirname, 'public', htmlFileName);
      const html = FileSystem.readFileSync(outfile, 'utf8');

      const htmlOutput = html.replace(
        /<script\s+src=(["'])(.*?)bundle\.js\1/i,
        `<script src=$1$2${stats.assetsByChunkName.main[0]}$1`
      );

      const tempOutfile = Path.join(__dirname, '.tmp', htmlFileName);
      FileSystem.writeFileSync(tempOutfile, htmlOutput);
      FileSystem.renameSync(tempOutfile, outfile);
    }
  });
};

exports.modifyWebpackConfig = function(config, env) {
  if (env === 'production') {
    config.plugin('done plugin', RevisionPlugin);
  }
  return config;
}

The regex could be a user option if it needs to be more complex.

[benstepp]

A done plugin stage which opens up every page, regexes a file for a bundle tag, replaces the tag with the contenthash, and rewrites the file back to the disk (with or without a tmp file) is going to have a lot of I/O. We should try and look for a solution which writes the html files correctly the first time.

---

Here's my idea for getting this in. It will be a breaking change (as bundle.js will no longer exist and most people right now are doing <script src={prefixLink('bundle.js')}) />.

Create the javascript and css prior to building the pages (#253 has a pretty good start on this). We would just change the order.

Pass the compiled assets information as an argument to the static site generation stage (maybe using DefinePlugin as __GATSBY_ASSETS__).

Change the userland request of the bundle.js to something like the following. These helper components will just rip the asset information out of the defineplugin during build and just a standard bundle tag in development.

import { GatsbyJs, GatsbyCss } from 'gatsby-helpers'
// defaults to bundle.js
<GatsbyJs />

// get a different entry point
<GatsbyJs entry='custom' />

Maybe rename the default bundle.js and styles.css to gatsby-[chunkhash].js and gatsby-[contenthash].css as subtle marketing while everything is being broken.

[KyleAMathews]

I agree avoiding rewriting every page would be much much better. And no need to fear breakage — that's why https://github.com/gatsbyjs/gatsby#warning is there 😝

Also to support code splitting, we'd need the ability to track and pass to different pages which additional JS chunks they should load.

[benstepp]

I didn't really think about the multiple chunks. For entry points with multiple chunks we would probably have to aim for a more react-helmet like API and return arrays of components, rather than a single component.

import { assets } from 'gatsby-helpers'
// defaults 
{assets.js}
{assets.css}

// if the user wants to grab a specific chunk
{assets.byChunk('chunkname')}
{assets.byChunk('chunkname', 'js')}
{assets.byChunk('chunkname', 'css')}

// if the user wants to get it all
{assets.raw}

---

In order to figure out which chunks to load based on the current path, we can use the fact that node-modules are singletons and register the current path into the gatsby-helpers node module in the static-site-generator-plugin render. Then when the call stack reaches the html.js and the user calls for the assets, it will know that the user is on a specific page.

[KyleAMathews]

Yeah, that sounds like a good plan.

[KyleAMathews]

Until we can get hashed bundle names in, I added a cache buster per @scottnonnenberg's suggestion https://github.com/gatsbyjs/gatsby-starter-default/commit/c1eee904c1737b151e93ddf65651c82b8fe92b1b

[KyleAMathews]

(to the starters)

[KyleAMathews]

Good resources:

• https://medium.com/@okonetchnikov/long-term-caching-of-static-assets-with-webpack-1ecb139adb95#.lsxm249f1
• https://github.com/mjackson/web-starter/commit/bc94edd2d4d6270e4905c33da2cf673b8eb3b6ce

[stale[bot]]

This issue has been automatically closed due to inactivity.