Lighthouse: Remove unused JavaScript is unclear

[OZZlE]

Provide the steps to reproduce

1. Run PageSpeed Insights on https://www.etonshirts.com/no/ => https://developers.google.com/speed/pagespeed/insights/?hl=sv&url=https%3A%2F%2Fwww.etonshirts.com%2Fno%2F
2. Check the "Remove Unused JavaScript" Opportunity

Page is almost entirely rendered by js clientside.

32fea5ec9d2b45fb50db382738d00be2.js contains the main React bundle starting point without it you have an almost blank page since this is the app..

MadeCampaign is the content part of the page lazy loaded with https://github.com/gregberge/loadable-components, Cart is the cart, capture-apps-5.0.0.js = tracking, hotjar = tracking, nosto = tracking, pinimg = tracking, sentry = js error tracking, fbevents = tracking, gtm = tracking, fb = tracking, analytics = tracking, adservices = tracking, webfont = font, igodigital = tracking.

What is the current behavior?

Claims of unused js.

What is the expected behavior?

In Reality pretty much all of it is used.

Environment Information

• Affected Channels: Pagespeed
• Lighthouse version: ?
• Chrome version: Version 83.0.4103.61 (Officiell version) (64 bitar)
• Node.js version: N/A
• Operating System: MacOs

Related issues

[patrickhulce]

The audit is not saying all of that file is unused, it's saying much of those files are unused. For example for the first file 131 KiB is "waste" out of 277 KiB = 47% unused

While you may not directly control the third party bundles it demonstrates how much of the feature set you're actually using (or how inefficient they are at bundling 😕). Try reaching out to see if they offer slimmer bundles with a more direct feature set or a way to self-host instead.

[OZZlE]

@patrickhulce aha I see for third party services we probably can't do much (other than stop using it) since my experience is that they tend to barely fix bugs even unless it crashes the entire website.. Unfortunately I am not allowed to decide wether we use these or not :<

we're using core-js to polyfill features mainly for IE11 and Safari, it doesn't lazy load only needed polyfills so I guess it's a waste for Chrome but not for IE.. I know there are services like polyfill.io that only loads necessary polyfills but from what I've heard several of the polyfills it uses aren't working properly and not as solid as core-js so I'm afraid to use it... Similar story with babel I guess, I guess one could build different bundles for newer/older browsers but that is also complex to set up and if not done 100% correctly it can have devastating effects..

I also found out that using an alias to a file which exports several other objects which imports files and then importing them somewhere makes it import all of them even if only one is used when using webpack and UglifyJsPlugin, I've removed that approach which should save a lot of unused js. E.g. doing this is bad:

webpack.config.js:

resolve: {
  alias: {
    accountPages: path.resolve(__dirname, './src/accountPages.js'),
  },
},

accountPages.js:

export { default as AccountSignin } from 'components/AccountPages/AccountSignin';
export { default as AccountSignup } from 'components/AccountPages/AccountSignup';
export { default as AccountOrder } from 'components/AccountPages/AccountOrder';

and then somewhere import { AccountSignin } from 'accountPages'; actually seems to import all of them and it's not removed by UglifyJsPlugin even.

[patrickhulce]

Similar story with babel I guess, I guess one could build different bundles for newer/older browsers but that is also complex to set up and if not done 100% correctly it can have devastating effects

We have guides on how to do this exact thing safely and with minimal impact to your existing process. In the next version of Lighthouse there will even be an audit specifically calling out the use of large polyfills :)

I also found out that using an alias to a file which exports several other objects which imports files and then importing them somewhere makes it import all of them even if only one is used when using webpack and UglifyJsPlugin, I've removed that approach which should save a lot of unused js

🎉 awesome to hear! Let us know if you think there's any way to automatically detect these types of problems in a way that would be useful to others :)

[OZZlE]

@patrickhulce great big thanks!

[OZZlE]

@patrickhulce I noticed however that the guide only deals with Transpiling = Babel

So you can use ES6 and it is converted to ES5. That's one thing.

core-js deals with polyfills => missing features in browsers. https://stackoverflow.com/questions/31205640/what-is-the-difference-between-polyfill-and-transpiler

I don't believe changing "@babel/preset-env" affects what core-js polyfills are loaded at all. Why do I believe this? Because currently we include IE11 and I changed to "latest major, not dead" and it didn't affect the js size at all.. If this affected which core-js polyfills were loaded it should have become much smaller!

I will implement it soon but I'm pretty sure it's still going to complain about unused code as all core-js polyfills will still be present. And the only alternative I know of to core-js is polyfill.io but their polyfills aren't up to standards unfortunately from what I've heard.

[patrickhulce]

Thanks for following up @OZZlE ! Yes, I'm aware of the difference between transpiling and polyfills :)

Are you using useBuiltIns to manage your polyfills or are you requireing them yourself? Perhaps the guide should be expanded to include mentioning this feature of babel-env.

[OZZlE]

@patrickhulce yes I was already using useBuiltIns: 'usage' however we also had this:

// webpack.config.js
module.exports = function config(env, argv = {}) {
  return {
    entry: ['core-js', 'formdata-polyfill', 'whatwg-fetch', `${APP_DIR}/index.js`], // core-js

I'm guessing it was needed in older babel versions. I just removed core-js from there and our main starting point bundle decreased 71 KB :)

However after following that guide it seems like IE11 can load the main bundle without problems but all chunks/lazy loaded components aren't transpiled for some reason and therefor the website is broken in IE11.

webpack.config.js:
https://gist.github.com/OZZlE/f5e57dec4943de51643518352ef61059

packages.json
https://gist.github.com/OZZlE/e38d457ef241ce4f95e2df69abef00a7

We're lazy loading components with https://github.com/gregberge/loadable-components maybe it's an issue there or if it was because the .babelrc should be removed, not sure..

I had to update webpack version from 3 to 4 to get this to work at all also and a lot of other related things.

[patrickhulce]

I just removed core-js from there and our main starting point bundle decreased 71 KB :)

Hurray 🎉

all chunks/lazy loaded components aren't transpiled for some reason and therefor the website is broken in IE11

I don't see anything immediately wrong with the webpack config, but when everything's done correctly you should see two versions of all of your chunks as well (one for the nomodule path to load and one for the esmodules path to load). If that's not the case you might need to tweak the config a bit to ensure the chunks aren't being de-duped into a single esmodules version.

I wouldn't expect loadable-components to have anything to do with it. .babelrc and explicit babel options might have something to do with it? I'm not sure.

[OZZlE]

I have two versions of the chunks as well it's just that the legacy ones doesn't seem to be transpiled, I will try to restore the .babelrc to see if that helps, if that doesn't break the non transpiling for the module version.. will update here if that helps

[OZZlE]

Did this https://github.com/babel/babel/issues/10273#issuecomment-517838332 + I had forgotten to add babel options to svg loader also. Now it works :) It's not mentioned at all the tutorial. .

[OZZlE]

Finally working in IE11 also:

const path = require('path');
const webpack = require('webpack');

const BUILD_DIR = path.resolve(__dirname, 'build/');
const APP_DIR = path.resolve(__dirname, 'src/');

const LiveReloadPlugin = require('webpack-livereload-plugin');
const dartSass = require('dart-sass');

const HtmlWebpackPlugin = require('html-webpack-plugin');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const entry = {};

const getBabelOptions = (targets, useBuiltIns = 'entry') => {
  return {
    "presets": [
      [
        "@babel/preset-env",
        {
          "targets": targets,
          "corejs": {
            version : "3",
            proposals : true
          },
          "useBuiltIns": useBuiltIns,
          modules: false
        }
      ],
      "@babel/preset-react"
    ],
    "plugins": [
      "@babel/plugin-proposal-class-properties",
      "@babel/plugin-proposal-function-bind",
      "@babel/plugin-transform-runtime"
    ]
  };
};

const ruleCss = {
  test: /\.css$/,
  include: APP_DIR,
  exclude: /node_modules/,
  use: [MiniCssExtractPlugin.loader, 'css-loader'],
};
const ruleScss = {
  test: /\.scss$/,
  include: APP_DIR,
  exclude: /node_modules/,
  use: [
    'style-loader',
    'css-loader',
    {
      loader: 'sass-loader',
      options: {
        implementation: dartSass,
      },
    },
  ],
};
const ruleSvg = (targets, useBuiltIns = 'entry') => {
  return {
    test: /\.svg$/,
    use: [
      {
        loader: "babel-loader",
        options: getBabelOptions(targets, useBuiltIns)
      },
      {
        loader: 'react-svg-loader',
        options: {
          svgo: {
            plugins: [
              { removeTitle: false },
            ],
          },
        },
      },
    ],
  };
}

const getPlugins = (nodeEnv) => {
  const plugins = [
    new MiniCssExtractPlugin({ filename: "[name].css", allChunks: true }),
    new MiniCssExtractPlugin({ filename: "[name].scss", allChunks: true }),
    new HtmlWebpackPlugin({ template: "./src/index.html" })
  ];
  if (nodeEnv === 'development') {
    plugins.push(new LiveReloadPlugin({
      appendScriptTag: true,
      hostname: 'company.vagrant.shitagency.se',
      ignore: 'node_modules|build/.*'
    }));
    plugins.push(new webpack.DefinePlugin({
      'process.env.NODE_ENV': JSON.stringify(process.env.NODE_ENV || 'development'),
    }));
  }

  return plugins;
}
const resolve = {
  extensions: ['.js', '.jsx'],
  modules: [
    path.resolve(__dirname, 'src/'),
    'node_modules',
  ],
};

const optimization = {
  namedChunks: true,
  namedModules: true,
  splitChunks: {
    chunks (chunk) {
      // exclude `vendors~main`
      return chunk.name.indexOf('vendors') !== -1; // since it gets a dynamic hash and we don't host using node we don't want to manually update it every time it changes
    }
  },
  chunkIds: "total-size",
  concatenateModules: true,
};

const legacyTargets = {
  "browsers": [
    ">0.25%",
    "IE 11"
  ]
};
const legacyUseBuiltIns = 'entry';
const legacyConfig = (env, argv) => {
  const nodeEnv = env.NODE_ENV ? env.NODE_ENV : 'production';
  const plugins = getPlugins(nodeEnv);
  return {
    entry,
    mode: nodeEnv,
    entry: ['core-js', 'formdata-polyfill', 'whatwg-fetch/dist/fetch.umd.js', `${APP_DIR}/index.js`],
    output: {
      path: BUILD_DIR,
      publicPath: '/skin/frontend/rwd/company/js/react-frontend/build/',
      filename: '[name].react-frontend.js',
      chunkFilename: '[name].[chunkhash].react-frontend.js',
    },
    optimization: optimization,
    resolve: resolve,
    module: {
      rules: [
        {
          test: /\.jsx?$/,
          include: APP_DIR,
          loader: "babel-loader",
          options: getBabelOptions(legacyTargets, legacyUseBuiltIns)
        },
        ruleCss,
        ruleScss,
        ruleSvg(legacyTargets, legacyUseBuiltIns),
      ],
    },
    stats: {
      colors: true,
    },
    devtool: nodeEnv === 'development' ? 'source-map' : '',
    watch: argv.watch,
    watchOptions: {
      poll: 100,
      ignored: ['node_modules'],
    },
    plugins,
  };
}

const modernTargets = { "esmodules": true };
const modernUseBuiltIns = 'usage';
const moduleConfig = (env, argv) => {
  const nodeEnv = env.NODE_ENV ? env.NODE_ENV : 'production';
  const plugins = getPlugins(nodeEnv);
  return {
    entry,
    mode: nodeEnv,
    entry: [`${APP_DIR}/index.js`],
    output: {
      path: BUILD_DIR,
      publicPath: '/skin/frontend/rwd/company/js/react-frontend/build/',
      filename: '[name].module.react-frontend.js',
      chunkFilename: '[name].[chunkhash].module.react-frontend.js',
    },
    optimization: optimization,
    resolve: resolve,
    module: {
      rules: [
        ruleCss,
        {
          test: /\.jsx?$/,
          include: APP_DIR,
          loader: "babel-loader",
          options: getBabelOptions(modernTargets, modernUseBuiltIns)
        },
        ruleScss,
        ruleSvg(modernTargets, modernUseBuiltIns),
      ],
    },
    stats: {
      colors: true,
    },
    devtool: nodeEnv === 'development' ? 'source-map' : '',
    watch: argv.watch,
    watchOptions: {
      poll: 100,
      ignored: ['node_modules'],
    },
    plugins,
  };
}


module.exports = function config(env, argv = {}) {
  return [
    legacyConfig(env, argv), moduleConfig(env, argv)
  ];
};

EDIT: Made a fix for loading the correct module chunk also now

[patrickhulce]

Many thanks for sharing your working configuration so others may benefit @OZZlE! 🎉

[OZZlE]

@patrickhulce after all this Page Speed Insights still complains about unsused js (but not the same amount) - Chrome Lighthouse doesn't mention it and gives a much higher score.. very confusing..

[patrickhulce]

You might need to wait for #10906 to make its way to PSI to make the audit completely go away.