Webpack-cli: Infinite loop / build timing out when running NODE_ENV=production webpack

[karinathomasbbc]

Describe the bug

We are seeing an infinite loop when running NODE_ENV=production webpack

What is the current behavior?

To Reproduce

Steps to reproduce the behavior:

• Checkout the following repo: https://github.com/bbc/simorgh
• git clone https://github.com/bbc/simorgh.git && cd simorgh
• git checkout dependabot/npm_and_yarn/webpack-cli-4.2.0
• npm install
• npm run build:local

Alternatively, observe that GitHub actions are failing at the "Install & Build Simorgh" phase, due to the build timing out - https://github.com/bbc/simorgh/runs/1419886343 - the build was cancelled after 5h+ of trying to build the application.

Expected behavior

Screenshots

Please paste the results of webpack-cli info here, and mention other relevant information

System:
    OS: macOS 10.15.7
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Memory: 1.24 GB / 16.00 GB
  Binaries:
    Node: 12.16.2 - ~/.nvm/versions/node/v12.16.2/bin/node
    Yarn: 1.22.5 - /usr/local/bin/yarn
    npm: 6.14.8 - ~/.nvm/versions/node/v12.16.2/bin/npm
  Browsers:
    Chrome: 86.0.4240.198
    Firefox: 82.0.2
    Safari: 14.0.1
  Packages:
    brotli-webpack-plugin: ^1.1.0 => 1.1.0 
    compression-webpack-plugin: ^6.0.2 => 6.1.0 
    copy-webpack-plugin: ^6.2.0 => 6.3.0 
    start-server-webpack-plugin: ^2.2.5 => 2.2.5 
    terser-webpack-plugin: ^4.2.2 => 4.2.3 
    webpack: ^4.44.2 => 4.44.2 
    webpack-bundle-analyzer: ^3.9.0 => 3.9.0 
    webpack-cli: ^4.2.0 => 4.2.0 
    webpack-dev-server: ^3.11.0 => 3.11.0 
    webpack-merge: ^5.1.4 => 5.3.0 
    webpack-node-externals: ^2.5.2 => 2.5.2 
  Global Packages:
    webpack-cli: 4.2.0

Additional context

[alexander-akait]

Looks at warnings:

WARNING in Conflict: Multiple assets emit different content to the same filename static/js/.gz

WARNING in Conflict: Multiple assets emit different content to the same filename static/js/.gz

It is very bad, you need to fix it, maybe even add tests to catch this ploblems, ideally you should not have warhings/errors in your build, it is very easy to tests, just do test build on CI and look in stats (stats.warnings/stats.errors).

There is no infinity loop, you write/change files/directories inside directories which you watch (no magic :mage: ), so webpack starts compilation again.

This file every time changed node_modules/@bbc/moment-timezone-include/tz/2020a-raw.json

Note - you don't need brotli-webpack-plugin, compression-webpack-plugin supports brotli (https://github.com/webpack-contrib/compression-webpack-plugin#using-brotli)

Investigate why it happens, answer will be soon

[alexander-akait]

Also my recommendation to add the name for configurations, you will see:

[webpack-cli] Compilation client starting...
[webpack-cli] Compilation server starting...

it is more readable

[alexander-akait]

But here the interesting problem, webpack@4 has bug, when some of configuration have watch: true it run watch for both compilations, fixed for v5, workaround (in webpack.config.client.js):

watchOptions: {
  ignored: [path.resolve(__dirname, 'node_modules/@bbc/moment-timezone-include/tz/2020a-raw.json')]
},

[alexander-akait]

Feel free to feedback

[karinathomasbbc]

Hi @evilebottnawi, thanks for your comments. We have resolved it by setting watch: false in webpack.config.client.js

[alexander-akait]

@karinathomasbbc Anyway changing something in node_modules is very bad, PnP will be broken, cache will be ineffective + it is unsafe, because webpack@5 will load always old version of file