Html-webpack-plugin: TypeError: Cannot read property 'get' of undefined when used with webpack 5

[capaj]

Expected behaviour

TypeError should not be thrown

Current behaviour

TypeError is thrown

Environment

capaj@capaj-i7-4771:~/git_projects/looop/project-alpha$ node -e "var os=require('os');console.log('Node.js ' + process.version + '\n' + os.platform() + ' ' + os.release())"
Node.js v8.11.4
linux 4.18.0-12-generic
capaj@capaj-i7-4771:~/git_projects/looop/project-alpha$ npm --version
6.4.1

capaj@capaj-i7-4771:~/git_projects/looop/project-alpha$ npm ls html-webpack-plugin
looop@ /home/capaj/git_projects/looop/project-alpha
└─┬ @storybook/[email protected]
  └─┬ @storybook/[email protected]
    └── [email protected] 

I have [email protected]

/home/capaj/git_projects/looop/project-alpha/node_modules/html-webpack-plugin/lib/compiler.js:341
    const timestamp = fileTimestamps.get(fileDependency);
                                     ^

TypeError: Cannot read property 'get' of undefined
    at childCompiler.fileDependencies.some (/home/capaj/git_projects/looop/project-alpha/node_modules/html-webpack-plugin/lib/compiler.js:341:38)
    at Array.some (<anonymous>)
    at isChildCompilerCacheOutdated (/home/capaj/git_projects/looop/project-alpha/node_modules/html-webpack-plugin/lib/compiler.js:340:59)
    at Object.hasOutDatedTemplateCache (/home/capaj/git_projects/looop/project-alpha/node_modules/html-webpack-plugin/lib/compiler.js:321:16)
    at compiler.hooks.thisCompilation.tap (/home/capaj/git_projects/looop/project-alpha/node_modules/html-webpack-plugin/index.js:132:25)
    at SyncHook.eval [as call] (eval at create (/home/capaj/git_projects/looop/project-alpha/node_modules/tapable/lib/HookCodeFactory.js:19:10), <anonymous>:7:1)
    at Compiler.newCompilation (/home/capaj/git_projects/looop/project-alpha/node_modules/webpack/lib/Compiler.js:526:30)
    at hooks.beforeCompile.callAsync.err (/home/capaj/git_projects/looop/project-alpha/node_modules/webpack/lib/Compiler.js:567:29)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/capaj/git_projects/looop/project-alpha/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)
    at Compiler.compile (/home/capaj/git_projects/looop/project-alpha/node_modules/webpack/lib/Compiler.js:562:28)
    at compiler.hooks.watchRun.callAsync.err (/home/capaj/git_projects/looop/project-alpha/node_modules/webpack/lib/Watching.js:107:19)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/capaj/git_projects/looop/project-alpha/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:15:1)
    at compiler.cache.endIdle.err (/home/capaj/git_projects/looop/project-alpha/node_modules/webpack/lib/Watching.js:70:33)
    at AsyncParallelHook.eval [as callAsync] (eval at create (/home/capaj/git_projects/looop/project-alpha/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)
    at Cache.endIdle (/home/capaj/git_projects/looop/project-alpha/node_modules/webpack/lib/Cache.js:39:22)
    at Watching._go (/home/capaj/git_projects/looop/project-alpha/node_modules/webpack/lib/Watching.js:68:23)

Sorry I don't have a minimal webpack.config.js

[jantimon]

Thanks that was already mentioned here.

I don't understand completely why and how the filestamps api changed.

Could anyone please provide some insides or pull request to make html-webpack-plugin compatible with webpack 5?

Can anyone please give me some hints how and why the timestamps api changed?

I am using it here to find out wether the html template has to be recompiled.

For webpack 4 it just loops over all compilation file dependency timestamps.
Is getFileTimestamp as fast as that look up or will it add an additional fs lookup?

[sibelius]

here are another error we are getting on webpack 5

 TypeError: Cannot add property htmlWebpackPluginAlterChunks, object is not extensible
@entria/web:     at compiler.hooks.compilation.tap.compilation (/Users/sibelius/Dev/entria/entria-fullstack/node_modules/html-webpack-plugin/index.js:59:56)

you can run it here https://github.com/entria/entria-fullstack/pull/46

it is using 3.2.0

[sibelius]

4.0.0-beta.5 works on our config

[jantimon]

Thanks for the feedback 👍

[PlayMa256]

Although It works on the first build, as you save a file and it is on watch mode, we get the same error, more details here: https://github.com/entria/entria-fullstack/pull/46#issuecomment-452123511 running 4.0.0-beta.5

[jantimon]

Oh sorry you are right - its a breaking change of webpack 5.
I already tried to get some insides on this:

https://github.com/webpack/webpack/issues/8537#issuecomment-449666629

Crash after editing a JS file while webpack-dev-server watches for changes:

const timestamp = fileTimestamps.get(fileDependency);
TypeError: Cannot read property 'get' of undefined
    at childCompiler.fileDependencies.some (/node_modules/html-webpack-plugin/lib/compiler.js:341:38)

webpack-dev-server: 3.1.12 and html-webpack-plugin: 4.0.0-beta.5

Can anyone please give me some hints how and why the timestamps api changed?

I am using it here to find out wether the html template has to be recompiled.

For webpack 4 it just loops over all compilation file dependency timestamps.
Is getFileTimestamp as fast as that look up or will it add an additional fs lookup?

Maybe @sokra can help us here.

[PlayMa256]

friendly ping @sokra

[sokra]

There is a new API in FileSystemInfo.

FileSystemInfo.createSnapshot(startTime, files, directories, missing, options, callback) creates a snapshot of the filesystem to check for validity later.

FileSystemInfo.checkSnapshotValid(snapshot, callback) checks if a the filesystem changed (for the passed files and directories) compared to the snapshot.

[jantimon]

This is the first time I see the FileSystemInfo.createSnapshot api and I must say it is impressive.
It abstracts the entire "do I have to recompile" problem in a very simple way! 👍

Will FileSystemInfo also be back ported to webpack 4 in order to help use so we can support webpack 4 and 5 with one code base?

Is a snapshot automatically created for every compilation or for every compiler?
If not is there an util to create a snapshot for all dependencies of a given compiler?

[benmccann]

@sokra do you have any thoughts on @jantimon's question https://github.com/jantimon/html-webpack-plugin/issues/1129#issuecomment-456086752?

edit: I see some of the same question also asked in https://github.com/webpack/webpack/issues/8982

[jantimon]

@sokra added webpack/webpack#8982 to the "probably" list:

lets see if this moves to yes in the next weeks.

[PutziSan]

Hi, no matter if FileSystemInfo also comes for weppack 4, html-webpack-plugin has to be adapted for webpack 5 in any case and use the new FileSystemInfo API, right? Does this new implementation already exist somewhere, so that we can use html-webpack-plugin with webpack 5?

[jantimon]

@PutziSan that's correct do you want to start and send a pull request?

[PutziSan]

I had patched my packages already, but it's more quick-and-dirty:
This change-comparison reflects the changes I've made locally, so everything is working fine with Webpack 5 again. I didn't really understand the previous caching algorithm and adapted it in the easiest way for me, that's certainly not right for all cases and needs to be looked at again. Also, it doesn't fit perfectly yet, because the first time you render it, it also returns that the cache is invalid.
Unfortunately, I don't have time to understand or improve it any further. But if someone wants a (not perfect) solution with Webpack 5 for now, he can use my changes as a patch.
[edit: added some code, that I forgot to commit]

[jantimon]

@PutziSan wow cool thanks a lot I'll take a look at it! 💯

[egemon]

Does anyone make it work with webpack 5?)

[RichiCoder1]

The above patch appears to work for me still for Webpack 5.0,0-beta0

[jwarkentin]

If anyone is looking for a quick solution, I can't vouch for the correctness of this but it seems to work for me. Install the patch-package NPM module and set the postinstall script as instructed. Then, create a directory named "patches" and a file named something like "html-webpack-plugin+4.0.0-beta.8.patch" with the following contents:

diff --git a/node_modules/html-webpack-plugin/lib/compiler.js b/node_modules/html-webpack-plugin/lib/compiler.js
index 8960360..2f431d7 100644
--- a/node_modules/html-webpack-plugin/lib/compiler.js
+++ b/node_modules/html-webpack-plugin/lib/compiler.js
@@ -336,9 +336,9 @@ function isChildCompilerCacheOutdated (mainCompilation, childCompiler) {
     return false;
   }
   // Check if any dependent file was changed after the last compilation
-  const fileTimestamps = mainCompilation.fileTimestamps;
+  const fileTimestamps = mainCompilation.fileSystemInfo._fileTimestamps;
   const isCacheOutOfDate = childCompiler.fileDependencies.some((fileDependency) => {
-    const timestamp = fileTimestamps.get(fileDependency);
+    const timestamp = fileTimestamps.get(fileDependency).timestamp;
     // If the timestamp is not known the file is new
     // If the timestamp is larger then the file has changed
     // Otherwise the file is still the same

Run npx patch-package and it will at least stop the build from crashing and seems to allow proper hot reloading when I edit things so far.

[OneCyrus]

has may be someone a maintained fork of this?

[stavalfi]

I had patched my packages already, but it's more quick-and-dirty:
This change-comparison reflects the changes I've made locally, so everything is working fine with Webpack 5 again. I didn't really understand the previous caching algorithm and adapted it in the easiest way for me, that's certainly not right for all cases and needs to be looked at again. Also, it doesn't fit perfectly yet, because the first time you render it, it also returns that the cache is invalid.
Unfortunately, I don't have time to understand or improve it any further. But if someone wants a (not perfect) solution with Webpack 5 for now, he can use my changes as a patch.
[edit: added some code, that I forgot to commit]

I confirm that forking his repo is working great and sovled the bug.

[hustcer]

Any update on this issue?

[sorenhoyer]

Seems to still be a problem with webpack in watch mode, but works only on the first build. Webpack 5.0.0-beta.4 and html-webpack-plugin 4.0.0-beta.11

[AlonMiz]

+1
using @PutziSan patch for now...

[hustcer]

+100

[fishman17]

use @PutziSan @jwarkentin patch can build on the first time, but the hot reloading still doesn't seem to work

[jhines2k7]

The patch solution provided by @jwarkentin works for me with hot reloading on a Windows 10 machine running Node v12.14.0 with webpack:5.0.0-beta.11, webpack-cli:3.3.10, html-loader:0.5.5, html-webpack-plugin:4.0.0-beta.11, and webpack-dev-server:3.10.1

[sokra]

Accessing _fileTimestamps is not a perfect solution. It's private and may not include all timestamps.

Instead use the snapshot api to create a snapshot and later check if the snapshot is still valid. Methods: createSnapshot and checkSnapshotValid in FileSystemInfo

[jwarkentin]

@sokra I thought I'd see if I could implement a proper fix real quick but it doesn't have any documentation and looking at the source for it just leaves me with too many questions. I can't tell if there's supposed to be a general cache managed by Webpack that is just accessed by plugins or if the individual plugins actually have to call createSnapshot. I also don't know what each parameter for that call is supposed to be and the jsdoc comments are extremely unhelpful.

I'm not sure much can be done about a proper solution until there's real documentation on Webpack 5 but the main thing needed until then is just for this plugin to not completely crash the build.

[elchicofrommo]

Hey team, so I did the patch locally and it worked for me but I'm not checking in my node_module into my repo so I have to re-apply every time I deploy to a new env. There has to be a better way than this while we're waiting for a solution to get pushed. I'm newish to node/npm. I suppose I could maintain my own repo of http_webpack_plugin and install that but is that the best way to go?

[AlonMiz]

u can use patch-packageuntil a proper fix will be available

Hey team, so I did the patch locally and it worked for me but I'm not checking in my node_module into my repo so I have to re-apply every time I deploy to a new env. There has to be a better way than this while we're waiting for a solution to get pushed. I'm newish to node/npm. I suppose I could maintain my own repo of http_webpack_plugin and install that but is that the best way to go?

[jwarkentin]

For anyone interested in an even quicker and dirtier foolproof patch, here's what I'm doing now:

1. npm i patch-package
2. Open package.json and add this in the scripts section:
   js "postinstall": "patch-package"
3. Create a patches directory in your project root, add a file named html-webpack-plugin+4.0.0-beta.11.patch and paste the following in the file:

diff --git a/node_modules/html-webpack-plugin/lib/compiler.js b/node_modules/html-webpack-plugin/lib/compiler.js
index a13abf3..011294d 100644
--- a/node_modules/html-webpack-plugin/lib/compiler.js
+++ b/node_modules/html-webpack-plugin/lib/compiler.js
@@ -335,6 +335,7 @@ function isChildCompilerCacheOutdated (mainCompilation, childCompiler) {
   if (!childCompiler.compilationStartedTimestamp) {
     return false;
   }
+  return true;
   // Check if any dependent file was changed after the last compilation
   const fileTimestamps = mainCompilation.fileTimestamps;
   const isCacheOutOfDate = childCompiler.fileDependencies.some((fileDependency) => {

1. If you don't need to run an npm install and just need to apply the patch, run npx patch-package or npm run postinstall.

@elchicofrommo I added the step-by-step here for your benefit. Hope it helps!

[jantimon]

Has anyone already tried to use the new FileSystemInfo api to solve this?
Would love to release it as alpha as soon as it works out.

[saravanan10393]

even in 4.0.0-beta.14 of html-webpack plugin i am getting this error
`
/app/node_modules/html-webpack-plugin/lib/compiler.js:341
const timestamp = fileTimestamps.get(fileDependency);

  TypeError: Cannot read property 'get' of undefined
  at /app/node_modules/html-webpack-plugin/lib/compiler.js:341:38

`

is there any timeline to fix this bug. i am using v5.0.0.beta.14 of webpack.

[fivethreeo]

I think it might be possible to do some conditionals with webpack.version to make PutziSan‘s changes backward compatible.

[fivethreeo]

I forked this repo and cherry picked PutziSan’s changes, working on fixing tests so they pass on webpack5. If I get it fixed I will release a scoped package until changes can be incorporated in master.

[jantimon]

@fivethreeo if we have a working solution I would love to release it as html-webpack-plugin 5 alpha however the current solution is using an unsupported private feature of webpack which might break once they publish the new major

I am happy to publish it as alpha if anyone has time to develop a solution based on the new FileSystemInfo.createSnapshot proposed by the webpack core team

[fivethreeo]

I got all tests but two to work in webpack5 with snapshot solution. But testing is a bit cumbersome since the webpack recompilation simulator does not currently support webpack5. Should the next release be backward compatible?

[fivethreeo]

A alpha release sounds great 😀

[elchicofrommo]

I'm new to webpack and open source contributing. I'd love to help out. Where do I find the api specs? Also I've been looking around this repo for the tests but i'm not seeing them. Its probably something obvious I'm just overlooking. Any pointers?

[fivethreeo]

They are in spec/

[ibufu]

Thank @PutziSan! I forked the version of @PutziSan and published it to npm as a temporary solution.

npm i html-webpack-plugin-compatible-5 -D

[elchicofrommo]

@jantimon are you still working on that alpha? It might be a nice intermediary until a final solution can be completed.

[fivethreeo]

@elchicofrommo https://github.com/fivethreeo/html-webpack-plugin/tree/webpack5-dev just a testing dep needs to be updated first

[jantimon]

Wow @fivethreeo that looks amazing 👍👍

[fivethreeo]

But regarding backward compatibility. Just keep both solutions until we deprecate webpack4?

[jantimon]

What about having two compiler.js files? One for webpack 4 and one for webpack 5?

[fivethreeo]

Yes, that might be the best solution.

On Thu, 2 Apr 2020 at 18:18, Jan Nicklas [e-mail] wrote:

What about having two compiler.js files? One for webpack 4 and one for
webpack 5?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jantimon/html-webpack-plugin/issues/1129#issuecomment-607945705,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAADGXAJOUBPKQW4OIEQJ3RKS3DRANCNFSM4GMAANFA
.

--
Øyvind Saltvik

[jantimon]

Added some feedback to your pr here: #1390

[fivethreeo]

Typescript did not like two webpack versions.

[fivethreeo]

I think we need to do paralell releases with typescript. Unless there is a way to do dynamic typedef imports.

[jantimon]

Fixed in html-webpack-plugin 4.1.0