Webpack-dev-server: Page does not refresh on HTML change when HMR is on

[andreyvolokitin]

• Operating System: OS X 10.11.6
• Node Version: 9.2.1
• NPM Version: 5.6.0
• webpack Version: 3.8.1

• webpack-dev-server Version: 2.10.1

• [ ] This is a bug

• [x] This is a modification request

Code

https://github.com/andreyvolokitin/test-webpack-dev-server

Expected Behavior

Editing ./src/index.html should cause recompiling and page refresh in the browser

Actual Behavior

Recompiling is happening, but the page in the browser does not refresh

For Bugs; How can we reproduce the behavior?

1. git clone https://github.com/andreyvolokitin/test-webpack-dev-server && cd test-webpack-dev-server
2. npm install && npm start
3. Edit ./src/index.html — compiling is happening, but the page in the browser does not refresh
4. Open browser console and edit ./src/test.js — HMR works

[andreyvolokitin]

I found that it is possible to get a list of changed files which caused recompilation. Would it be feasible to add some logic here, so that when the list of changed files contains certain extensions — we pass a flag to _sendStats() to indicate that full page reload is needed? It is unclear by now what extensions should be included: .html is fine, but what about countless templating engines which could be used instead and would need the same logic applied?

[alexander-akait]

@andreyvolokitin we use this snippet for php:

    before(app, server) {
      const chokidar = require("chokidar");
      const files = [
        // Refreshing php files
        "**/*.php"
      ];

      chokidar
        .watch(files, {
          alwaysStat: true,
          atomic: false,
          followSymlinks: false,
          ignoreInitial: true,
          ignorePermissionErrors: true,
          ignored,
          interval: typeof poll === "number" ? poll : null,
          persistent: true,
          usePolling: Boolean(poll)
        })
        .on("all", () => {
          server.sockWrite(server.sockets, "content-changed");
        });
    }

[alexander-akait]

@shellscape what do you think about add to documentation example above?

[andreyvolokitin]

@evilebottnawi thanks! It works, but what about async issues — i.e. we pass a content-changed to a socket, and at the same time webpack is starting to compile the same files. We need to reload the page after webpack compilation, but as far as I understand with this snippet there is no guarantee of that? Probably we can add some delay then, but we can not precisely know the exact current compilation time to accommodate it in a delay (and this may cause unwanted delay time). Or this is actually no-issue?

[alexander-akait]

@andreyvolokitin can you describe you issue on example? You can add own logic to snippet above, also you can use browser-sync plugin for webpack.

[andreyvolokitin]

@evilebottnawi my .html files are generated using PostHTML, so webpack is compiling them from the source during each recompilation (they are used by HtmlWebpackPlugin in a template option). This snippet is using chokidar to watch .html files. Webpack is watching the same files. When files change — webpack starts a recompilation, and at the same time chokidar is executing its callback (server.sockWrite(server.sockets, "content-changed");). So webpack recompilation needs to complete before page refresh happens, so that newly generated HTML actually appears in the browser. Might this be a race condition, like if page refresh happens before webpack completes recompilation, so that refreshed page will contain old HTML?

[andreyvolokitin]

What I mean is there are two separate processes: webpack compilation of updated HTML and chokidar callback on this HTML changes. They need to complete one after another, but there is no guarantee for that

[alexander-akait]

@andreyvolokitin HtmlWebpackPlugin works through webpack api and should be compatibility with dev-server no need manually reload (i am not familiar with html plugin)

[andreyvolokitin]

@evilebottnawi with HtmlWebpackPlugin the page is not reloaded when .html changes. There are numerous issues about this in html-webpack-plugin repo (i.e. https://github.com/jantimon/html-webpack-plugin/issues/232), as well as this repo and probably others too. The example code from this issue is actually using html-webpack-plugin and shows this (https://github.com/andreyvolokitin/test-webpack-dev-server). But it is clear that this issue comes from webpack-dev-server which simply does not take .html changes into account when hot: true

[andreyvolokitin]

@evilebottnawi Would it be a bad thing to add an option to webpack-dev-server, containing a list of required file extensions (like ['.html', '.jade']) which then would be used as described here: https://github.com/webpack/webpack-dev-server/issues/1271#issuecomment-359815525 ? I know I can watch source html files and reload the page on their changes, but it looks like a hack considering that my html is compiled. Page reload should be more like compilation callback and not a parallel process of compilation. And it is clear that this feature is needed either way

[andreyvolokitin]

If it would be possible though to subscribe a one-time function to compiler event hook like "done" within chokidar callback inside devServer.before(), then I could get a page reload guaranteed after compiling. But I am afraid devServer.before() does not expose webpack compiler... And I guess there is no way do define a "once" callback on compiler

[andreyvolokitin]

Maybe add "onCompile" callback here and expose compiler and server to it? Then on each compile, it will be possible to get changed files with compiler.watchFileSystem.watcher.mtimes and to do page reload with server.sockWrite(server.sockets, "content-changed"):

devServer: {
  hot: true,
  onCompile(compiler, server) {
    const watchFiles = ['.html', '.hbs'];
    const changedFiles = Object.keys(compiler.watchFileSystem.watcher.mtimes);

    if (
      this.hot && 
      changedFiles.some(filePath => watchFiles.includes(path.parse(filePath).ext))
    ) {
      server.sockWrite(server.sockets, "content-changed");
    }
  }
}

[andreyvolokitin]

I think I found a minimal change that can suit the need: simply expose compiler to before and after callbacks. Then everything could be done within the custom handler, because apparently, it is possible to add multiple hooks for same compiler event (compiler.plugin('done', fn)):

devServer: {
  hot: true,
  before(app, server, compiler) {
    const watchFiles = ['.html', '.hbs'];

    compiler.plugin('done', () => {
      const changedFiles = Object.keys(compiler.watchFileSystem.watcher.mtimes);

      if (
        this.hot &&
        changedFiles.some(filePath => watchFiles.includes(path.parse(filePath).ext))
      ) {
        server.sockWrite(server.sockets, 'content-changed');
      }
    });
  }
}

And for simple static files watching we again can use before and chokidar, or watchContentBase. Though probably in the future it would be worth to include all this within webpack-dev-server.

I guess for now even this proposed minimal change can't be added because it may qualify as a "new feature", and there is a maintenance mode happening. Hope this will be resolved sooner or later...

---

The only concern is this quote from CONTRIBUTING.md, as my custom handler using compiler.watchFileSystem:

A user should not try to implement stuff that accesses the webpack filesystem. This lead to bugs (the middleware does it while blocking requests until the compilation has finished, the blocking is important).

[ripeshade]

devServer: {
    contentBase: path.join(__dirname, 'src'),
    watchContentBase: true,
    hot: true,
  }

Maybe, it's some sort of a solution.

[andreyvolokitin]

I guess this is similar to the @evilebottnawi solution — server will start to watch files in src directory using chokidar, so there is a possibility of the same race condition issue if HTML gets compiled

[TsaiZhuoLin]

@ripeshade
watchContentBase this optional setting fixed the issue I have. Now I can use HMR and the HTML also can auto refresh. Thx mate. I think this might be the reason why webpack team did not fix the issue. Because there is a solution already!

[avdd]

@andreyvolokitin give this a try

I don't think that contentBase is the right place to put HTML templates.

[andreyvolokitin]

@avdd, unfortunately, it breaks hot reloading: I suppose on each CSS/JS change html-webpack-plugin triggers html-webpack-plugin-after-emit which then causes full page reload

[avdd]

@andreyvolokitin good catch. I've updated that snippet to compare with the previous emit.

[andreyvolokitin]

@avdd very nice, thanks! That's exactly what was needed: comparing previous and current HTML after compilation; if it changes, then we need to reload. Hope there are no hidden edge cases, will take my time to use and test it.

Now the question remains: could this issue be relevant without html-webpack-plugin? I think @evilebottnawi mentioned that they had this issue without using html-webpack-plugin. Currently, HTML changes are not reloaded with webpack-dev-server "by design", html-webpack-plugin can solve this, but what if we not using it? To put it another way: should this issue be a concern of webpack-dev-server?

[cloudratha]

I've had good success using the private watch method provided by the Server.js

before(app, server) {
    server._watch(`some/path/to/watch/*/**.html`);
}

Although using an internal (private) method doesn't seem like a good idea.

[SassNinja]

I'm facing the same issue: changing my template doesn't cause a reload in webpack-dev-server

The suggested solution of both, @avdd and @cloudratha, are working (thanks btw!)
However none of them has 100% convinced me yet to use it for all my projects.
Let me explain why:

1. server._watch

before(app, server) {
    server._watch(`src/*/**.html`);
}

What I really like about this one is the simplicity: only one line
However I'm a bit afraid to rely on an internal method and am not sure how this exactly works internally and if there's no risk of race conditions.

@evilebottnawi is there a specific reason why the _watch method is not public (without underscore)?

1. reloadHtml

plugins: [
    new HtmlWebpackPlugin({
        template: 'src/index.html',
    }),
    reloadHtml
]

function reloadHtml() {
    const cache = {}
    const plugin = {name: 'CustomHtmlReloadPlugin'}
    this.hooks.compilation.tap(plugin, compilation => {
        compilation.hooks.htmlWebpackPluginAfterEmit.tap(plugin, data => {
            const orig = cache[data.outputName];
            const html = data.html.source();
            if (orig && orig !== html) {
                devServer.sockWrite(devServer.sockets, 'content-changed')
            }
            cache[data.outputName] = html;
        });
    });
}

What I really like about this one is that it does not cause a reload when nothing has changed in the template (it's probably not likely you save the template several times without any changes but nevertheless I like it).
However this solution also has downsides:

• specific to the html-webpack-plugin relying on the compilation.hooks.htmlWebpackPluginAfterEmit what will probably break with next major update
• doesn't seem efficient to diff the html everytime (though not sure about how relevant this is)

I think for now I'm staying with the server._watch solution because it doesn't require much additional code.
But it will definitely be great if one solution gets integrated into the plugin (either in webpack-dev-server or in html-webpack-plugin) because this is a common use case imo.

[alexander-akait]

@SassNinja in next major release we will implement watchPaths option for this cases, now you can use _watch method, it is legacy reason

[SassNinja]

good to hear there's going to be an option for this case :)
thanks

[gremo]

Hi, I have the same problem... any update on this?

[hamtramvayniy]

devServer: {
    contentBase: path.join(__dirname, 'src'),
    watchContentBase: true,
    hot: true,
  }

Maybe, it's some sort of a solution.

This is hot reload html is absolute worked!!! Thank you man!

[gremo]

The solution proposed by @ripeshade isn't working form me. The reload works, but i'ts a full reload.

Any update?

[phxism]

me too @ripeshade
It seems to make HMR not work, even I just edit a style file (e.g. src/css/app.css) that triggers web browser a full reload/refresh. any changes in src/cs/app.js or src/js/app.js should trigger a HMR , src/index.html triggers a browser refresh is exact what I want.

I use the lastest stable webpack and follow the guides of official webpack website.

"webpack": "^4.43.0",
"webpack-cli": "^3.3.11",
"webpack-dev-server": "^3.11.0",
"webpack-merge": "^4.2.2",

devServer: {
    contentBase: path.join(__dirname, 'src'),
    watchContentBase: true,
    hot: true,
  }

Maybe I need to change the contentBase to path.join(__dirname, path/to/your/html-template),

[LeoniePhiline]

Maybe this helps someone:

This post shows some 2020 (webpack@4 / [email protected]) solution with Hot Module Reloading and liveReload of the compiled and assets-injected index.html - but with the live reload happening only after compilation has finished.

I needed to solve this scenario for integrating a single page application into a CMS page tree; in this case the TYPO3 CMS: The compiled index.html file is used as a CMS page template at the point in the page tree where the SPA should be located.

The problem with webpack@4 / [email protected] was that the dev server's _watch() sends the content-changed socket message (which causes live reload of the entire hmr-enabled page) before compilation is even started.
This caused an always outdated CMS page template. (I always had to reload twice to pull the latest changes made to the Vue SPA's index.html template).

My successful configuration to fix this issue was based on many comments here, but altered because the solutions posted above appear to be no longer compatible.
(I did not want to use the _watch() option in Server.js, because it would cause a double liveReload (once before and once after compilation). Instead, I wanted to disable automatic liveReload and send the 'content-changed' socket message manually.)
Using the devServer's before(app, server, compiler) function did not work for me, since the compiler argument was always undefined. Using after() works like a charm, though.

I would have liked to extract the functionality into an actual plugin, but I did not find out how to gain access to the server in order to sockWrite() inside a plugin. Please enlighten me if you have ideas about that. :)

const fs = require('fs');
const path = require('path');
const { HotModuleReplacementPlugin } = require('webpack');
const WatchReportChangesPlugin = require('./webpack/plugins/watch-report-changes-plugin.js');
const BeforeCompileClearPageCachePlugin = require('./webpack/plugins/before-compile-clear-page-cache-plugin.js');

module.exports = {
  publicPath: '/typo3conf/ext/my_sitepackage/Resources/Public/Build/SPA/',
  outputDir: path.resolve(__dirname, '../../../Public/Build/SPA/'), // Used as TYPO3 `templateRootPaths.10`.
  indexPath: 'Index.html',
  configureWebpack: {
    plugins: [
      new WatchReportChangesPlugin(),
      new BeforeCompileClearTYPO3PageCachePlugin(['spa_app']), // Tagged via TypoScript `stdWrap.addPageCacheTags`.
      new HotModuleReplacementPlugin(),
    ]
  },
  devServer: {
    // The dev server is running on the local host `https://mylocalmachine:8080/`.
    // It is forced writing files to disk, to make sure that TYPO3
    // can pick up the `Index.html` template for the SPA page,
    // containing the initial assets. All assets changed during
    // runtime are hot-reloaded through the webpack dev server.
    //
    // When changing the ./public/index.html Vue app base template,
    // the `DelayedLiveReloadPlugin` below manually triggers a
    // `liveReload` in the browser. The automatic `liveReload`
    // is disabled, because it fires before any compilation
    // has been done.
    after: function(app, server, compiler) {
      compiler.hooks.done.tapAsync(
        'DelayedLiveReloadPlugin',
        async (compilation, callback) => {
          // Only look out for changed html files.
          const watchFiles = ['.html'];
          const changedFiles = Object.keys(
            compiler.watchFileSystem.watcher.mtimes
          );
          // Send 'content-changed' socket message to manually tigger liveReload.
          if (
            this.hot &&
            changedFiles.some(filePath => watchFiles.includes(path.parse(filePath).ext))
          ) {
            server.sockWrite(server.sockets, "content-changed");
          }

          callback();
        }
      );
    },
    cert: fs.readFileSync('./webpack/tls/mylocalmachine.pem'), // `mkcert mylocalmachine`
    disableHostCheck: true,
    port: 8080,
    host: '127.0.0.1',
    hot: true,
    https: true,
    key: fs.readFileSync('./webpack/tls/mylocalmachine-key.pem'), // `mkcert mylocalmachine`
    liveReload: false,
    logLevel: 'debug',
    writeToDisk: true, // Make sure TYPO3 can pick up the compiled file. By default webpack-dev-server does not emit.
  }
};

Just for completeness, here's the BeforeCompileClearTYPO3PageCachePlugin (TYPO3 runs in docker containers, while the webpack dev server (wrapped by vue run serve via @vue/cli) runs on my docker host machine):

const {
  info,
  execa,
} = require('@vue/cli-shared-utils');

class BeforeCompileClearTYPO3PageCachePlugin {
  constructor(tags) {
    this.tags = tags;
  }
  get tagsFormatted() {
    return this.tags.join(', ');
  }
  apply(compiler) {
    // Use async hook, so we can decide when the compilation should start.
    compiler.hooks.beforeCompile.tapAsync(
      'BeforeCompileClearTYPO3PageCachePlugin',
      async (compilation, callback) => {
        // Flush provided TYPO3 page cache tags before (re-)compiling source.
        info(`Flush TYPO3 cache tags ${this.tagsFormatted}.`);

        await execa(
          'docker-compose',
          [
            'exec', '-T', 'www-php-cli', // Name of the php:7-cli-alpine docker-compose service.
            'vendor/bin/typo3cms', 'cache:flushtags', this.tags.join(','),
            '--groups', 'pages'
          ],
          {
            stdio: 'inherit',
          }
        );

        // Start compilation now
        callback();
      }
    );
  }
}

module.exports = BeforeCompileClearTYPO3PageCachePlugin;

Hope this helps someone. :)

[alexander-akait]

@LeoniePhiline It is under developers for v4, we will have watchFiles: [...globs] option for simple integration with php/ruby/etc projects

[LeoniePhiline]

@evilebottnawi Wow, fantastic news! Thanks for sharing. :)

PS:
I am not sure if the watchFiles option fixes this specific issue: Is this new option about watching not only the JS source but also e.g. php or ruby files? In that case it wouldn't help. But maybe I am misunderstanding what watchFiles might be about.

The issue many here in this thread had was that in Server.js, a liveReload-enabled content-changed message is sent through the websocket, triggering page reload before the files are compiled, instead of after all compilation is done.

This might work well in cases where webpack-dev-server serves all files, including the index.html from memory, but it does not work in cases where you use writeToDisk and use the built index.html (with all injected assets) as base template for another web framework.

E.g. my html-webpack-plugin template index.html contains not only tags like <%= htmlWebpackPlugin.options.title %> but also tags for TYPO3 fluid, like {headerData -> f:format.raw()}.

All I needed was to have the content-changed message delayed until e.g. compiler.hooks.done.
Will watchFiles help with that?

[alexander-akait]

@LeoniePhiline yes, watchFiles will send changes only after compilation is done to prevent weird behavior

[foxwoods369]

This is currently working for me with html-webpack-plugin^4.1.5, webpack^5.20.1, webpack-dev-server^3.11.2:

    devServer: {
        watchContentBase: true,
        contentBase: path.resolve(__dirname, 'dist'),
        writeToDisk: true,
    },
    plugins: [
        new HtmlWebpackPlugin({
            filename: 'index.html',
            template: './templates/home.html'
        }),

Key part is the writeToDisk: true setting - this ensures webpack-dev-server outputs to my dist folder which serves as the content base.