Fable-loader is failing HMR

[grishace]

Description

i ｢wdm｣: Compiling...
× ｢wdm｣: Hash: ce06062f60f1c79afc61
Version: webpack 4.16.5
Time: 585ms
Built at: 12/11/2018 2:41:05 PM
                                     Asset       Size  Chunks                    Chunk Names
                                    app.js   1.02 MiB     app  [emitted]  [big]  app
    app.7ff4fb80ca2fc55de4c9.hot-update.js   2.67 KiB     app  [emitted]         app
      7ff4fb80ca2fc55de4c9.hot-update.json   45 bytes          [emitted]
                                  main.css    262 KiB     app  [emitted]         app
                                app.js.map   1.13 MiB     app  [emitted]         app
app.7ff4fb80ca2fc55de4c9.hot-update.js.map  115 bytes     app  [emitted]         app
                              main.css.map   85 bytes     app  [emitted]         app
 + 3 hidden assets
Entrypoint app [big] = vendors.js vendors.js.map app.js app.7ff4fb80ca2fc55de4c9.hot-update.js main.css app.js.map app.7ff4fb80ca2fc55de4c9.hot-update.js.map main.css.map
[./App.fs] 375 bytes {app} [built] [failed] [1 error]
    + 426 hidden modules

ERROR in ./App.fs
Module build failed (from C:/Users/gbelenkiy/source/repos/TSO/node_modules/fable-loader/index.js):
Error: Cannot call write after a stream was destroyed
    at Loader.getCompiler.send.then.catch.err (C:\Users\gbelenkiy\source\repos\TSO\node_modules\fable-loader\index.js:125:18)
    at process._tickCallback (internal/process/next_tick.js:68:7)
 @ ./Client.fs 3:0-208 20:11-18 21:5-14 21:16-23 34:15-35
 @ ./Client.fsproj
 @ multi (webpack)-dev-server/client?http://localhost:8080 (webpack)/hot/dev-server.js @babel/polyfill url-polyfill whatwg-fetch ./Client.fsproj ./main.scss ./favicon.png
i ｢wdm｣: Failed to compile.

Repro code

I followed instructions on https://elmish.github.io/hmr/ to enable HMR.
Didn't check before, but HMR was working with dotnet-based fable version.

Expected and actual results

HMR

Related information

• Fable version (dotnet fable --version):

fable-babel-plugins@~2.1.0:
version "2.1.0"

fable-compiler@^2.1.1:
version "2.1.8"

fable-loader@^2.0.0:
version "2.1.1"

• Operating system: win10-x64

[alfonsogarciacaro]

Hi @grishace! I just tested HMR on Windows with fable-loader 2.1.1 and fable-compiler 2.1.8 and it's working. Can you please make sure you're using Fable.Elmish.HMR >= 3.0.0 and that the open statement goes after Elmish.React?

open Elmish.React
open Elmish.HMR

cc @MangelMaxime

[MangelMaxime]

For me HMR is working we latest fable-loader.

@grishace For example fulma-demo, use Fable 2.1 and works. Make sure to have open Elmish.HMR as the latest open statement.

And have all the Elmish dependencies upgraded to their latest version if possible.

[grishace]

@alfonsogarciacaro @MangelMaxime

yes, Elmish.HMR is the last open.

I cannot rewrite my whole app (and its structure) +configuration to be the same as in fulma-demo at the moment. I've started with SAFE-Template about 3 months ago and gradually added and updated stuff.
Any ideas what possibly could go wrong?

The app works as expected. Only HMR is failing.

[0x53A]

So I have no actual clue what the error is, but here is how I would try to investigate, take it as you want ....

Error: Cannot call write after a stream was destroyed

Webpack

this looks like something inside webpack is borked.

---

First thing I'd try is to just update everything, worked often enough for me (yarn upgrade --all).

---

If that isn't feasable because of too many breaking changes, then I'd update all fable stuff and webpack itself selectively.

---

Is maybe a webpack plugin at fault? Can you temporarily disable all plugins?

---

Stream?

It says a stream was destroyed.

Does node_modules\fable-loader\index.js line 125 give you any hints what stream that could be?

Is it the websocket stream from webpack-dev-server to the HMR module inside the browser?
Websockets are still a bit sensitive, do you have a corporate proxy or antivirus? Could that somehow mess with the connection?

Is it the IO stream between fable-loader and fable-compiler? Does the fable-compiler process maybe crash hard? Can you check the windows even-log for any crashes?

---

[grishace]

@0x53A Thanks for the tips!

1. Disabled all the plugins except HMR. The error is still there.
2.

    getCompiler(this._compiler, opts.cli).send(msg).then(data => {
....
    })
    .catch(err => {
        callback(new Error(err.message)) // this is the line...
    })

I've added msg logging and it contains the full path to the changed file...
Not sure what's in getCompiler result - will keep digging.

1. Everything's updated. Ne errors in the EventLog, no corp AV logs, no proxy (I'm on the localhost).

[alfonsogarciacaro]

I think I did see that error when working on 2.1, but it went away after adding CloseOutput=false here.

@grishace When you say that only HMR fails, you mean when you disable Elmish.HMR the app reloads automatically without problems? Does HMR work for you in other projects? For example, can you please try cloning the fulma-demo, run yarn webpack-dev-server and make a change in the code to see if HMR works?

[grishace]

@alfonsogarciacaro I mean when I stop webpack-dev-server and start everything again the app is still functioning. And everything's fine until I edit and save the App.fs. Then I get the error and have to start over.

fulma-demo works though - it was my first test when @MangelMaxime mentioned that everything works on his end.

My master is on 2.0.11 and fable-loader works.

Apologize if I don't understand all the machinery. I even don't need automatic reload - app state will be wrong most of the cases anyways - I just need to refresh the page and see my changes.

[alfonsogarciacaro]

No need to apologize! I know it's tricky to learn how all the pieces work together :) Hot Module Replacement (HMR) injects the new code without refreshing the page, but if you don't need it you can just try with Hot Reload (automatically refresh the page on code changes) which automatically happens by default when using webpack dev server.

• Try removing Elmish.HMR (you just need to delete the open Elmish.HMR statement).
• How do you start the project? If you already upgraded to Fable 2.1, are you running webpack dev server directly (yarn webpack-dev-server)?
• If you haven't already, it's worth to remove node_modules folder (or better yet, run git clean -xfd but be sure to commit new files beforehand!) and redownload everything again just in case.

[grishace]

@alfonsogarciacaro I went back and checked out a pre-2.1 commit. fable-loader and HMR are working ok. The next commit was with fable-compiler@^2.1.1: version "2.1.2" fable-loader@^2.0.0: version "2.1.0" and build changes to run yarn webpack instead of dotnet fable.
And here's when it started to break for me. Was doing server-side stuff until recently so that's why I did not notice it :( Would I get a "normal" build if yarn command line/webpack config are wrong? Why only on the fly compilation is failing?

[alfonsogarciacaro]

Honestly I've no idea. It's difficult to know without being able to reproduce :/ Maybe updating other dependencies like webpack, webpack-dev-server and @babel/core can have some effect? Did you try any of the suggestions in my previous comment?

[grishace]

@alfonsogarciacaro yep. and after getting no result just stepped back one commit at the time. so that's how I found when it happened

[alfonsogarciacaro]

Sorry @grishace, could you please give more information? What do you mean exactly by "getting no result"? After removing Elmish.HMR you still got the error? That would mean it's not an HMR problem, but a hot reload or watch recompilation issue. Is your project public or could you please create a repository that we can use to reproduce the problem?

Please understand that the latest version is working for other people (hopefully most of them, as I haven't received any other similar report) so it may be that the problem happens because of a combination of the new version and your project config. We need to find it out to spot the issue.

[grishace]

@alfonsogarciacaro yes the error is the same. That's what I apologized for - to me HMR is the whole thing - watch-recompile-reload. The error logged to the console gives no details why it is failing. Just Cannot call write after a stream was destroyed from the fable-loader.

2.0.11 - everything's fine.
2.1.2 (with command line changes https://www.npmjs.com/package/fable-loader#usage) - issue is there.

I will try to change webpack config to match fulma-demo, but my folders structure is different and I'm not using html template - js/css references are already in there. And the issue is reproducible even when I leave Client.fsproj only (without polyfill and other assets).

I'll keep digging.

[grishace]

@alfonsogarciacaro btw - is there any way to improve logging? I mean maybe I will change something locally to find out why and where this "stream is destroyed"? Tried to add more logging to the fable-loader/index.js and the only thing I see logged in the message is the full path to the file I changed to trigger the build

[grishace]

@alfonsogarciacaro And here's the full exception object (from 2.1.2 so line numbers might be off the latest version)

i ｢wdm｣: Compiling...
*** { Error [ERR_STREAM_DESTROYED]: Cannot call write after a stream was destroyed
    at doWrite (_stream_writable.js:406:19)
    at writeOrBuffer (_stream_writable.js:394:5)
    at Socket.Writable.write (_stream_writable.js:294:11)
    at Promise (C:\Users\gbelenkiy\source\repos\TSO\node_modules\fable-compiler\dist\index.js:85:29)
    at new Promise (<anonymous>)
    at Object.send (C:\Users\gbelenkiy\source\repos\TSO\node_modules\fable-compiler\dist\index.js:82:20)
    at Object.Loader (C:\Users\gbelenkiy\source\repos\TSO\node_modules\fable-loader\index.js:75:43)
    at LOADER_EXECUTION (C:\Users\gbelenkiy\source\repos\TSO\node_modules\loader-runner\lib\LoaderRunner.js:119:14)
    at runSyncOrAsync (C:\Users\gbelenkiy\source\repos\TSO\node_modules\loader-runner\lib\LoaderRunner.js:120:4)
    at iterateNormalLoaders (C:\Users\gbelenkiy\source\repos\TSO\node_modules\loader-runner\lib\LoaderRunner.js:229:2)
    at Array.<anonymous> (C:\Users\gbelenkiy\source\repos\TSO\node_modules\loader-runner\lib\LoaderRunner.js:202:4)
    at Storage.finished (C:\Users\gbelenkiy\source\repos\TSO\node_modules\enhanced-resolve\lib\CachedInputFileSystem.js:43:16)
    at provider (C:\Users\gbelenkiy\source\repos\TSO\node_modules\enhanced-resolve\lib\CachedInputFileSystem.js:79:9)
    at C:\Users\gbelenkiy\source\repos\TSO\node_modules\graceful-fs\graceful-fs.js:90:16
    at FSReqWrap.readFileAfterClose [as oncomplete] (internal/fs/read_file_context.js:53:3)
  [stack]:
   'Error [ERR_STREAM_DESTROYED]: Cannot call write after a stream was destroyed\n    at doWrite (_stream_writable.js:406:19)\n    at writeOrBuffer (_stream_writable.js:394:5)\n    at Socket.Writable.write (_stream_writable.js:294:11)\n    at Promise (C:\\Users\\gbelenkiy\\source\\repos\\TSO\\node_modules\\fable-compiler\\dist\\index.js:85:29)\n    at new Promise (<anonymous>)\n    at Object.send (C:\\Users\\gbelenkiy\\source\\repos\\TSO\\node_modules\\fable-compiler\\dist\\index.js:82:20)\n    at Object.Loader (C:\\Users\\gbelenkiy\\source\\repos\\TSO\\node_modules\\fable-loader\\index.js:75:43)\n    at LOADER_EXECUTION (C:\\Users\\gbelenkiy\\source\\repos\\TSO\\node_modules\\loader-runner\\lib\\LoaderRunner.js:119:14)\n    at runSyncOrAsync (C:\\Users\\gbelenkiy\\source\\repos\\TSO\\node_modules\\loader-runner\\lib\\LoaderRunner.js:120:4)\n    at iterateNormalLoaders (C:\\Users\\gbelenkiy\\source\\repos\\TSO\\node_modules\\loader-runner\\lib\\LoaderRunner.js:229:2)\n    at Array.<anonymous> (C:\\Users\\gbelenkiy\\source\\repos\\TSO\\node_modules\\loader-runner\\lib\\LoaderRunner.js:202:4)\n    at Storage.finished (C:\\Users\\gbelenkiy\\source\\repos\\TSO\\node_modules\\enhanced-resolve\\lib\\CachedInputFileSystem.js:43:16)\n    at provider (C:\\Users\\gbelenkiy\\source\\repos\\TSO\\node_modules\\enhanced-resolve\\lib\\CachedInputFileSystem.js:79:9)\n    at C:\\Users\\gbelenkiy\\source\\repos\\TSO\\node_modules\\graceful-fs\\graceful-fs.js:90:16\n    at FSReqWrap.readFileAfterClose [as oncomplete] (internal/fs/read_file_context.js:53:3)',
  [message]: 'Cannot call write after a stream was destroyed',
  [Symbol(kExpandStackSymbol)]:
   { [Function: bound enhanceStackTrace] [length]: 0, [name]: 'bound enhanceStackTrace' } }

[alfonsogarciacaro]

Hmm, the error seems to happen here. So for some reason the stdin stream of the subprocess (the dotnet Fable) has been closed. You can try running a local build and add printfn statement to try to see when the stream is closed. For example, right after writing a JSON response.

[grishace]

@alfonsogarciacaro I've added extra logging to the spot you've mentioned and got bunch of output lines during initial compilation (something like

*** {"path":"C:\\Users\\gbelenkiy\\source\\repos\\TSO\\.fable\\Fulma.1.0.0\\Elements\\Table.fs","rootDir":"C:\\Users\\gbelenkiy\\source\\repos\\TSO","define":["DEBUG"],"typedArrays":true,"clampByteArrays":false,"extra":{}}
fable: Compiled .fable\Fulma.1.0.0\Elements\Table.fs

but no extra logging when I change a file:

i ｢wdm｣: Compiling...
× ｢wdm｣: Hash: 8a53da3842158c132f7e
Version: webpack 4.16.5
Time: 733ms
Built at: 12/13/2018 2:59:15 PM
                                     Asset       Size  Chunks                    Chunk Names
                                    app.js   1.02 MiB     app  [emitted]  [big]  app
    app.2ed5aba7cce045cd20f4.hot-update.js   2.67 KiB     app  [emitted]         app
      2ed5aba7cce045cd20f4.hot-update.json   45 bytes          [emitted]
                                app.js.map   1.13 MiB     app  [emitted]         app
app.2ed5aba7cce045cd20f4.hot-update.js.map  115 bytes     app  [emitted]         app
 + 2 hidden assets
Entrypoint app [big] = vendors.js vendors.js.map app.js app.2ed5aba7cce045cd20f4.hot-update.js app.js.map app.2ed5aba7cce045cd20f4.hot-update.js.map
[./src/TSO.Operations/TSO.Operations.SIMON.Web/Client/App.fs] 341 bytes {app} [built] [failed] [1 error]
    + 280 hidden modules

ERROR in ./src/TSO.Operations/TSO.Operations.SIMON.Web/Client/App.fs
Module build failed (from ./node_modules/fable-loader/index.js):
Error: Cannot call write after a stream was destroyed
    at Loader.getCompiler.send.then.catch.err (C:\Users\gbelenkiy\source\repos\TSO\node_modules\fable-loader\index.js:125:18)
    at process._tickCallback (internal/process/next_tick.js:68:7)
 @ ./src/TSO.Operations/TSO.Operations.SIMON.Web/Client/Client.fs 3:0-208 20:11-18 21:5-14 21:16-23 34:15-35
 @ ./src/TSO.Operations/TSO.Operations.SIMON.Web/Client/Client.fsproj
 @ multi (webpack)-dev-server/client?http://localhost:8080 (webpack)/hot/dev-server.js ./src/TSO.Operations/TSO.Operations.SIMON.Web/Client/Client.fsproj
i ｢wdm｣: Failed to compile.

[alfonsogarciacaro]

Is it possible to remove most of the code, just leave the infrastructure and zip it so I can give a try? Also, which node version are you using?

[alfonsogarciacaro]

Ok, I'm finally able to reproduce this on Windows :) I'll try to find the cause of the problem.

[grishace]

@alfonsogarciacaro oh, it's good news! LMK if I can help with testing - already have reference to the local fable-master in my webpack.config.

From my C#ish experience it could happen if you wrap a stream in using statement (effectively disposing it when goes out of scope), but creating a closure over it, with a callback for example, inside the block. So by the time the callback is called the stream is disposed already.

[grishace]

Is it possible to remove most of the code, just leave the infrastructure and zip it so I can give a try? Also, which node version are you using?

node - v10.14.2
yarn - 1.12.3
webpack - 4.16.5 (visible in the previous messages)

[alfonsogarciacaro]

Seems the problem is fable-loader is relying on the watchMode field that it's more recent than I expected. Can you please try with latest webpack version (yarn upgrade webpack)?

[grishace]

https://github.com/webpack/webpack/releases/tag/v4.23.0

[grishace]

@alfonsogarciacaro just updated to 4.23.0 and HMR works!

[alfonsogarciacaro]

Great, thanks for confirming! We should give a proper error messages for this situation :+1:

[kunjee17]

Faced same issue and solved same issue by same solution. :)

@alfonsogarciacaro is it possible that we can put minimum dependency on read me page? We are doing similar thing for Fulma so CSS update stays in sync with Fulma update.

[alfonsogarciacaro]

@kunjee17 fable-loader 2.1.2 has webpack >= 4.23.0 as peerDependency, so when installing it, npm will raise a warning if you have a lower Webpack version instead.

[kunjee17]

@alfonsogarciacaro got it... Thanks... I did updated to latest and things are working pretty cool. 🙏