Wasm-bindgen: Building wasm project for node is fine, but loading the `pkg` dir fails with "Can't resolve 'fs'"

[jjpe]

Describe the Bug

(BTW: In order to verify that it wasn't an error on my end, I repeated the procedure without changing anything and still got the same result).

I just created a tiny example project, based on the tutorial in the book.
All goes well, including building the wasm with --target nodejs.

The problem is that when I try to load the pkg dir that contains the generated WASM and JavaScript from a consuming project (which was created using npm init wasm-app www), then that load fails with this output:

$ npm run build   

> [email protected] build /home/j/dev/accept/webdbg/delta-client/www
> webpack --config webpack.config.js

Hash: 3858bef569ea39a7a738
Version: webpack 4.42.0
Time: 157ms
Built at: 03/04/2020 11:42:10 PM
         Asset       Size  Chunks             Chunk Names
0.bootstrap.js   37.2 KiB       0  [emitted]  
1.bootstrap.js    3.1 KiB       1  [emitted]  
  bootstrap.js   8.73 KiB    main  [emitted]  main
    index.html  297 bytes          [emitted]  
Entrypoint main = bootstrap.js
[../pkg/delta_client.js] 1.65 KiB {1} [built]
[./bootstrap.js] 279 bytes {main} [built]
[./index.js] 97 bytes {1} [built]
    + 5 hidden modules

ERROR in ../pkg/delta_client.js
Module not found: Error: Can't resolve 'fs' in '/home/j/dev/accept/webdbg/delta-client/pkg'
 @ ../pkg/delta_client.js 63:14-27
 @ ./index.js
 @ ./bootstrap.js
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] build: `webpack --config webpack.config.js`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/j/.npm/_logs/2020-03-04T22_42_10_680Z-debug.log

Steps to Reproduce

1. Follow the tutorial until this point.
2. Then try to execute "npm run start" in the generated www dir
3. See similar output as mine

If applicable, add a link to a test case (as a zip file or link to a repository we can clone).

Expected Behavior

I expected my wasm file to be loaded

Actual Behavior

The npm run start command exits with the error log presented above.

Additional Context

OS: Ubuntu 19.10
node version: 10.17.0
npm version: 6.11.3
rust version: 1.41.1
wasm-pack version: 0.9.1

EDIT: The problem is traceable to the generated JavaScript file:

// Omitted first part of the file. What is shown is the last number of lines of the JS file.

const path = require('path').join(__dirname, 'floof_bg.wasm');
const bytes = require('fs').readFileSync(path); // <----    offending line

const wasmModule = new WebAssembly.Module(bytes);
const wasmInstance = new WebAssembly.Instance(wasmModule, imports);
wasm = wasmInstance.exports;
module.exports.__wasm = wasm;

[Pauan]

@jjpe The nodejs target is designed to be run in Node.js (hence its name). If you want to use Rust with Webpack you should use --target bundler instead.

The tutorial does not say to use nodejs, so I'm curious why you're doing that.

[jjpe]

I'm not using node itself, but I do use NPM. Thus as far as I know I need npm style packages, and thus build for NodeJS.

What I ultimately need is to mix Rust/WASM and VueJS together in the same web app.
Haven't figured out how webpack can do that tho, and not for lack of trying.

[alexcrichton]

Thanks for the report! As @Pauan pointed out though the --target nodejs output isn' intended to go into webpack, and for that you'll want --target bundler. That should also help you put everything into the same web app with webpack. You can also experiment with --target web to get a more standalone feeling for wasm

[jjpe]

I repeat: I'm not using webpack, because it always fails to build the project no matter what I do.
Currently I'm using the generated vue build script (which uses NPM) to build the client. So whatever that does is what I use.

I need the NodeJS project structure (including package.json) because that is what the environment seems to demand of me.
Bundler is what I don't want precisely because webpack fails so miserably.

So as far as I'm concerned, this bug is far from solved.

[alexcrichton]

@jjpe the error logs you gisted above with npm run build include webpack, which is why we're thinking you're running webpack. Have you adjusted how your build process works and are there new error logs?

Unfortunately it's difficult for us to help you if you're not familiar with your own build environment.

[jjpe]

Have you adjusted how your build process works

No because when I try to take control of the build process for the client, in the most positive scenario so far I get no build errors, but a nonfunctional web client nonetheless. So nonfunctional in fact that I just get a black on white default error screen rather than the GUI that's supposed to show up.

I've also grepped for any webpack configs I can find in the web client repo since if it existed, it would have been generated by VueJS or Vuetify when I created the project. But grepping doesn't yield any results, so I can't just take that config and build on it, either. It's also why I don't think I'm using webpack (unless it's used internally by the vue tools, in which case it's more or less supposed to be an implementation detail from my POV).

familiar with your own build environment.

Yeah about that. Contrary to Rust documentation in general, I find the documentation of webpack really disappointing, in that there is a lot of text but almost no information that actually helps me achieve my goal (very reminiscent of Apple's developer documentation in that regard). In other words, it has a very low signal to noise ratio. So unfortunately that's no help in improving my understanding either.

The reason I reject building with --target bundler or --target web is that I don't see how that will help me. IIUC, building for either target means I can't use the build scripts which run using NPM, but then how do I get everything bundled if the answer isn't webpack?

I'm not home atm, but when I am I'll post the contents of the project's package.json. However I do believe that one of the commands just uses the vue CLI utilities to build, and the other just runs the vue dev server.

[Pauan]

@jjpe Thus as far as I know I need npm style packages, and thus build for NodeJS.

That's not how it works. Npm is a package manager for the entire JavaScript ecosystem, it's not just used for Node, it's also used for browsers, Electron, React Native, Angular, React, Vue, etc.

So just because you're using npm doesn't mean you're running your code in Node.

The nodejs target for wasm-bindgen is only if you're running your code in Node. If you aren't, then you should be using bundler, web, or no-modules. Using bundler is correct if you're using Webpack.

What I ultimately need is to mix Rust/WASM and VueJS together in the same web app.

In that case you want to use Webpack and --target bundler, definitely not --target nodejs.

Currently I'm using the generated vue build script (which uses NPM) to build the client. So whatever that does is what I use.

Vue uses Webpack. Here is a guide on how to configure Webpack when using the Vue CLI.

I need the NodeJS project structure (including package.json) because that is what the environment seems to demand of me.

package.json is used by npm, but that isn't the same as Node, because npm is used by all sorts of things in the JavaScript ecosystem.

The reason I reject building with --target bundler or --target web is that I don't see how that will help me.

It will help you because then your code will actually work. --target nodejs cannot work in a browser, it is impossible. It will only work in Node.

In order to run your code in a browser, you must use the bundler, web, or no-modules targets.

IIUC, building for either target means I can't use the build scripts which run using NPM, but then how do I get everything bundled if the answer isn't webpack?

That is incorrect, you can use all of the targets with npm, because npm is only a package manager, it is not the same thing as Node.

The --target is for the target where you will be running your code. It is not for the target where you are compiling the code. It is important to distinguish those two things, because they are completely different.

The answer for bundling is Webpack, which is built into the Vue CLI, so the Vue CLI runs Webpack automatically.

---

Here is a step by step guide on how to use Rust + Wasm in a Vue project:

1. Create the Vue project by using npx @vue/cli create my-project

2. cd my-project

3. Use npm run serve and then open http://localhost:8080/ in your browser. This is to verify that Vue is setup and working correctly.

4. Run npm install --save-dev @wasm-tool/wasm-pack-plugin (this installs the wasm-pack plugin for Webpack)

5. Create a new vue.config.js file which contains the following:

   const path = require("path");
   const WasmPackPlugin = require("@wasm-tool/wasm-pack-plugin");
   
   module.exports = {
    configureWebpack: {
      plugins: [
        new WasmPackPlugin({
          crateDirectory: path.join(__dirname, "rust"),
          outDir: path.join(__dirname, "pkg"),
          outName: "index"
        }),
      ]
    }
   }
   

6. Now you can add your Rust code into the rust subfolder.

7. Finally you can import the Rust code by using import("../pkg/index.js"). This returns a Promise, so you'll need to await it in order to access the Rust functions.

8. You can now test that it works by using npm run serve again.

There is a lot of complexity in the JavaScript world, so it can be quite difficult to know how to set things up properly.

Unfortunately there isn't much we can do about that, since we can only control the Rust ecosystem and tooling, not the JavaScript ecosystem and tooling.

[Pauan]

P.S. Here is our official template for using Rust with Webpack. It may help you a bit.

[jjpe]

Thanks for the help! The long reply helped me get rid of some of my misconceptions about the JS ecosystem.

As a result now I have a basic wasm loaded in VueJS, but I had to get a little creative as well, since top level async doesn't seem to work in .vue files for whatever reason.

So instead I had to go old skool with a .then() call on the wasm import, and initialize the main Vue instance within that .then() call. This provides a hard guarantee that the wasm module is loaded when the Vue object is initialized.