Binaryen: wasm-opt crashes

[droundy]

I'm having wasm-opt crash, with an insane amount of error message... it seems to be outputting my entire wasm file.

The error text (> 4 megabytes) is available at https://api.travis-ci.org/v3/job/712739691/log.txt or https://travis-ci.org/github/droundy/paradigms/jobs/712739691

The most relevant bit is probably:

[wasm-validator error in module] unexpected true: Exported global cannot be mutable, on 
global$0

This is with code generated with rust (as you can see with the travis-ci link above).

[kripken]

What version of binaryen is that with? I think we stopped printing the entire module on error a while ago.

[kripken]

That error might just be that the module uses non-MVP features. Does -all (enable all features) fix it?

If not, please attach the wasm file and I can take a look.

[droundy]

Sounds like it might be a bug in wasm-pack, see https://github.com/rustwasm/wasm-pack/issues/886

[Urhengulas]

That error might just be that the module uses non-MVP features. Does -all (enable all features) fix it?

If not, please attach the wasm file and I can take a look.

@kripken This seems to do the trick!

I was running:

$ wasm-pack build
# error
# ...omitted
Fatal: error in validating input
Error: failed to execute `wasm-opt`: exited with exit code: 1
  full command: "~/.cache/.wasm-pack/wasm-opt-4d7a65327e9363b7/wasm-opt" "$REPO/pkg/repo_bg.wasm" "-o" "$REPO/pkg/repo_bg-opt.wasm" "-O"
To disable `wasm-opt`, add `wasm-opt = false` to your package metadata in your `Cargo.toml`.

$ ~/.cache/.wasm-pack/wasm-opt-4d7a65327e9363b7/wasm-opt pkg/repo_bg.wasm -o pkg/repo_bg-opt.wasm -O
[wasm-validator error in module] unexpected true: Exported global cannot be mutable, on global$0
Fatal: error in validating input

$ ~/.cache/.wasm-pack/wasm-opt-4d7a65327e9363b7/wasm-opt pkg/repo_bg.wasm -o pkg/repo_bg-opt.wasm -O -all
# works just fine!

version

My wasm-opt version is 90 and is automatically installed by wasm-pack:

~/.cache/.wasm-pack/wasm-opt-4d7a65327e9363b7/wasm-opt --version                    
wasm-opt version_90

[RReverser]

Unfortunately, wasm-pack is just stuck on a bit old version of wasm-opt. They haven't had a release for some time, but if they do, that should fix it.

[Urhengulas]

Unfortunately, wasm-pack is just stuck on a bit old version of wasm-opt. They haven't had a release for some time, but if they do, that should fix it.

@RReverser

I've build wasm-opt from master and optimized my .wasm with it. Though this works fine (with -all set) I get a new error on application startup:

Failed to compile

./node_modules/rav1e/rav1e_js_bg.wasm
Module parse failed: Unexpected section: 0xc
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
Error: Unexpected section: 0xc

Do you have an idea what could be the cause of that? (Everything works fine with the unoptimized .wasm)

[kripken]

Section 12 is the data count section I believe, which is part of bulk memory. Running with -all enables that feature, and then it will emit that section, which older VMs might not support.

As a solution you may need to only enable the specific features wasm-pack needs, which based on the error above, might be --enable-mutable-globals. But I think it would be optimal for wasm-pack to use the features section so this works automatically. Or maybe there's a way to build with wasm-pack that ensures only MVP features are used?

cc @tlively

[tlively]

It looks like something in the wasm-pack toolchain either introduced a new use of mutable-globals without updating the target features section or stripped the target features section. Otherwise wasm-opt would have detected all the necessary features automatically from that section. @ashleygwilliams, does that sound possible to you?

[Urhengulas]

As a solution you may need to only enable the specific features wasm-pack needs, which based on the error above, might be --enable-mutable-globals. But I think it would be optimal for wasm-pack to use the features section so this works automatically. Or maybe there's a way to build with wasm-pack that ensures only MVP features are used?

@kripken This works (this time actually :smile:). Thank you!

---

And a small question: What do you mean with "it would be optimal for wasm-pack to use the features section"? Is there a place in the .wasm-binary where you can indicate which features are used?

[tlively]

@Urhengulas, yes, see https://github.com/WebAssembly/tool-conventions/blob/master/Linking.md#target-features-section. LLVM emits this section into wasm object files, and wasm-ld (i.e. lld) emits it into the linked binary so that postprocessing tools can use it.

[jacogr]

I also use wasm-pack. With --enable-mutable-globals can get wasm-opt working, but then have issues with the wasm2jstooling which fails with the same issue. (Passing -all there generates an output, but that fails my tests)

[kripken]

@jacogr - the wasm2js problem may be something else, please open another issue with a testcase if you can.

[bryanperris]

Using --enable-mutable-globals with wasm2js will cause some broken javascript to be generated in the case where an export is missing: __wbindgen_export_2

Webpack complains that the export is missing.

To be more specific, its not the flag itself that breaks the output of wasm2js, its the flag + using wasm-bindgen > 2.62.

[kripken]

@bryanperris I'm not sure offhand what could cause such a bug, so a testcase would be very useful here.

[bryanperris]

The project I am working on is under a NDA, but if I get some time I can try to reproduce the issue in a simple example.

[bryanperris]

Here is a sample that reproduces this issue: https://github.com/bryanperris/wasm2js-failure-sample

After everything builds, webpack generates these warnings which does crash the application

WARNING in ./mylib/pkg/index_bg.js 161:27-51
"export '__wbindgen_export_2' (imported as 'wasm') was not found in './index_bg.wasm.js'
 @ ./mylib/pkg/index.js
 @ ./src/index.ts

WARNING in ./mylib/pkg/index_bg.js 162:12-36
"export '__wbindgen_export_2' (imported as 'wasm') was not found in './index_bg.wasm.js'
 @ ./mylib/pkg/index.js
 @ ./src/index.ts

WARNING in ./mylib/pkg/index_bg.js 168:12-36
"export '__wbindgen_export_2' (imported as 'wasm') was not found in './index_bg.wasm.js'
 @ ./mylib/pkg/index.js
 @ ./src/index.ts

If you use wasm-bindgen = "=0.2.62", then run build.sh, no issues.

[MaxGraey]

btw why you use specific only to Emscripten and AssemblyScript passes here while your language is Rust?

If you use wasm-bindgen = "=0.2.62", then run build.sh, no issues.

As I know this some problem with wasm-bindgen. Probably better primarily ask about this issue here

[bryanperris]

We moved a lot of math from typescript into rust to generate web assembly for performance reasons but we cannot publish the wasm module itself. The production server is way beyond our control, only a single JS bundle file can be published to this server. I was forced to manually use wasm2js to produce javascript that be can be bundled in. The performance of the converted wasm module still is as a good as the wasm module itself. This issue came up unexpectedly when one day I randomly removed the Cargo.lock file.

[RReverser]

We moved a lot of math from typescript into rust to generate web assembly for performance reasons but we cannot publish the wasm module itself. The production server is way beyond our control, only a single JS bundle file can be published to this server. I was forced to manually use wasm2js to produce javascript that be can be bundled in. The performance of the converted wasm module still is as a good as the wasm module itself. This issue came up unexpectedly when one day I randomly removed the Cargo.lock file.

Still, you haven't answered @MaxGraey's question:

btw why you use specific only to Emscripten and AssemblyScript passes here while your language is Rust?

You're running AssemblyScript and Emscripten postprocessing on code that wasn't produced by them. This is almost guaranteed to break those modules and make them incompatible with wasm-bindgen generated JS (which might be the reason you're seeing those issues).

More generally, you also shouldn't be running wasm-opt yourself - wasm-pack already does this for you. You might want to just configure its options in Cargo.toml as per https://rustwasm.github.io/wasm-pack/book/cargo-toml-configuration.html.

[bryanperris]

I didn't fully understand question, I can only explain what we have been trying to do to allow us execute the rust library in a limited sandbox environmnet. I googled many things but a lot this has been trial and error to make all of this to work on the product device. In my perspective, all I know is that the wasm-pack webpack plugin compiles our Rust source into a WebAssembly file which works fine. However in my specific case, I am forced to transform it into Javascript for the bundle file. Is there something I should be doing to directly compile Rust into Javascript instead? I am only calling wasm2js in my own script which is the solution that has been working for me until wasm-bindgen 2.63+. I do set a configuration to always compile rust-wasm in optimized mode through because the debug version of the wasm module is way too big for the embedded device. Trust me the project I am working on is an unusual one; The backend runtime service is a chromium tab that runs our app based on a specific condition set by the consumer, we have no control of the sandbox except execute some javascript based on user intents. We are forced to resort to some unusual methods here.

[tlively]

Specifically the question is about these two flags: --post-assemblyscript-finalize --post-emscripten. Both of these passes modify the module in ways you probably don't want if you're not using AssemblyScript or Emscripten.

[bryanperris]

Thank you for that info, I really had no idea, I was only trying to optimize the js output as best as I can. I took out those 2 flags, and I still get the issue from wasm2js.