Emscripten: Wasm memory cannot be grown in pthreads builds

[dschuff]

The current pthreads code has no support for growing the memory (https://github.com/kripken/emscripten/blob/incoming/src/library.js#L470). In wasm, the memory itself can be grown from any thread. However growing the memory causes all of the JS heap views to become detached (it's why we call updateGlobalBufferViews after we grow). This means we will probably have to proxy the call to the main thread.

[kripken]

Discussing this more in person, what I'd suggest is

• We keep disallowing growth+pthreads builds for now.
• We move to a model of only allowing wasm in Workers. That is, no JS views of typed arrays etc., which can cause problems here. JS glue will only be on the main thread, period.
  
  
  
  • Disallowing JS will be possible when wasm has GC support and hence access to JS APIs, or when wasm adds non-Worker threading.
  
  
• We eventually remove --proxy-to-worker mode, which is a single-threaded mode that is not needed once multithreaded builds are normal on the web.

[dschuff]

OK, I talked to @dtig and got some clarification. Currently growing a shared wasm memory just doesn't work at all in V8 yet. But the intention in the spec is that the views of a shared memory do not detach (this is also the case already for SharedArrayBuffers in JS). The intent is to avoid exactly this problem. So for now we probably should just keep disallowing growth, but eventually we should be able to just enable it and not worry. We can probably also avoid updating the views in the shared case as an optimization.

[Auron52]

@dschuff: What is the plan going forward for this? This is a showstopper for using Wasm threads in my product.

[dschuff]

I guess just wait until some browser supports growing a shared memory, and then enable it in emscripten :)

I looked and didn't see a V8 bug specifically for that feature (there's just https://bugs.chromium.org/p/v8/issues/detail?id=6532 for all the atomics stuff). I don't think Firefox supports but not 100% sure.

[Auron52]

@dschuff: Should I be submitting a V8 bug for this, then? The whole thing is kind of a pain because PNaCl (which is soon to be removed from Chrome) has supported this feature for ever with no problem. If the removal of PNaCl proceeds without addressing this issue we will be left without a threaded implementation... :(

[dschuff]

Memory growth support is required by the wasm atomics spec, so it's definitely the plan to support it ASAP. And wasm threads support is definitely being taken into account with PNaCl deprecation too (that's why we've delayed it as long as we have already).

[dtig]

@Auron52, I'm currently in the process of making sure this works for v8. I've filed a bug - https://bugs.chromium.org/p/v8/issues/detail?id=8564 so it's easier to track when this is done outside of the atomics bug.

[Auron52]

Thanks @dtig!

[wffurr]

This is causing a problem for us on https://canvas.apps.chrome/

If we allocate more than the 128 MB we already do, then we sometimes fail to start up on low memory devices, e.g. Acer Dru. But without more memory, we run out on complex drawings.

[hostilefork]

I was looking into what the plans were for allowing memory growth in pthreads builds, and noticed this bit that @kripken said:

We move to a model of only allowing wasm in Workers. That is, no JS views of typed arrays etc., which can cause problems here. JS glue will only be on the main thread, period.

I'm confused as to what "only allowing wasm in Workers" plus "JS glue will only be on the main thread" means. What's "glue"? (Presumably not cwraps, because that works with the wasm and would thus be on the worker thread.)

But secondly, why would disallowing wasm calls in a pthread build from multiple threads be viable?
I'm building something that is a language interpreter which can be called from either the main thread or the worker--having that them both be able to call the cwrapped routines is critical. The main thread also needs to be able to poke in and request cancellation of long operations. I can't imagine that this would be a unique desire...

[kripken]

@hostilefork The current status is that we can close this issue, as it was fixed by #8365. There may be some slowness (see that PR) when accessing memory from JS, but it should work. Depending on data, we may try to optimize that further. I think those are the only current plans in this space.

So that is all in the same model of using web workers. The wasm in each worker can call into JS there, but the JS isn't shared/multithreaded itself of course.