Design: Is exporting and importing native code across global execution contexts within the scope of WebAssembly design?

[guest271314]

WindowOrWorkerGlobalScope.fetch() is native code in the browser in window and Worker execution contexts.

Running fetch and Response, respectively, in the console at Chromium logs

ƒ fetch() { [native code] }

and Response

ƒ Response() { [native code] }

Response and fetch are not defined in AudioWorkletGlobalScope execution context, in brief see https://github.com/WebAudio/web-audio-api-v2/issues/79.

import statement and WebAssembly are available in AudioWorkletGlobalScope.

Use case:

Implementing an infinite media stream from arbitrary user input in AudioWorkletGlobalScope, media broadcast channels (both audio and video, using Web Audio API clock currentTime to synchronize video as image and, or bitstream or serialized video content) without the cost of transferring data across execution contexts via MessageChannel.

Is it possible to compile native code shipped with the browser, for example, Response, fetch exportable from window or Worker execution context to WebAssembly required format for usage in a different execution context, in this case AudioWorkletGlobalScope?

[guest271314]

WebAssembly.compileNativeCode(['Request', 'fetch']) => export {_Request, _fetch} => import {_Request, _fetch}

[binji]

This functionality is out of scope for WebAssembly, yes. WebAssembly provides no way to express native platform functionality, just computation.

For your specific use case, you may be able to take advantage of SharedArrayBuffer to reduce the cost of communicating between contexts w/o MessageChannel.

[guest271314]

For your specific use case, you may be able to take advantage of SharedArrayBuffer to reduce the cost of communicating between contexts w/o MessageChannel.

Not sure how SharedArrayBuffer could be used in the described use case without using MessageChannel?

Currently transferring one ReadableStream to AudioWorkletProcessor (with --enable-experimental-web-platform-features flag at Chromium) instead of hundreds of thousands of Float32Arrays.

The goal is to not transfer anything, to perform the necessary task in AudioWorkletGlobalScope.

Asked here to determine the limitations of WebAssembly design, which is not simple to implement at 32-bit architectures, even after downloading all of the wasm-bindgen, et al. that is required. Found it simpler to just use inotify-tools and Native File System to execute the locally installed binary directly, then get result as a file https://github.com/guest271314/captureSystemAudio, instead of trying to compile Emscripten or use WebAssembly and re-compile code again to do the same task.

[guest271314]

@binji Does this Rust code https://github.com/phodal/rust-wasm-d3js-sample actually import native alert and prompt?

[guest271314]

@binji

This functionality is out of scope for WebAssembly, yes. WebAssembly provides no way to express native platform functionality, just computation.

Does this https://rustwasm.github.io/wasm-bindgen/examples/fetch.html not provide a means to use fetch() in AudioWorkletGlobalScope?

[binji]

@guest271314 In all of these cases, WebAssembly is importing the JavaScript function, not the native functionality. When you instantiate the module, you can replace the import with your own function, and the wasm module will have no way to access the underlying alert/prompt/etc.

[guest271314]

@binji Ok. Have been experimenting the past few days attempting to use import across execution contexts by several means. Have not tried SharedArrayBuffer and Atomics yet. Currently the input am testing is a 291MB WAV file, and a version with a a single file split into parts. Frankly, am not sure that use cases outside of just using an AudioBufferSourceNode were considered in the design pattern. Perhaps MessagePort is the best that had at that point, though transferring hundreds of thousands to millions of Float32Arrays through that means invariably results in observable glitches and gaps in playback. Using a transferable stream significantly reduces the glitches and gaps. However, Firefox does not implement transferable streams.

Basically, there is no way to disambiguate glitches and gaps in playback where the input is a "real-time" stream from one or more fetch() calls with the ReadableStreams being piped to the WritableStream in Worker Issue 910471: Transferable streams don't work in AudioWorklet https://bugs.chromium.org/p/chromium/issues/detail?id=910471 from Issue 825823: [AudioWorklet] Glitches from irregular callback timing of Linux/PulseAudio issue https://bugs.chromium.org/p/chromium/issues/detail?id=825823.

Have a concept for direct communication between JavaScript and a native application, the proof-of-concept is linked in https://github.com/WebAssembly/design/issues/1350#issuecomment-640945792 at demonstrated by capturing system audio working example, and further described at the "Transferable streams don't work in AudioWorklet".

The basic concept is a MessageChannel or BroadcastChannel communication channel directly to native application already installed on the OS, a local shell. We do not need a MessageEvent, though the C++, Rust, Python, Jelly, et al. language could interpret the connection in that way, ideally the language would use the stream I/O modules, etc. of the application(s) already installed and being used natively where on the browser side we just get the readable or writable side of a TransformStream.

If post an issue have already tried to solve the problem the best know how within the bounds of given specifications and or implementations (generally requires crashing the browser and machine thousands of times per project) https://bugs.chromium.org/p/chromium/issues/detail?id=1094653. After that just try to solve the problem by any means.

[binji]

Sorry, I don't really understand. In any case, I don't think WebAssembly is going to solve your problem here, unfortunately.

[guest271314]

@binji

Sorry, I don't really understand.

One application configuration, consider

<WebAsembly|MessagePort|TransformStream> => native connecting to, opening, closing, dynamically writing from web platform context at or to local filesystem => imported module streaming data from code, ideally, zero-copy, at local filesystem back to any context

Does this https://github.com/WebAssembly/function-references/blob/master/proposals/function-references/Overview.md change anything?

[binji]

Does this https://github.com/WebAssembly/function-references/blob/master/proposals/function-references/Overview.md change anything?

That proposal is primarily an optimization and refinement of functionality that is already present in WebAssembly. It doesn't have any knew functionality w.r.t. interacting with the web platform or OS. So I don't think it will help your use case.

[guest271314]

@binji FWIW Finally composed a version using

const memory = new WebAssembly.Memory({
            initial: Math.floor(length / 65536) + 1,
            maximum: Math.floor(length / 65536) + 1,
            shared: true,
          });

that writes to the SharedArrayBuffer from ReadableStream of Response.body.getReader() during the read from fetch().

The result is audio playback from AudioWorkletProcessor without gaps or glitches.

Thank you kindly for your effort in this issue.

[guest271314]

@binji FWIW https://bugs.chromium.org/p/chromium/issues/detail?id=1115640