Deno: Workers should maybe be standalone subprocesses

[ry]

It would simplify our logic a lot if we treated Workers as full subprocesses. It's probably not much overhead considering Workers new a whole new isolate.

The main thing would be to facilitate message passing via IPC.

cc @bartlomieju

[kevinkassimo]

I would imagine some overhead over this decision. There are still a lot of things that could be shared more easily under the same process (module loading can be one, and if we were to move more things to GlobalState), and one part that we haven’t implemented is SharedArrayBuffer support across workers with atomics. It might be harder for us if we were to go multiprocessing instead of userland multithreading.

I don’t really know how much quantitative cost it is to allocate a new isolate though

[bartlomieju]

I'm not sure about treating Workers as full subprocesses. To give somme context to other people; Ry and I discussed idea of having Worker (and consequently Isolate) being a Future, one of the ideas was that Isolate should have it's dedicated thread.

This discussion arose after the transition to new Futures 0.3 API (#3358) - we're facing following problem: we have some ops that are synchronous but their implementation is fully asynchronous
and we're forced to block on those ops (eg. op_create_worker, blocking on execute_mod_async). That leads to situation when we have "nested" blocking calls - execute_mod_async might be downloading/compiling modules and in turn it also blocks (because TS compiler requires sync file fetching). This nested blocking calls are not a good idea, we managed to get it to work with new futures API but at the cost of a few hacks (see benchmarks) - trying to nest block_on result in panic.

My idea for this solution was to have dedicated thread per worker/isolate + thread pool for actual ops. So TS compiler would have its own dedicated thread that it can block freely, same situation for "main" worker and all other web workers - each of them has its own thread they they can block for ops that need it. All of the op work would be done on actual threadpool - that probably requires workers to have channels they use to send ops to threadpool and receive results. No idea how that would work with "current thread" option.

I will iterate a bit more on this idea and get back to you.

[bartlomieju]

Deno switched to worker-per-thread model recently, I guess this issue might be closed now.