Sdk: Lightweight Isolates & Faster isolate communication

[mkustermann]

This issue is tracking our ongoing work on making isolates more lightweight and improving communication between them. This work will mostly focus on isolates which were spawned from the same source.

As part of this work we will investigate a number of approaches, including:

• [ ] Introducing support for transitively freezing an object graph (i.e. making it immutable)
• [ ] Sharing of immutable messages across isolates
• [ ] Faster sending of mutable messages across isolates
• [ ] Sharing of program structures (e.g. libraries, classes, functions, fields), JITed code, ...
• [ ] Allowing multiple mutator threads to run in parallel the same code (yet using separate global state)

We would like to implement this in several stages, here's our current progress:

• [x] [done] Change embedder API to allow creation and initialization of isolate groups
• [x] [done] Add support for standalone/flutter embedders to initialize isolate-group spawnees
• [x] [done] Enable multiple isolates to share one heap.
• [x] [done] Split up existing field objects into mutable and immutable part.
• [x] [done] Share program structure in AOT mode.
• [x] [done] Implement SendPort.sendAndExit to allow close to 0-cost sending of messages
• [ ] [in-progress] Migrate embedders to take advantage of isolate groups.
• [ ] Make installing code faster by avoiding safepointing if possible
• [ ] Start sharing JITed code: Will require making major parts of VM re-entrant from multiple mutators

Subtasks for AOT enable by-default

• [ ] Ensure all existing embedders that use Isolate.spawn() API have been augmented to supply Dart_InitializeParams.initialize_isolate callback with correct implementation.

Subtasks for JIT

• [ ] Ensure Resolver code is re-entrant by adding appropriate locks (already now BG compiler is doing method resolution while mutator can change modify the data structures)
• [x] Reduce RunWithMutatorsStopped in SwitchableCall miss handler to only around patching the tuple in the pool (and remove hack in symbol table)
• [ ] Enable class finalization on BG compiler by adding appropriate locking: Allows us to test this in normal JIT mode and is pre-requisite for many mutator threads
• [ ] Make shared BG compiler queue that multiple mutators can use (i.e. move BG compiler functionality from isolate to the group)
• [ ] Make debugging of multiple isolates work.
• [ ] Look into remaining changes needed, also for hot-reload.

NOTICE

This work will require substantial restructuring of the Dart VM internals and is therefore relatively risky and giving a concrete timeline is not possible - that being said we expect to make good progress on this in the coming months.

The (old) isolates will continue to be available as they were before and we will try to ensure to not regress any existing code.

What the actual improvements in terms of isolate spawning time, parallelization as well as communications will be can only really be seen once the implementation is done. That being said, we can get numbers for some features in an intermediate stage.

[mkustermann]

Thread local allocation buffers (TLABs) have been restored in df34f65d90, 5737b452b and the background compiler has been changed to allow allocating into new space in fb88c983e7.

[mkustermann]

A prototype for faster sending of mutable messages across isolates has been made in cl/92280, which might allow 2-3x faster serialization and close to O(1) deserialization time.

[mkustermann]

A prototype for transferrable byte arrays is available in cl/86640

[mkustermann]

Before we can start working on heap unification we have to change the isolate spawning functionality which currently just calls out to the embedder and assumes the embedder happens to load the exact same code into the isolate. See https://github.com/dart-lang/sdk/issues/36648.

[aam]

transferable typeddata landed in https://dart-review.googlesource.com/c/sdk/+/99623

[xster]

Eagerly looking for the outcome of this feature request :D

Considering you guys seem to have made a lot of progress on this already, do you have a design doc outlining the general intended outcome or usage of the new paradigm?

[shafqatevo]

It is understood that lot more work is still to be done in this direction. Is it still possible to share how lightweight isolates may become? In terms of what parameters the lightweightness is aimed?

There are some references in the web that Dart isolates have a memory overhead of 2MB, which is more in the range of OS thread overhead. Is the goal of this work to make isolates lightweight like green threads? There are also references in the web that the original design goal of Dart isolates was to follow actor model - is that still the direction?

Isolates are not for lightweight concurrency as such. But for green threads or lightweight process concurrency, comparisons would be memory overheads of Kotlin’s coroutines (“a few dozen bytes”), Akka actors (~300 bytes), Pony actors (256 bytes), Erlang processes (309 words = 309x8 bytes in 64-bit architectures).