Sdk: Isolate leaking

[therenca]

I spawned a few isolates from main and monitored my memory usage using top and htop. The isolates are taking up too much RAM. The isolates are sleeping for a specified timeout

[a-siva]

/cc @aam

[a-siva]

@therenca can you give us more specifics on your app, some sample code of what the isolates were doing after being spawned would be helpful. Also if you can post actual memory usage numbers that would be helpful .

[aam]

Thank you for bringing this up, we are aware of high cost of spawning an isolate at this point(higher in JIT, lower in AOT). Even if it comes with advantages of having isolates heaps completely isolated(therefore one isolates memory usage and GC won't affect other isolates) and simpler implementation, it limits how isolates can be used in a real world(at present isolates have to be long-living workers). There is https://github.com/dart-lang/sdk/issues/36097 where we are tracking our progress towards leaner isolates. Feel free to upvote that issue.

[a-siva]

There are two aspects here

• cost of spawning an isolate
• memory leak
  The title in the issue seems to suggest a leak, so we should be sure there is no leak. More specifics from the app would be helpful in determining if there is a leak

[maks]

I have a small test that I've been working on spawning large numbers of Isolates using the current dev channel Dart SDK (Dart SDK version: 2.12.0-110.0.dev (dev) (Unknown timestamp) on "linux_x64") and I'm not seeing anything that looks like a memory leak after spawning upto 10k Isolates.

Indeed using the work done in #36097 I'm seeing an approx 5-10x improvement in time to spawn new Isolates and around 2x improvement in memory usage, both in AOT of course, so big thanks to @mkustermann and others working on this!

[mraleph]

@maks are you running with --enable-isolate-groups? We would expect that memory usage is improved by orders of magnitude and not just 2x.

[mkustermann]

Indeed using the work done in #36097 I'm seeing an approx 5-10x improvement in time to spawn new Isolates and around 2x
improvement in memory usage, both in AOT of course, so big thanks to @mkustermann and others working on this!

Very happy to hear, @maks! We're working on finishing the JIT support atm. Once everything is done, we hope to be able to enable this by-default. Stay tuned.

@maks are you running with --enable-isolate-groups? We would expect that memory usage is improved by orders
of magnitude and not just 2x.

Maybe to make this statement a bit more precise: We expect 10x+ memory reduction per additional isolated spawned, not taking into account application memory usage (i.e. difference between before Isolate.spawn(<entry>, ...) and after the spawn in <entry>, e.g. measuring using ProcessInfo.currentRss). Of course the memory the application itself allocates (and needs) we won't be able shrink.

[maks]

@mraleph Thanks for taking a look at this so quickly! Yes I tried to follow what I saw was done in the tests for commits attached to 36097 so I used the --enable-isolate-groups flag to test for improvements.

I've quickly pushed up the VERY basic test I did: https://github.com/maks/isolate_tests

I expect I've done something wrong as I also would have expected a huge difference in mem usage given that the new work essentially just reuses almost everything for each new isolate from what I understand and given my test just prints to stdout in the Isolate I guess Isolates mem is also being GC'd as the test runs? But still the memory usage is tiny compared to what I was initially expecting for 10k of isolates which makes me think I'm doing something wrong...
My plan is to do some basic benchmarking with using websockets for an article I'm writing so anyfeedback on this most appreciated, though I don't want to hijack this issue as I only came here to provide feedback that I wasn't seeing any mem leaks as originally reported here, at least not with current Dart dev channel as its unclear what version the original issue reporter was using.

[maks]

Actually sorry to spam this issue one more time but @mraleph and @mkustermann is there a preferred place by Dart team where I can go to ask questions and discuss Dart Isolate perf, is it dart channel on Flutter Discord, Gitter or somewhere else?

[mkustermann]

I've quickly pushed up the VERY basic test I did: https://github.com/maks/isolate_tests

Thanks for the example, @maks!

The reason for not seeing big memory reduction is because those isolates die immediately, thereby making isolates die before more isolates are created. If you keep all isolates alive to the end, you'll see the memory impact, please give https://github.com/maks/isolate_tests/pull/1/files a try ;-)

Actually sorry to spam this issue one more time but @mraleph and @mkustermann is there a preferred place by Dart team
where I can go to ask questions and discuss Dart Isolate perf, is it dart channel on Flutter Discord, Gitter or somewhere else?

We don't have anything isolate specific atm - the current lightweight isolate work is still in-progress, so it wasn't even announced yet. You can reach out via email directly if you want. If others would benefit from the answer, you can consider using StackOverflow and making us aware of the question there.