Trio: big picture & beyond Python

[belm0]

I see the benefit of the nursery and cancellation control structures presented by Nathaniel. Also Trio's stance of avoiding threading, along with merging scheduling and cancellation concepts makes sense. Relegating all concurrency interaction to cooperative, explicit switching within a single thread seems to be a clear sweet spot for Python given the GIL and CPU-bound threading issues.

What I'm trying to reconcile is how to apply these concepts elsewhere. For example, Lua doesn't have a GIL issue. So do we borrow the nursery and cancellation, but allow full threading (and associated headaches) to enable scaling to multiple cores? Or would limiting preemptive multitasking to the equivalent of run_sync_in_worker_thread() be sufficient for 99% of use cases? Or how about allowing individual threads to each run their own Trio event loop with all state thread-local, and somehow have apps carefully dispatch work to each thread?

Another question: how well can a library like Trio be implemented in languages lacking colored function support? (E.g. Lua or D language)

[goodboy]

Or how about allowing individual threads to each run their own Trio event loop with all state thread-local, and somehow have apps carefully dispatch work to each thread?

@belm0 FWIW I have this working quite well under a minimal actor model here. Though no larger uses (yet) to prove these concepts can scale, the adherence to trionic principles is demonstrating much easier to understand failure modes so far. tractor also uses multiprocessing so there is no GIL constraint and the thread-local state is implicit.

I personally am very interesting to see how trionic concepts can adapted in new languages and threading models. Given this recent project, I'm starting to think the lifetime and cancellation constraints can be applied to distributed processes on a much grander scale (under certain caveats) with good results.

You should give it a shot in lua and tell us your results!
I for one would be very interested as I think lua is a lovely minimalist language.

Another question: how well can a library like Trio be implemented in languages lacking colored function support? (E.g. Lua or D language)

IIRC the coroutine support is there and could be paired with a with statement solution and allow for designing something quite similar to trio. The lack of GIL should avoid the need to go full message passing between threads (if that's what you're after) - a fast queue could do the trick?

[njsmith]

@belm0 Yeah, good question! I actually think most of Trio's ideas would translate very well to other languages, and would love to see that happen. Of course concurrency and cancellation are pretty deep features that interact a lot with other language features, so the exact details are definitely going to be different for different languages, but the core ideas I think are very general.

You asked specifically about translating to different scheduling models, so let me talk about that a bit.

Obviously any concurrency system is going to need some strategy for scheduling multiple concurrent "things" (threads or tasks or whatever you want to call them). The popular choices are:

• User-space scheduling on a single thread (like trio, javascript, ...)
• Using kernel threads for everything
• Hybrid models, e.g. maybe you have 10,000 concurrent "things" and 5 threads, and each thread runs a user-space scheduler so that all 10,000 things get their turn to run on the 5 threads (go is a notable example of this approach)

The trade-offs between these are pretty complicated. Kernel schedulers have powers that user-space schedulers don't, like preemption, and running on multiple cores; making user-space scheduling robust is quite challenging. OTOH, kernel threads have a ton of memory overhead when compared to coroutines, which is a huge problem if you want to run 10,000 things at once; also, on some systems the kernel scheduler can't handle that many threads anyway (the rumor I heard is that Linux's scheduler handles it fine, but other OSes will choke). So if you're implementing a concurrency library, this is a complicated multi-dimensional choice that will matter a lot. BUT, it mostly doesn't affect the API you expose to your users.

There are two main exceptions, where your choice might leak out and change your library's public API:

First, if you use kernel threads for everything, then your users can use regular blocking I/O, which is usually what their existing libraries already do, so that's handy. OTOH, if you do user-space scheduling (either exclusively, or as part of a hybrid model), then you have to make sure that everyone uses only your special I/O routines. This generally means people have to rewrite their libraries, which is a hassle.

The other is function color. This is a bit complicated; it's addressing a few different things, all of which are only relevant in the user-space scheduling world (either single-threaded or hybrid).

One reason for function color is that your compiler might appreciate having some warning ahead of time whether a particular function call can invoke the scheduler or not, because it might want to lay out the stack differently or something. AFAIK both Lua and D have their own ways to deal with this though, so that's fine, not your problem.

Another is that in some systems with cooperative scheduling, it's possible to accidentally create a tight loop that doesn't contain any schedule points, and which can then starve out other threads. Go doesn't have this problem, because their compiler automatically inserts a schedule point at every loop iteration (among other places), but in other languages this can be an issue that your users are forced to deal with manually. Function color – specifically the part where call sites are marked – at least makes it visible where the schedule points are, and gives your users a better chance of figuring this out. OTOH if you don't have this it's not the end of the world; maybe you can invest in good tools for debugging scheduler starvation or something instead.

And finally, in the single-threaded case specifically, function color has one more advantage: since everything is actually single-threaded, being able to see where the schedule points are makes it much easier to reason about and debug race conditions. (Of course, there are much better options for avoiding race conditions, including shared-nothing architectures like Erlang or ownership tracking like Rust, but if you're not working in a language with those features, then hey, you make do with what you've got.) Again, though, if you don't have function color, then your users may have to think harder and hit more bugs, but it's probably not going to sink your design. People use golang for stuff, and it's classic fully-preemptive shared-everything concurrency.

---

Okay, that was the background, now how does this interact with Trio's two big ideas: cancel scopes and nurseries?

Nurseries actually don't care about this stuff at all. They basically work the same no matter which scheduling model you choose. Their syntax is going to be different in different languages; e.g., in Lua you'd probably do something like:

with_nursery(function (nursery)
  nursery.start_soon(...)
  nursery.start_soon(...)
end)

But the idea is basically equivalent. Nurseries do depend on cancel scopes though.

And cancel scopes are mostly orthogonal to your choice of scheduling model, but they overlap in two places.

First, if you want cancellation to work, you need to make sure that every blocking I/O operation has cancellation support implemented. In practice this means you need to make sure everyone uses your special I/O routines, and everyone has to rewrite their libraries to do that. So this has some synergy with user-space scheduling models: if you're going to have to go to the trouble of migrating to a whole new set of I/O routines, it's nice to get cancellation and scalable user-space scheduling out of the deal.

And second, you have to somehow communicate to users where the cancel points are, because handling cancellation usually requires some thought. In Trio, since we're using function color to make schedule points visible anyway, we re-use it to make cancel points visible. But if you don't have function color, then there are other options that could work too. If your language has a type system for describing what errors a function can raise, that might work. Or worst case, just make sure to make it clear in the docs. Maybe invest in tooling to help users track down places where they run too long without passing a cancel point, and to write tests to exercise all possible cancel points?

Does that help? These are just some ideas – you won't really know what the best design is until you've really dug into exactly what the trade-offs in your language are, and obviously I haven't done that, so I'm probably missing stuff. But hopefully it helps you get started at least :-). And if you do end up implementing something Trio-inspired libraries in other languages, definitely let us know!

[jab]

As mentioned in https://twitter.com/vorpalsmith/status/1040777480473214976

apparently Kotlin is redoing its whole coroutine API based on ideas from Trio

https://github.com/Kotlin/kotlinx.coroutines/issues/410

🎉

[imrn]

Can you provide some technical details?

[oremanj]

Closing this in favor of the "structured concurrency" topic on the Discourse forum, since there's no action item.