Cats-effect: Proposal: Don't default to ExecutionContext.global in IOApp

[djspiewak]

So IOApp appears to default to a ContextShift[IO] based on ExecutionContext.global. This is a semi-sane default on the surface and it aligns well with other Scala applications, but it seems to cause a ton of problems in practice due to the fact that it's backed by a fork-join pool. I've observed absolutely horrible things like over 250 threads sitting in that pool (on an 8 core machine!), spawned by fork-join in response to long-lived CPU-bound tasks.

Fork-Join appears to be optimized with two assumptions in mind. First, it assumes that people are lazy-moding their pool usage and dropping blocking IO in the middle of their compute pool. This assumption triggers pool growth well beyond num CPUs when threads aren't being returned to the pool in a timely manner, even when the reason those threads aren't being returned is simply long-lived CPU-bound computations. Second, it seems to assume that people are tuning their applications for latency rather than throughput, and thus frequently returning control back to the thread pool even when CPU-bound.

Both of these assumptions mesh very well with typical use of scala.concurrent.Future, which yields on every map and is frequently used for blocking IO (mostly because many people simply don't realize this is bad). However, due to some very intentional and very carefully thought out API design, cats-effect (and all frameworks within its ecosystem) tends to do a very good job of pushing users to keep their blocking IO off the main pool, and also tends to bias towards throughput rather than latency (due to explicit rather than implicit yield semantics). This makes fork-join in general a very, very poor fit for cats-effect, fundamentally.

And this is all ignoring the perils of the global pool in general and the fact that many frameworks abuse it!

All of this, in general, suggests that we would be better off creating our own fixed thread pool in IOApp. We experimented with this recently at SlamData and saw our context shift time massively decrease as soon as we got off of fork-join, and we're not doing anything weird at all in terms of our pool usage (additionally, none of our dependencies are using the global pool at all, as verified in thread dumps, so it was definitely just our own compute-bound pool usage that was triggering this behavior).

Anyway, that's my proposal and the rationale. I just really, really don't think it's a good idea for IOApp to be defaulting to fork-join anything, since I don't believe that any app written in the cats-effect ecosystem is going to behave particularly well on fork-join for fundamental reasons.

[rossabaker]

I agree with this in theory. I've had a hard time producing breathtaking differences in practice, but I'm :+1: if people do see a difference.

If we do proceed, I think this is a candidate for 2.0 rather than 1.3.