Crystal: Proposal: merge single-msqueue into master

[bararchy]

This might raise a few eyebrows, but, I believe that merging a "slow and un-optimized" version of multi-thread support is much better then having non at all.

This will in turn allow for people to actively work and improve it without it being a hidden forgotten branch that sits dead for a year.
Having more then 4 years Crystal community background I can see this taking more then a year or two (if at all) to finish in the current format, lets be brave about it.

[bararchy]

Pinging a few people for responses and feedback

@RX14 @bew @ysbaddaden @jhass @asterite @mverzilli @bcardiff

[j8r]

I agree @bararchy . I think also it would be good to put the multi-threads feature behind a --with-threads style flag like python does, and not set by default.

[bararchy]

So.... we got 20+ community members wanting this, is there a chance to get some feedback from core devs? @asterite @bcardiff @jhass @mverzilli @RX14 @ysbaddaden

[ysbaddaden]

Personal opinion! Please prove me I'm wrong!

The upvotes are because the feature is long awaited, not because they reviewed the branch and think it can be merged. I tried giving the branch a look, and I can't grasp how parallelism was implemented —it looks like there is a scheduler per thread, and work stealing when a scheduler has nothing to resume, but that's pure guess.

I think the current implementation (Fiber, Scheduler, ...) needs a rework, too. I'd like Fibers to have public yield/pause/resume methods (e.g. for HTTP/2 control-flow) and keep the Scheduler private, but still allow to create/cancel events (e.g. to bind external libraries).

Also, nothing in the stdlib is ready for parallelism, not even Mutex is. I'm afraid rushing to merge an very outdated branch won't do any good.

IMHO: we need to plan things up (what we want, how to achieve it, and how to implement it). Of course use the branch as reference to plan and implement this properly, with the goal to merge! But that will take a lot more time, I'm afraid.

[waj]

I agree 100% with you, @ysbaddaden. The current implementation is mostly experimental and not only is slow but it also breaks a lot of existing functionality.

We’ll be definitely planning about this in the following months and it would be really helpful to have some input about scenarios where parallel execution is desider. @bararchy could you share anything about your use cases?

[bararchy]

@waj and @ysbaddaden thanks for the replay and feedback.

Let me maybe explain and shed more light into what drives me and my companies needs regarding multi-threading support.

[] = single machine or process
<=> = push and pull of data

Right now we have a basic but efficient topology

[frontend] <=> [backend] <=> [[api_interface <=> Fiber(workers)]]

The issue happens where the workers are doing heavy math work (Machine learning), I get stuck at a situation where the API interface wont respond to requests until the work at the fiber will finish.

It's "easy" to fix if I use a new process for each worker and spin up IPC of a sort but it's a big overhead for something I rather handle in a single process.
now, this means that instead of a simple memory share between threads (Channel-Fiber) I need to install redis and manage a full queue using some tech or the other.

Anyway, the point is that without multi-thread support my project needs to be much more complicated and with higher overhead to have the same performance (ie use all cores)

[RX14]

I'm certain we're not going to do this. Knowingly breaking the entire stdlib is an incredibly bad idea, and there's no certainty that once we do merge the branch we'll have the resources or expertise to fix the bugs. There's a reason that the branch hasn't been worked on for a long time.

[ysbaddaden]

I have a similar use case: a product I work on has many small services.

Some are independent and communicate through AMQP (which increases reliability in a few key-points), but some services process a message through many sub-handlers (sometimes dozens).

Some are IO-intensive and could be okayish on one thread (thought limited), but some are CPU-intensive and would block. Starting many sub-processes and going through AMQP or some other queue would be very overkill (we're talking dozens of them, with complex chains). Spawning fibers and communicating with channels would be simpler and enough.

We could even start a single monolithic process, instead of dealing with many.

[bararchy]

@ysbaddaden Sounds indeed very similar, and I would think that this is not some unique usecase where adding IPC is a huge overkill.

[kostya]

i have a use case, when i need to do many concurrent http requests with libcurl (https://github.com/kostya/curl-downloader), because it have some features that crystal http-client not. LibCurl is thread blocking library. Right now i use fork (run_with_fork), but it quite hack. I also test libcurl multi - bad it much worse than fork.

[RX14]

@kostya that's a usecase which would be better served by integrating libcurl with libevent. It's possible to do, just more complex than using it blocking. Not as complex as implementing parallelism though. Alternately, just implementing the desired features in crystal (what features are you missing, by the way).

[kostya]

i integrate it with libevent and multi interface https://github.com/kostya/curl-downloader/blob/multi/src/curl-downloader/multi.cr, it works ok for small concurrency (10). But in production it works so bad, unpredictable brakes, so i switch to fork.

[straight-shoota]

So, what are you missing from Crystal's implementation? @kostya

[bararchy]

@straight-shoota Me or @kostya ?

[kostya]

@straight-shoota, fork also have some problems with io and sigpipe, and i dont know how to reproduce it. But it rare, so works actually pretty good. If crystal have true thread pool it would be much better, i think.

[RX14]

@kostya what are you missing from crystal's stdlib that libcurl provides.