Logstash: Move from Concurrent Ruby gem to Java Concurrent

[andrewvc]

I'm writing this post in the wake of #7372 , where we found that the concurrent ruby gem broke on some javas (for reasons that are still unclear).

The concurrent gem is an extra place where things can go wrong as we have seen. It doesn't add much, just adds an extra maintenance burden.

I propose that we put a moratorium on new uses of the concurrent gem, and make it a goal to remove its existing uses as we refactor various parts of logstash. We can easily migrate all uses of concurrent classes to plain java.util.concurrent.*. JRuby makes this relatively easy.

In place of Concurrent::TimerTask a java.util.concurrent.ScheduledThreadPoolExecutor can be used. JRuby proc objects work as Runnable, so you can easily pass those in directly.

In place of Concurrent::Map you can use java.util.concurrent.ConcurrentHashMap which as of JDK8 performs a superset of the functions the ruby version uses.

Thoughts?

[andrewvc]

I'll also add that we can use ScheduledThreadPoolExecutor to consolidate some of our timer threads. The concurrent gem uses one thread per task, which is not as efficient when you have a number of short-lived infrequent tasks.

[ph]

I am OK to slowly migrate the use of concurrent ruby to Java equivalent.

[jsvd]

I'm +1 on stop using/removing it as much as possible, this 0.9.x 1.0.x situation is creating too many problems. And I agree there's not much gain in using it since java concurrent is easily accessible from jruby.

[ph]

I would just clarify, I +1 to stop using it, I don't think we should just go crazy and replace it all of it right away. The removal will come from the java migration.

[andrewvc]

Nothing more to discuss here. I'm going to close this because I think this will naturally happen as we move classes from ruby -> java