Hystrix: Please clarify the behavior of a maxQueueSize of -1

[JasonMathison]

The existing documentation for maxQueueSize includes the following implementation impact of setting the value to -1:

If you set this to -1 then SynchronousQueue will be used, otherwise a positive value will be used with LinkedBlockingQueue.

That description doesn't explicitly state the behavior impact. I believe that the impact is that with a maxSize of -1 once you have all of the threads in your threadpool busy additional calls will immediately fail until a thread becomes available. Documenting that, or the actual behavior, would probably save a lot of head scratching for people in the future :)

[mattrjacobs]

The behavior is described in detail at: http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadPoolExecutor.html, as Hystrix is just providing arguments to a standard JDK thread pool. I've added a link to that in the Configuration wiki.

Does that make sense?

[JasonMathison]

I have read the ThreadPoolExecutor and SynchronousQueue documentation, but by itself it doesn't really clarify Hystrix's behavior.

In the SynchronousQueue case Hystrix could use offer(E e), and immediately fail the operation if the threadpool is at max size with all of the threads in use (this is my guess for how Hystrix behaves). Hystrix could also use offer(E e, long timeout, TimeUnit unit) where threads would wait for a thread from the thread pool to free up.

The ThreadPoolExecutor documentation only mentions SynchronousQueues in the context of "Direct Handoffs". The documentation there mentions that "Direct handoffs generally require unbounded maximumPoolSizes to avoid rejection of new submitted tasks." This didn't help me make sense of Hystrix's behavior as having an unbounded number of threads would completely defeat the purpose of Hystrix.

Can you understand why I find this all ambiguous?

[mattrjacobs]

Yeah, thanks for being specific - definitely makes for more accurate answers. And those are good questions. The quote you provided includes the clause "to avoid rejection of new submitted tasks". In the SynchronousQueue case, Hystrix explicitly chooses to reject, not to avoid rejection. So it maintains the bounded queue and drops the work which would otherwise queue.

The alternative to this is to specify a positive int for the queue size. In this case, tasks are first inserted into that queue. As room is available in the threadpool, it consumes those tasks and executes them. The only way to reject in this case is to find a full queue, which implies either a high saturation of the threadpool, or a very high arrival rate to the queue (faster than the threadpool can take tasks off). In this case, the amount of outstanding work is still bounded to (threadpool size + queue size).

Hope that helps, and happy to answer further questions.

[mrumpf]

I also find the current documentation confusing in this area.

However, your explanation makes the behavior clearer. It should be added to the section
https://github.com/Netflix/Hystrix/wiki/Configuration#maxqueuesize

What I'm still missing is a recommendation on how to set the queue size. I find it confusing that the default queue size is -1 and in the calculation diagram the value is set to 5-10:
https://github.com/Netflix/Hystrix/wiki/Configuration#ThreadPool
Without mentioning this settings in the text and explaining why you choose to set the value like this.

Please also consider updating the diagram:
https://github.com/Netflix/Hystrix/wiki/Configuration#ThreadPool
In the diagram you find the hint "Threadpool queue size: 5-10 (0 does not work but get close to it)".
What do you mean by "0 does not work". Colleagues even assume that there is a misbehavior when settings the queuesize to -1.
I assume that a queue size of 0 cannot be used to initialize maximumPoolSize parameter of the underlying ThreadPoolExecutor because it would throw an IllegalArgumentException in this case.

I find myself discussing with colleagues whether we can avoid RejectedExceptions by switching from queuesize -1 to a value that has some correlation to the size of the thread pool.
I cannot see why a queue size should prevent any RejectedExceptions. With over 1000Req/s and the thread pool full, increasing the queuesize, to 80 would only buy you another 80ms until you see RejectedExceptions. So if your backend service has latencies in the area of seconds, adapting the queuesize like this does not really help.