Librdkafka: RoundRobin assignor question

[plachor]

Hi, I'm experiencing not well balanced partition assignments (with round-robin assignor selected). My issue is that partitions from topic 'A' are not properly distributed among all group members with active subscription to topic 'A'. This happens if in consumer group there are mixed subscriptions like:

memeber-1: 'A'
memeber-2: 'A'
memeber-3: 'B'
memeber-4: 'B','C'

I've been looking for an issue on my side without any luck so far. So I checked your implementation on Github and I wonder if on:
https://github.com/edenhill/librdkafka/blob/55995b9afed08dc33a0da0c161507fa429f10322/src/rdkafka_roundrobin_assignor.c#L87-L92
this modulo should not take members count instead? I might been misunderstanding it as I have not opened this code yet in any IDE and it been years since I written any thing in c++.

[edenhill]

Huhm, yes, or that we iterate eligible_topic->members instead.

[plachor]

Good at least we know now were is the issue :) we had like 5 instance subscribing topic 'A' and all 45 partitions assigned to single instance

[plachor]

Could it lead to infinite loop also if not lucky in first range ( limited by this module ) ?

[edenhill]

Seems like that, yeah.

I'll fix it.

[plachor]

hi, @edenhill could you pre-release this hot-fix? I believe that I falling in too this endless loop after update to 1.5.2, and 1.5.2 addresses earlier issues found in round-robin #3024.

[edenhill]

@plachor I'll roll a v1.5.3 maintenance release this week

[plachor]

Thanks ;)

[edenhill]

v1.5.3 is now released