Confluent-kafka-dotnet: Why is group.id mandatory? Update Exception with more information

[mohoch1]

Hi.
In my use case there are 2 consumers, each of which need to read the entire topic.
I understand that I can do that by specifying a different group.id for each of them, but was wondering why can't I just leave the group.id empty?

The same question would apply to a use case in which there is only 1 consumer.

Currently, if I don't specify the group.id, I get this exception:" '_group.id' configuration parameter is required and was not specified._"

It would seem strange to me if I had to _make-up_ group IDs in a case they are not logically required. As far as I remember, the Java client library does permit not providing a group.id.

Am I missing something?
Thanks

[treziac]

If you use subscribe (to have automatic discovery of partitions and dynamic assign if new one are created), you have to use a group-id - this is how kafka works with coordinator.

If you use only assign or other functions which doesn't involve the Coordinator, you still have to use it - this is how librdkafka works for now, changing this is not pioritized (see https://github.com/edenhill/librdkafka/issues/593 for example)
As a workaround is easy to provide (see below), I don't think it will change if community don't make a change for this

For now you should use:

• unique group.id (Guid.NewGuid())
• enable.auto.commit=false (so you are sure you won't commit it and mess up coordinator with unused groupid)
• auto.offset.reset=smallest (or earliest/beginning, to start from beginning when using Subscribe)

[mhowlett]

leaving this open because i think we should update the exception message to acknowledge this requirement doesn't make sense in some situations. As things are, it's going to keep confusing people.

[lobinhojr]

The default behavior should be the consumers non-associated by groupId, then randomic/unique groupId doesn´t sounds like a good idea.