Confluent-kafka-go: Does kafka.Producer or kafka.Consumer thread safe?

[yangxikun]

A variable type of kafka.Producer or kafka.Consumer can be used by concurrent goroutine?

[edenhill]

Yes, absolutely.

[freeformz]

Please make a note of this in the Go API docs. The go standard is to assume that a type (or function) is not safe for concurrent use unless the docs indicate that it is.

I spent a bunch of time reading and re-reading things in this repo and a few things in the librdkafka repo, but didn't see anything that mentioned the library was "safe for concurrent use".

Let me know if I should open a new issue about this.

Thanks!

[jenu9417]

@edenhill

Here it is mentioned that, the kafka.Producer struct is safe for concurrent usage. But what happens when we are listening to the events channel for acknowledgement after calling 'producer.Produce()'
In this case, ideally two routines are waiting for the output from 'producer.Events()' channel, and the first output can be received by either of the go routines. In this case,
a) How can we say that, producer API is thread safe?
b) How to handle the scenario properly?

Sample Code:

events := producer.Events()
err := producer.Produce(message, nil)
ev := <-events
switch event := ev.(type) {
case *kafka.Message:
.....
case *kafka.Error:
.....
default:
.....
}