Confluent-kafka-dotnet: [Question] Producing messages using Protobuf requires 1 Producer per schema, limiting performance gains of message batching and transactions

[DrorOzgaon]

Description

We're migrating our models to Profobuf schemas to validate them against our schema registry and boost performance. We have a small question regarding the way we should produce messages.

It seems like we need to create a new Producer for every single schema we want to produce. Doesn't that defeat the purpose of having 1 Producer that can improve performance with multiple messages? (I read about message batching, but I'm sure there are other optimisations.)

It will also prevent us from having transactions if we want to produce multiple message schemas at once. Am I missing something or is that the correct behavior?

I followed the example provided below:
https://github.com/confluentinc/confluent-kafka-dotnet/blob/master/examples/Protobuf/Program.cs

[mhowlett]

you can use the DependentProducerBuilder.

[DrorOzgaon]

you can use the DependentProducerBuilder.

Thanks! I think this solves the first part of my question, but for the second part with transactions - am I right to say that it's not possible for different schemas? Or is the transaction part of the DependentProducerBuilder Handler?

[mhowlett]

transactions will work across instances derived from the same Handle. note that you can only have one transaction open per Handle (there is no concept of sessions).

[DrorOzgaon]

Amazing, thanks a lot!