Right now, we only support Avro-encoded data. This will likely expand the syntax of CREATE SINK.
@rjnn is there a milestone we should assign to this issue?
This is fairly high priority, so adding it to public beta.
There are a couple things we'll need to tackle in this ticket:
CREATE SINK to take a format option that indicates either avro or json. Potentially creating multiple KafkaSinkConnector structs, one for each format type?
Figuring out how to turn Materialize Row objects into JSON objects. Looking around, it seems like there are three options:
Useful references:
https://docs.mongodb.com/kafka-connector/master/kafka-sink-data-formats/
Because the POC that depends on this is delayed, this should be completed by or soon after 0.2.
Any new milestone for this?
Any update on this? Looking forward to it.
Most helpful comment
This is fairly high priority, so adding it to public beta.