Node-rdkafka: Streams questions

[mjafarpour]

Hi,
1- Is there any way to set poll interval when using stream mode?
2- Can use multiple consumer stream in one function that use same topic as input? that means send separate delivery message for each stream in specific topic.
3- How can use exactly once scenario in stream mode?
4- How can use partitions.

[webmakersteve]

1- Is there any way to set poll interval when using stream mode?

Yes. You can set the poll interval just as you would on any other producer. You can access the producer as stream.producer and it has all of its normal methods available to you.

2- Can use multiple consumer stream in one function that use same topic as input? that means send separate delivery message for each stream in specific topic.

If you want the same messages to go to different streams you need to consume them on a different consumer group id.

3- How can use exactly once scenario in stream mode?

Exactly once in Kafka is a big topic, and I'm not sure librdkafka itself fully supports it yet, so the answer is currently, no.

4- How can use partitions.

I'm not sure exactly what you mean. You are using partitions automatically by using the stream consumer and producer. If you'd like to specify your partition in the producer you can do so by making it an object mode stream and giving it objects with the partition id to produce to.

[mjafarpour]

Thanks for your answers.
Which methods not exist in stream mode?

[webmakersteve]

The methods meant for public use are documented here: https://blizzard.github.io/node-rdkafka/current/

[mjafarpour]

Thanks,
and one more question:
When I use consumer stream, i have a five or more seconds delay at start time. how can remove this delay?

[webmakersteve]

You can't. Kafka does not start sending you data until the group has stabilized. If you want to reduce the delay as much as possible use a different consume group each time. But I don't recommend doing this.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.