Kafka-node: How to set consumer option "auto.offset.reset" ?

Created on 28 Mar 2016  路  8Comments  路  Source: SOHU-Co/kafka-node

I'd like to consume from latest message when I don't have committed offset (ex: new consumer group).
At v0.3.2, HighLevelConsumer consumes from oldest message, but it isn't expected.

enhancement

Most helpful comment

I'll have a PR coming for picking up from the latest offset.

All 8 comments

You can set fromOffset true and give the offset you wish to consume in payload, example:

consumer = new HighLevelConsumer(
        client,
        [
            { topic: 't', offset: 1000 }
        ],
        { fromOffset: true }
    );

Thanks haio, but it doesn't match my needs.
I just intend to start consume latest message for "fresh consumer" (= there is no committed offset entry in zookeeper). In other language like Java, Kafka client API supports to set option map, like "auto.offset.reset=latest".
Is there any way to set Kafka client API option map directly?

This feature is not supported at the moment.

I'll have a PR coming for picking up from the latest offset.

@CharlesWall any option like auto.offset.reset=latest , sometime soon for 0.8.* version of kafka for node client.

@shubsk I wouldn't hold my breath waiting for that.

I haven't looked at any of this stuff since May. I learned how this module worked to write the fix I needed. @shubsk You should consider doing the same.

The solution is to use ConsumerGroup which supports both earliest and latest. I don't think it's worth updating the HLC since it has been deprecated by Kafka project.

Was this page helpful?
0 / 5 - 0 ratings