Rocketmq: when client init and first time read offset from store with a timeout requset, offset back to minLogicOffset

Created on 3 Mar 2021  ·  2Comments  ·  Source: apache/rocketmq

最近碰到一个问题,某个业务的同学说他们服务使用的某个 topic 的某个 Queue 发生了位点回退。
我看了下确实是回退了,然后看了下日志,是初始化 client 时请求 offset timeout,然后 请求的 offset 变成了 -1。
broker 在接收到 -1 的请求时,返回了 minLogicOffset ,然后 client 带着 minLogicOffset 把之前 broker 保存的正常的位点覆盖了。

想请教下为什么当拿着 offset = -1 来请求时返回的是 minLogicOffset,而不是返回 broker 保存的该 group 该 Queue 的位点呢?

client日志

2021-03-02 15:15:12 WARN PullMessageService - invokeSync: wait response timeout exception, the channel[brokerip:10911]
2021-03-02 15:15:12 WARN NettyClientWorkerThread_2 - receive response, but not matched any request, brokerip:10911
2021-03-02 15:15:12 WARN NettyClientWorkerThread_2 - RemotingCommand [code=0, language=JAVA, version=317, opaque=6774, flag(B)=1, remark=null, extFields={offset=5757922}, serializeTypeCurrentRPC=JSON]
2021-03-02 15:15:12 WARN PullMessageService - fetchConsumeOffsetFromBroker exception, MessageQueue [topic=tp_inner_order_operation, brokerName=broker-06, queueId=10]
2021-03-02 15:15:12 WARN NettyClientPublicExecutor_3 - the pull request offset illegal, PullRequest [consumerGroup=ts_biz_operation_group, messageQueue=MessageQueue [topic=tp_inner_order_operation, brokerName=broker-0
6, queueId=10], nextOffset=-1] PullResult [pullStatus=OFFSET_ILLEGAL, nextBeginOffset=5531762, minOffset=5531762, maxOffset=5757977, msgFoundList=0]
2021-03-02 15:15:22 WARN PullMessageServiceScheduledThread - unlock messageQueue. group:ts_biz_operation_group, clientId:clientip@30487, mq:MessageQueue [topic=tp_inner_order_operation, brokerName=broker-06, queue
Id=10]
2021-03-02 15:15:22 WARN PullMessageServiceScheduledThread - fix the pull request offset, PullRequest [consumerGroup=ts_biz_operation_group, messageQueue=MessageQueue [topic=tp_inner_order_operation, brokerName=broker
-06, queueId=10], nextOffset=5531762]
2021-03-02 15:15:45 WARN PullMessageService - the consumer message buffer is full, so do flow control, minOffset=5531803, maxOffset=5532809, size=1008, pullRequest=PullRequest [consumerGroup=ts_biz_operation_group, me
ssageQueue=MessageQueue [topic=tp_inner_order_operation, brokerName=broker-06, queueId=10], nextOffset=5532810], flowControlTimes=1

 

broker日志

broker.log:2021-03-02 15:15:12 INFO PullMessageThread_5 - the request offset too small. group=ts_biz_operation_group, topic=tp_inner_order_operation, requestOffset=-1, brokerMinOffset=5531762, clientIp=/clientip:24944
broker.log:2021-03-02 15:15:12 WARN PullMessageThread_5 - PULL_OFFSET_MOVED:correction offset. topic=tp_inner_order_operation, groupId=ts_biz_operation_group, requestOffset=-1, newOffset=5531762, suggestBrokerId=0
broker.log:2021-03-02 15:15:22 WARN ConsumerManageThread_15 - [NOTIFYME]update consumer offset less than store. clientHost=clientip:24944, key=tp_inner_order_operation@ts_biz_operation_group, queueId=10, requestOffset=5531762, storeOffset=5757922
store.log:2021-03-02 04:54:08 INFO StoreScheduledThread1 - Compute logical min offset: 5531762, topic: tp_inner_order_operation, queueId: 10
bug client

Most helpful comment

这个可以算是一个BUG,具体表现为多条队列只有一部分位点变成了 Broker 最小位点(minOffset),在网络不稳定时会获取不到位点,而 SDK 没有区分原因到底是 Timeout 还是获取不到,修复方案是 SDK 区分这几种异常

All 2 comments

这个可以算是一个BUG,具体表现为多条队列只有一部分位点变成了 Broker 最小位点(minOffset),在网络不稳定时会获取不到位点,而 SDK 没有区分原因到底是 Timeout 还是获取不到,修复方案是 SDK 区分这几种异常

这个可以算是一个BUG,具体表现为多条队列只有一部分位点变成了 Broker 最小位点(minOffset),在网络不稳定时会获取不到位点,而 SDK 没有区分原因到底是 Timeout 还是获取不到,修复方案是 SDK 区分这几种异常

我感觉这块也是个BUG,但是我看最新版本一直没修复就有点困惑

Was this page helpful?
0 / 5 - 0 ratings