Cp-docker-images: java.io.IOException: Connection to 0 was disconnected before the response was read

[ethicalmohit]

I have initiated partition reassignment for topics partitions. This topic has around 30GB of data. I have 3 brokers and I have expanded the cluster with 3 more brokers. Since I have initiated the reassignment. I am seeing below errors. However, the data for the topic partitions which is not mentioned below are pilling up.

[2019-09-21 11:24:07,625] INFO [ReplicaFetcher replicaId=4, leaderId=0, fetcherId=0] Error sending fetch request (sessionId=514675011, epoch=INITIAL) to node 0: java.io.IOException: Connection to 0 was disconnected before the response was read. (org.apache.kafka.clients.FetchSessionHandler)

[2019-09-21 11:24:07,626] WARN [ReplicaFetcher replicaId=4, leaderId=0, fetcherId=0] Error in response for fetch request (type=FetchRequest, replicaId=4, maxWait=500, minBytes=1, maxBytes=10485760, fetchData={TOPIC-4=(offset=4624271, logStartOffset=4624271, maxBytes=104
8576), TOPIC-2=(offset=1704819, logStartOffset=1704819, maxBytes=1048576), TOPIC-8=(offset=990485, logStartOffset=990485, maxBytes=1048576), TOPIC-1=(offset=1696764, logStartOffset=1696764, maxBytes=1048576), TOPIC-7=(offset=991507, logStartOffset=991507, maxBytes=10485
76), TOPIC-5=(offset=988660, logStartOffset=988660, maxBytes=1048576)}, isolationLevel=READ_UNCOMMITTED, toForget=, metadata=(sessionId=514675011, epoch=INITIAL)) (kafka.server.ReplicaFetcherThread)

java.io.IOException: Connection to 0 was disconnected before the response was read
    at org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:97)
    at kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
    at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:220)
    at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:43)
    at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:146)
    at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:111)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)

Is this known ?

[ethicalmohit]

This has been solved by increasing the value of replica.socket.receive.buffer.bytes in all destination brokers.

After changing the above parameter and restarting broker. I was able to see the data in above-mentioned partitions.

[thasija]

@ethicalmohit to what value did you set the replica.socket.receive.buffer.bytes ? we are having the same issue in one of the production environments