Reactor-netty: IllegalReferenceCountException: refCnt: 0, decrement: 1 on large payloads

Created on 10 Nov 2020  路  13Comments  路  Source: reactor/reactor-netty

I am using a Socks5 proxy but this issue also occurs with a direct connection (no proxy).
This issue seems to be related with https://github.com/netty/netty/issues/9882 for in this case it happens all the time for the same URL.
I upgraded spring boot webflux version from 2.3.3.RELEASE to 2.3.5.RELEASE but I am still getting this issue.

Expected Behavior


Every time I make a GET request to a specific URL that generates a big text file (around 130 000 lines) I expect to get the text file parsed into a string.

Actual Behavior


Netty for some reason is throwing io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1

Steps to Reproduce

I am afraid I cannot share the url from where to get this text file as it is a private URL and I have no control over it.

This is the code to send the GET request:

httpReactiveClient.get()
                .uri(URI.create(BASE_URL + url)) 
                .retrieve()
                .bodyToMono(String.class)
                .map(s -> cleanResponseBody(url, s));

// Later call subscribe

Using exchange().flatMap() insteand of bodyToMono() and with debug mode on I can see that the server returns 200 OK response code so I believe the issue occurs while dealing the big response body (as also hinted here https://github.com/netty/netty/issues/9882)

Your Environment

  • Reactor version(s) used: Using spring-boot-starter-webflux 2.3.5.RELEASE which is using reactor-netty 0.9.13.RELEASE
  • Other relevant libraries versions (eg. netty, ...):
  • JVM version (java -version): OpenJDK 11
  • OS and version (eg. uname -a): Ubuntu 20.04.1 LTS

Here is the debug log ouput from my request:

2020-11-10 00:20:24.834 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba] Created a new pooled channel, now 1 active connections and 0 inactive connections
2020-11-10 00:20:24.834 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.tcp.SslProvider            : [id: 0x22cf32ba] SSL enabled using engine SSLEngineImpl and SNI subdomain.upstream-server.com/<unresolved>:443
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.channel.BootstrapHandlers  : [id: 0x22cf32ba] Initialized pipeline DefaultChannelPipeline{(Socks5InitialResponseDecoder#0 = io.netty.handler.codec.socksx.v5.Socks5InitialResponseDecoder), (Socks5InitialResponseDecoder#0.encoder = io.netty.handler.codec.socksx.v5.Socks5ClientEncoder), (reactor.left.proxyHandler = io.netty.handler.proxy.Socks5ProxyHandler), (reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.sslReader = reactor.netty.tcp.SslProvider$SslReadHandler), (BootstrapHandlers$BootstrapInitializerHandler#0 = reactor.netty.channel.BootstrapHandlers$BootstrapInitializerHandler), (PooledConnectionProvider$PooledConnectionAllocator$PooledConnectionInitializer#0 = reactor.netty.resources.PooledConnectionProvider$PooledConnectionAllocator$PooledConnectionInitializer), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/files/6FfTNtIl/, connection=PooledConnection{channel=[id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050]}}, [response_completed])
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] Removed handler: ReadTimeoutHandler, pipeline: DefaultChannelPipeline{(reactor.left.proxyHandler = io.netty.handler.proxy.Socks5ProxyHandler), (reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (WriteTimeoutHandler = io.netty.handler.timeout.WriteTimeoutHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] Removed handler: WriteTimeoutHandler, pipeline: DefaultChannelPipeline{(reactor.left.proxyHandler = io.netty.handler.proxy.Socks5ProxyHandler), (reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/files/6FfTNtIl/, connection=PooledConnection{channel=[id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050]}}, [disconnecting])
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] Releasing channel
2020-11-10 00:20:24.835 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x84c6a868, L:/127.0.0.1:47946 - R:localhost/127.0.0.1:9050] Channel cleaned, now 0 active connections and 1 inactive connections
2020-11-10 00:20:24.836 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Registering pool release on close event for channel
2020-11-10 00:20:24.836 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Channel connected, now 1 active connections and 0 inactive connections
2020-11-10 00:20:24.844 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x84c6a868, L:/127.0.0.1:47946 ! R:localhost/127.0.0.1:9050] onStateChange(PooledConnection{channel=[id: 0x84c6a868, L:/127.0.0.1:47946 ! R:localhost/127.0.0.1:9050]}, [disconnecting])
2020-11-10 00:20:25.196 DEBUG 19156 --- [or-http-epoll-4] io.netty.handler.ssl.SslHandler          : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] HANDSHAKEN: protocol:TLSv1.3 cipher suite:TLS_AES_256_GCM_SHA384
2020-11-10 00:20:25.196 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] onStateChange(PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050]}, [connected])
2020-11-10 00:20:25.196 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/, connection=PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050]}}, [configured])
2020-11-10 00:20:25.196 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Added decoder [ReadTimeoutHandler] at the end of the user pipeline, full pipeline: [reactor.left.proxyHandler, reactor.left.sslHandler, reactor.left.httpCodec, reactor.left.decompressor, ReadTimeoutHandler, reactor.right.reactiveBridge, DefaultChannelPipeline$TailContext#0]
2020-11-10 00:20:25.197 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Added decoder [WriteTimeoutHandler] at the end of the user pipeline, full pipeline: [reactor.left.proxyHandler, reactor.left.sslHandler, reactor.left.httpCodec, reactor.left.decompressor, ReadTimeoutHandler, WriteTimeoutHandler, reactor.right.reactiveBridge, DefaultChannelPipeline$TailContext#0]
2020-11-10 00:20:25.197 DEBUG 19156 --- [or-http-epoll-4] r.netty.http.client.HttpClientConnect    : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Handler is being applied: {uri=https://subdomain.upstream-server.com/feed/yjb05.dat, method=GET}
2020-11-10 00:20:25.197 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/feed/yjb05.dat, connection=PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050]}}, [request_prepared])
2020-11-10 00:20:25.197 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/feed/yjb05.dat, connection=PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050]}}, [request_sent])
2020-11-10 00:20:25.334 DEBUG 19156 --- [or-http-epoll-4] io.netty.handler.ssl.SslHandler          : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] HANDSHAKEN: protocol:TLSv1.3 cipher suite:TLS_AES_256_GCM_SHA384
2020-11-10 00:20:25.334 DEBUG 19156 --- [or-http-epoll-4] io.netty.handler.ssl.SslHandler          : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] HANDSHAKEN: protocol:TLSv1.3 cipher suite:TLS_AES_256_GCM_SHA384
2020-11-10 00:20:25.382 DEBUG 19156 --- [or-http-epoll-4] r.n.http.client.HttpClientOperations     : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] Received response (auto-read:false) : [Server=nginx, Date=Mon, 09 Nov 2020 23:20:25 GMT, Content-Type=text/javascript;charset=UTF-8, Connection=keep-alive, Expires=Mon, 09 Nov 2020 23:20:05 GMT, Pragma=no-cache, Vary=Accept-Encoding, Cache-Control=public, Last-Modified=Mon, 09 Nov 2020 23:19:55 GMT, Accept-Ranges=bytes, Age=29, X-Deliver=Mon, 09 Nov 2020 23:20:25 GMT, transfer-encoding=chunked]
2020-11-10 00:20:25.382 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/feed/yjb05.dat, connection=PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050]}}, [response_received])
2020-11-10 00:20:25.383 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.channel.FluxReceive        : [id: 0x22cf32ba, L:/127.0.0.1:47950 - R:localhost/127.0.0.1:9050] FluxReceive{pending=0, cancelled=false, inboundDone=false, inboundError=null}: subscribing inbound receiver
2020-11-10 00:20:25.487 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] Channel closed, now 0 active connections and 0 inactive connections
2020-11-10 00:20:25.487 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] Non Removed handler: ReadTimeoutHandler, context: ChannelHandlerContext(ReadTimeoutHandler, [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050]), pipeline: DefaultChannelPipeline{(reactor.left.proxyHandler = io.netty.handler.proxy.Socks5ProxyHandler), (reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (ReadTimeoutHandler = io.netty.handler.timeout.ReadTimeoutHandler), (WriteTimeoutHandler = io.netty.handler.timeout.WriteTimeoutHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-11-10 00:20:25.487 DEBUG 19156 --- [or-http-epoll-4] reactor.netty.ReactorNetty               : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] Non Removed handler: WriteTimeoutHandler, context: ChannelHandlerContext(WriteTimeoutHandler, [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050]), pipeline: DefaultChannelPipeline{(reactor.left.proxyHandler = io.netty.handler.proxy.Socks5ProxyHandler), (reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (ReadTimeoutHandler = io.netty.handler.timeout.ReadTimeoutHandler), (WriteTimeoutHandler = io.netty.handler.timeout.WriteTimeoutHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-11-10 00:20:25.488 ERROR 19156 --- [or-http-epoll-4] r.n.channel.ChannelOperationsHandler     : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] Error was received while reading the incoming data. The connection will be closed.

io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
    at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100) ~[netty-buffer-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.http.DefaultHttpContent.release(DefaultHttpContent.java:92) ~[netty-codec-http-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:340) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
    at reactor.netty.channel.ChannelOperations.onInboundNext(ChannelOperations.java:358) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
    at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:677) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
    at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:96) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1282) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1329) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.proxy.ProxyHandler.channelRead(ProxyHandler.java:253) ~[netty-handler-proxy-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
    at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at java.base/java.lang.Thread.run(Thread.java:832) ~[na:na]

2020-11-10 00:20:25.489  WARN 19156 --- [or-http-epoll-4] reactor.netty.channel.FluxReceive        : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] An exception has been observed post termination

io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
    at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100) ~[netty-buffer-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.http.DefaultHttpContent.release(DefaultHttpContent.java:92) ~[netty-codec-http-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:340) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
    at reactor.netty.channel.ChannelOperations.onInboundNext(ChannelOperations.java:358) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
    at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:677) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
    at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:96) ~[reactor-netty-0.9.13.RELEASE.jar:0.9.13.RELEASE]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1282) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1329) ~[netty-handler-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.handler.proxy.ProxyHandler.channelRead(ProxyHandler.java:253) ~[netty-handler-proxy-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
    at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) ~[netty-transport-native-epoll-4.1.53.Final-linux-x86_64.jar:4.1.53.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.53.Final.jar:4.1.53.Final]
    at java.base/java.lang.Thread.run(Thread.java:832) ~[na:na]

2020-11-10 00:20:25.489 DEBUG 19156 --- [or-http-epoll-4] r.n.resources.PooledConnectionProvider   : [id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050] onStateChange(GET{uri=/feed/yjb05.dat, connection=PooledConnection{channel=[id: 0x22cf32ba, L:/127.0.0.1:47950 ! R:localhost/127.0.0.1:9050]}}, [disconnecting])
fospringframework typbug
Source
picture dazito

Most helpful comment

Yes that should do it.

picture rstoyanchev on 10 Nov 2020
👍4

All 13 comments

@dazito The release here is correct
https://github.com/reactor/reactor-netty/blob/5711e2782535a813bbfd22cc5af2def5d7cb9bbc/reactor-netty-core/src/main/java/reactor/netty/channel/FluxReceive.java#L340

Having an exception at that point mean that Spring Framework does not use correctly retain/release.

@rstoyanchev Can you please help here?

violetagg picture violetagg on 10 Nov 2020

Generally we retain immediately in getBody() of ReactorClientHttpResponse. For this scenario the buffers are collected into a List via DataBufferUtils#join so there is on release anywhere, unless the accumulated buffers exceed the maxInMemorySize limit which is 256K by default, which would raise an error and cause all accumulated buffers to be released.

@dazito are customizing the maxInMemorySize at all? It seems with so many lines you'll likely exceed the default.

rstoyanchev picture rstoyanchev on 10 Nov 2020

@rstoyanchev no, I am not customizing the maxInMemorySize. It does make sense that the default ones may not be enough.

You mean something like:

ExchangeStrategies exchangeStrategies = ExchangeStrategies
                .builder()
                .codecs(configurer -> configurer.defaultCodecs().maxInMemorySize(someBiggerSizeHere))
                .build();

And then pass it to the WebClient like:

WebClient.builder()
                .exchangeStrategies(exchangeStrategies)
                .build();

Right?

dazito picture dazito on 10 Nov 2020
🚀1

Yes that should do it.

rstoyanchev picture rstoyanchev on 10 Nov 2020
👍4

Thanks @rstoyanchev
I will try it out and report back later today.

Is it possible to make the exception message more clear on what is the underlying issue? That would be very nice to have as from the exception message it looked like a netty internal issue.

dazito picture dazito on 10 Nov 2020

I was able to reproduce the issue by using WebClient with retrieve().bodyToMono(String.class) against content that exceeds the configured maxInMemorySize. So making sure you don't hit the limit should make it go away.

That said the IllegalReferenceCountException should not be happening. I've created an issue for that in the Spring Framework https://github.com/spring-projects/spring-framework/issues/26060.

In addition I think there might be some potential logging improvement for Reactor Netty? When the limit is exceeded, the DataBufferLimitException appears only at DEBUG level:

12:39:26.140 [reactor-http-epoll-2] DEBUG reactor.netty.resources.DefaultPooledConnectionProvider - [id: 0x07b1c47f, L:/127.0.0.1:52152 ! R:localhost/127.0.0.1:8081] Channel closed, now 0 active connections and 0 inactive connections
Exception in thread "main" org.springframework.web.reactive.function.client.WebClientResponseException: 200 OK from GET http://localhost:8081/text; nested exception is org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 64
    at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:212)
    at org.springframework.web.reactive.function.client.DefaultClientResponse.lambda$createException$1(DefaultClientResponse.java:216)
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106)
    at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
    at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2346)
    at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2154)
    at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onSubscribe(FluxOnErrorResume.java:74)
    at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:54)
    at reactor.core.publisher.Mono.subscribe(Mono.java:3987)
    at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)
    at reactor.core.publisher.Operators$MonoSubscriber.onError(Operators.java:1831)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onError(FluxMapFuseable.java:140)
    at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onError(FluxContextWrite.java:121)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onError(FluxMapFuseable.java:334)
    at reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onError(FluxFilterFuseable.java:382)
    at reactor.core.publisher.MonoCollect$CollectSubscriber.onError(MonoCollect.java:144)
    at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onError(FluxOnAssembly.java:392)
    at reactor.core.publisher.FluxMap$MapSubscriber.onError(FluxMap.java:132)
    at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221)
    at reactor.core.publisher.FluxMap$MapSubscriber.onError(FluxMap.java:132)
    at reactor.core.publisher.Operators.error(Operators.java:196)
    at reactor.netty.channel.FluxReceive.subscribe(FluxReceive.java:171)```

The IllegalReferenceCountException however appears at ERROR:

12:39:26.151 [reactor-http-epoll-2] ERROR reactor.netty.channel.ChannelOperationsHandler - [id: 0x07b1c47f, L:/127.0.0.1:52152 ! R:localhost/127.0.0.1:8081] Error was received while reading the incoming data. The connection will be closed.
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
    at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74)
    at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138)
    at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100)
    at io.netty.handler.codec.http.DefaultHttpContent.release(DefaultHttpContent.java:92)
    at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88)
    at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:340)
rstoyanchev picture rstoyanchev on 10 Nov 2020
👍1

@rstoyanchev I don't think this is the proper log as we do not log exceptions here

12:39:26.140 [reactor-http-epoll-2] DEBUG reactor.netty.resources.DefaultPooledConnectionProvider - [id: 0x07b1c47f, L:/127.0.0.1:52152 ! R:localhost/127.0.0.1:8081] Channel closed, now 0 active connections and 0 inactive connections

https://github.com/reactor/reactor-netty/blob/594c14641e3dea4be8ef9ba6ec4af812a848e1a1/reactor-netty-core/src/main/java/reactor/netty/resources/DefaultPooledConnectionProvider.java#L223-L229

https://github.com/reactor/reactor-netty/blob/594c14641e3dea4be8ef9ba6ec4af812a848e1a1/reactor-netty-core/src/main/java/reactor/netty/resources/DefaultPooledConnectionProvider.java#L325-L332

As for this log

12:39:26.151 [reactor-http-epoll-2] ERROR reactor.netty.channel.ChannelOperationsHandler - [id: 0x07b1c47f, L:/127.0.0.1:52152 ! R:localhost/127.0.0.1:8081] Error was received while reading the incoming data. The connection will be closed.
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
...
    at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88)
    at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:340)

This is intentional in order to point a problem in the retain-release lifecycle

violetagg picture violetagg on 10 Nov 2020

Right, in my case I reproduced it with simple client code that calls .block() at the end, and that WebClientResponseException stacktrace is to System.out but just happens to be interspersed with log messages.

@dazito check your code downstream from the .map(..). You're likely suppressing or ignoring the root cause exception that flows downstream and only seeing the IllegalReferenceCountException instead.

I think this issue can be closed. The only work remaining is https://github.com/spring-projects/spring-framework/issues/26060.

rstoyanchev picture rstoyanchev on 10 Nov 2020

OK
Closing the issue

violetagg picture violetagg on 10 Nov 2020

@rstoyanchev I double checked my code and I am not catching neither ignoring any exception on my downstream. On my logs I see no WebClientResponseException showing up.

I also set log level to debug for io.netty and reactor.netty as:

logging:
  level:
    io.netty: DEBUG
    reactor.netty: DEBUG

I can also confirm that increasing the default memory size fixed the issue :+1:

dazito picture dazito on 10 Nov 2020

It's getting dropped somewhere though. If you put a doOnError though before the .map(..) you'll probably see the error signal passing through. If you showed more of your downstream chain or how you subscribe then maybe we can figure out why the error wasn't showing.

rstoyanchev picture rstoyanchev on 10 Nov 2020

@violetagg, I am registering customEncodes and decoders as following:
.codecs(configurer -> configurer.customCodecs().register(new AvroHttpMessageEncoder<>(true)))

How am I supposed to set maxInMemorySize, because its hitting -

2021-03-23 11:41:04.002: [ctor-http-nio-1] ERROR: reactor.netty.channel.ChannelOperationsHandler: [id: 0x84e4f446, L:0.0.0.0/0.0.0.0:50052 ! R:/10.216.8.70:8080] Error was received while reading the incoming data. The connection will be closed. {|HDN|||||| - |} | HDN_HistVaRAnalysis
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74)
at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138)
at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100)
at io.netty.handler.codec.http.DefaultHttpContent.release(DefaultHttpContent.java:92)
at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88)
at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:340)
at reactor.netty.channel.ChannelOperations.onInboundNext(ChannelOperations.java:358)
at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:677)
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:96)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:425)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Vaibhav 2021-03-23 11:41:04.004: [ctor-http-nio-1] WARN : reactor.netty.channel.FluxReceive: [id: 0x84e4f446,

@rstoyanchev , could you please help?

vaibsgharge picture vaibsgharge on 23 Mar 2021

@vaibsgharge For Spring Framework functionality please use the corresponding Spring Framework support channels. Thanks.

violetagg picture violetagg on 23 Mar 2021
Was this page helpful?
0 / 5 - 0 ratings

Related issues

vchekan picture vchekan  路  8Comments
anh-dev picture anh-dev  路  3Comments
EdgarsM picture EdgarsM  路  7Comments
sdeleuze picture sdeleuze  路  7Comments
dsyer picture dsyer  路  8Comments