Reactor-netty: Document How To Switch DNS Resolver

Created on 14 Dec 2020 · 22Comments · Source: reactor/reactor-netty

I ran into an issue with the Netty DNS resolver (that became the default in https://github.com/reactor/reactor-netty/pull/1252) when running in gvisor:

2020-12-14 13:32:50.225 MST Caused by: io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException: Search domain query failed. Original hostname: 'api.github.com' failed to resolve 'api.github.com' after 2 queries
2020-12-14 13:32:50.225 MST at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1013) ~[netty-resolver-dns-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(DnsResolveContext.java:966) ~[netty-resolver-dns-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:414) ~[netty-resolver-dns-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.resolver.dns.DnsResolveContext.access$600(DnsResolveContext.java:63) ~[netty-resolver-dns-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.resolver.dns.DnsResolveContext$2.operationComplete(DnsResolveContext.java:463) ~[netty-resolver-dns-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.resolver.dns.DnsQueryContext.tryFailure(DnsQueryContext.java:225) ~[netty-resolver-dns-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.resolver.dns.DnsQueryContext$4.run(DnsQueryContext.java:177) ~[netty-resolver-dns-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) ~[netty-transport-native-epoll-4.1.55.Final-linux-x86_64.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.55.Final.jar:4.1.55.Final]
2020-12-14 13:32:50.225 MST at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
2020-12-14 13:32:50.225 MST Caused by: io.netty.resolver.dns.DnsNameResolverTimeoutException: [/169.254.169.254:53] query via UDP timed out after 5000 milliseconds (no stack trace available)

And it took me some digging to figure out how to switch it back:

val httpClient = HttpClient.create().resolver(DefaultAddressResolverGroup.INSTANCE)
val webClient = WebClient.builder().clientConnector(ReactorClientHttpConnector(httpClient)).build()

It'd be nice to have this documented somewhere (but maybe here is sufficient).

typenhancement

Source

jamesward

👍9 🎉4

Most helpful comment

@Toparvion This is the issue #1534. It revealed two issues

https://github.com/netty/netty/issues/9796, a fix is in Java >=15
https://github.com/netty/netty/issues/11142, I'm working on it

violetagg on 19 Apr 2021

👍2

All 22 comments

@jamesward PTAL https://projectreactor.io/docs/netty/snapshot/reference/index.html#_host_name_resolution_2
If we need to extend that part we will do it. Thanks.

violetagg on 18 Dec 2020

Wonderful! Thank you.

jamesward on 18 Dec 2020

Captureh
Plz can someone help me to resolve this error i dont know where is the probleme here?

BenyahyaF45 on 19 Dec 2020

@BenyahyaF45 Did you try the workaround above?

jamesward on 19 Dec 2020

@jamesward this solution ??
Capture

BenyahyaF45 on 20 Dec 2020

@BenyahyaF45 This:

HttpClient.create().resolver(DefaultAddressResolverGroup.INSTANCE)

jamesward on 21 Dec 2020

@BenyahyaF45 I suggest to open Netty issue for the resolution problem.

violetagg on 22 Dec 2020

Hello @violetagg,

I had the same problem of DNS resolution when upgrading to Spring Boot 2.4 (and Netty 4.1.xx).
I fixed the issue with the workaround given by @jamesward

Example of stacktrace:

org.springframework.web.reactive.function.client.WebClientRequestException: failed to resolve 'XXXXXXXX' after 5 queries ; nested exception is java.net.UnknownHostException: failed to resolve 'XXXXXXX' after 5 queries 
    at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:137)
    at reactor.core.publisher.MonoErrorSupplied.subscribe(MonoErrorSupplied.java:70)
    at reactor.core.publisher.Mono.subscribe(Mono.java:4046)
    at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)
    at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221)
    at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221)
    at reactor.core.publisher.FluxMap$MapSubscriber.onError(FluxMap.java:132)
    at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221)
    at reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93)
    at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onError(MonoFlatMapMany.java:204)
    at reactor.core.publisher.SerializedSubscriber.onError(SerializedSubscriber.java:124)
    at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.whenError(FluxRetryWhen.java:224)
    at reactor.core.publisher.FluxRetryWhen$RetryWhenOtherSubscriber.onError(FluxRetryWhen.java:273)
    at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:413)
    at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onNext(FluxConcatMap.java:250)
    at reactor.core.publisher.EmitterProcessor.drain(EmitterProcessor.java:491)
    at reactor.core.publisher.EmitterProcessor.tryEmitNext(EmitterProcessor.java:299)
    at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:97)
    at reactor.core.publisher.InternalManySink.emitNext(InternalManySink.java:27)
    at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onError(FluxRetryWhen.java:189)
    at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:189)
    at reactor.netty.http.client.HttpClientConnect$MonoHttpConnect$ClientTransportSubscriber.onError(HttpClientConnect.java:306)
    at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:189)
    at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onError(DefaultPooledConnectionProvider.java:166)
    at reactor.netty.internal.shaded.reactor.pool.AbstractPool$Borrower.fail(AbstractPool.java:427)
    at reactor.netty.internal.shaded.reactor.pool.SimpleDequePool.lambda$drainLoop$5(SimpleDequePool.java:310)
    at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onError(FluxDoOnEach.java:186)
    at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:189)
    at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnectionAllocator$PooledConnectionInitializer.onError(DefaultPooledConnectionProvider.java:565)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:192)
    at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:259)
    at reactor.netty.transport.TransportConnector$MonoChannelPromise.tryFailure(TransportConnector.java:464)
    at reactor.netty.transport.TransportConnector.lambda$doResolveAndConnect$6(TransportConnector.java:271)
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571)
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
    at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
    at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
    at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:109)
    at io.netty.resolver.InetSocketAddressResolver$1.operationComplete(InetSocketAddressResolver.java:62)
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571)
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
    at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
    at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
    at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
    at io.netty.resolver.dns.DnsNameResolver.tryFailure(DnsNameResolver.java:936)
    at io.netty.resolver.dns.DnsNameResolver.access$500(DnsNameResolver.java:90)
    at io.netty.resolver.dns.DnsNameResolver$5.operationComplete(DnsNameResolver.java:956)
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
    at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
    at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
    at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
    at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1021)
    at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(DnsResolveContext.java:966)
    at io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:414)
    at io.netty.resolver.dns.DnsResolveContext.access$600(DnsResolveContext.java:63)
    at io.netty.resolver.dns.DnsResolveContext$2.operationComplete(DnsResolveContext.java:463)
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571)
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
    at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
    at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
    at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
    at io.netty.resolver.dns.DnsQueryContext.tryFailure(DnsQueryContext.java:225)
    at io.netty.resolver.dns.DnsQueryContext$4.run(DnsQueryContext.java:177)
    at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
    at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:164)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.UnknownHostException: failed to resolve 'XXXXXXX' after 5 queries 
    at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1013)
    ... 23 common frames omitted
Caused by: io.netty.resolver.dns.DnsNameResolverTimeoutException: [/x.x.x.x.x] query via UDP timed out after 5000 milliseconds (no stack trace available)

rschlick on 8 Jan 2021

👀1 🚀1

@rschlick Please create a new issue with a reproducible example.

violetagg on 8 Jan 2021

@violetagg It's hard to provide you an reproductible example. The DNS resolution fails sometimes, sometimes not.
We have this problem since Spring Boot 2.4 (and Netty 4.1.xx) and I think many people using Spring Boot will have this problem.
For now, WebClient is able to resolve DNS on my local Windows 10 machine, but fails in Docker container _adoptopenjdk:11.0.7_10-jre-hotspot_

rschlick on 21 Jan 2021

👍1

@rschlick Provide the full logs related to the failed resolution not only the exception, also please try to take a tcp dump.

violetagg on 21 Jan 2021

@violetagg Thanks for your help.

Depending of the URL endpoint (endpoint in public Internet or endpoint in the company network), it's could work or not.

Windows 10 local machine : DNS resolution OK for public internet endpoints / KO for internal company endpoints
Docker container linux-based: DNS resolution KO for public internet endpoints / OK for internal company endpoints

WebClient configuration

HttpClient httpClient = HttpClient.create(resourceFactory.getConnectionProvider()) .resolver(spec -> spec.trace("somelogger", LogLevel.DEBUG)) .runOn(resourceFactory.getLoopResources()) .option(CONNECT_TIMEOUT_MILLIS, (int) connectionTimeout.toMillis()) .doOnConnected((connection) -> connection.addHandlerLast(new ReadTimeoutHandler(readTimeout.toMillis(), MILLISECONDS)));

On Windows 10 (local machine) - Test KO for an internal company endpoint

`2021-01-21 12:01:15.092 DEBUG 9396 --- [ctor-http-nio-4] somelogger : from /x.x.x.x:53 : DefaultDnsQuestion(xxxxxxxx. IN A) failure

io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No matching record type found
at io.netty.resolver.dns.DnsResolveContext.onResponseAorAAAA(..)(Unknown Source)

2021-01-21 12:01:15.093 DEBUG 9396 --- [ctor-http-nio-4] somelogger : from /x.x.x.x:53 : DefaultDnsQuestion(xxxxxxxx. IN AAAA) failure

io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No matching record type found
at io.netty.resolver.dns.DnsResolveContext.onResponseAorAAAA(..)(Unknown Source)

2021-01-21 12:01:15.117 DEBUG 9396 --- [ctor-http-nio-4] somelogger : from /x.x.x.x:53 : DefaultDnsQuestion(xxxxxxxx. IN AAAA) no answer ServFail(2)
2021-01-21 12:01:15.137 DEBUG 9396 --- [ctor-http-nio-4] somelogger : from /x.x.x.x:53 : DefaultDnsQuestion(xxxxxxxx. IN AAAA) no answer ServFail(2)
2021-01-21 12:01:20.141 DEBUG 9396 --- [ctor-http-nio-4] somelogger : from /x.x.x.x:53 : DefaultDnsQuestion(xxxxxxxx. IN AAAA) failure

io.netty.resolver.dns.DnsNameResolverTimeoutException: [/x.x.x.x:53] query via UDP timed out after 5000 milliseconds (no stack trace available)

2021-01-21 12:01:25.145 DEBUG 9396 --- [ctor-http-nio-4] somelogger : from /x.x.x.x:53 : DefaultDnsQuestion(xxxxxxxx. IN AAAA) failure

io.netty.resolver.dns.DnsNameResolverTimeoutException: [/x.x.x.x:53] query via UDP timed out after 5000 milliseconds (no stack trace available)

2021-01-21 12:01:25.145 DEBUG 9396 --- [ctor-http-nio-4] somelogger : DefaultDnsQuestion(xxxxxxxx. IN AAAA) query never written and failed

io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name servers returned an answer
at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)

2021-01-21 12:01:25.192 ERROR 9396 --- [ctor-http-nio-4] LOGGER_STACK :
2021-01-21 12:01:25.193 ERROR 9396 --- [ctor-http-nio-4] s.o.c.w.iolog.LocalHttpRequestLogger : Error with HTTP request

org.springframework.web.reactive.function.client.WebClientRequestException: failed to resolve 'xxxxxxxx' after 6 queries ; nested exception is java.net.UnknownHostException: failed to resolve 'xxxxxxxx' after 6 queries
at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:137)
at reactor.core.publisher.MonoErrorSupplied.subscribe(MonoErrorSupplied.java:70)
at reactor.core.publisher.Mono.subscribe(Mono.java:4046)
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)
at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221)
at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221)
at reactor.core.publisher.FluxMap$MapSubscriber.onError(FluxMap.java:132)
at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221)
at reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93)
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onError(MonoFlatMapMany.java:204)
at reactor.core.publisher.SerializedSubscriber.onError(SerializedSubscriber.java:124)
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.whenError(FluxRetryWhen.java:224)
at reactor.core.publisher.FluxRetryWhen$RetryWhenOtherSubscriber.onError(FluxRetryWhen.java:273)
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:413)
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onNext(FluxConcatMap.java:250)
at reactor.core.publisher.EmitterProcessor.drain(EmitterProcessor.java:491)
at reactor.core.publisher.EmitterProcessor.tryEmitNext(EmitterProcessor.java:299)
at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:97)
at reactor.core.publisher.InternalManySink.emitNext(InternalManySink.java:27)
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onError(FluxRetryWhen.java:189)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:189)
at reactor.netty.http.client.HttpClientConnect$MonoHttpConnect$ClientTransportSubscriber.onError(HttpClientConnect.java:306)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:189)
at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onError(DefaultPooledConnectionProvider.java:166)
at reactor.netty.internal.shaded.reactor.pool.AbstractPool$Borrower.fail(AbstractPool.java:427)
at reactor.netty.internal.shaded.reactor.pool.SimpleDequePool.lambda$drainLoop$5(SimpleDequePool.java:309)
at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onError(FluxDoOnEach.java:186)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:189)
at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnectionAllocator$PooledConnectionInitializer.onError(DefaultPooledConnectionProvider.java:565)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:192)
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:259)
at reactor.netty.transport.TransportConnector$MonoChannelPromise.tryFailure(TransportConnector.java:464)
at reactor.netty.transport.TransportConnector.lambda$doResolveAndConnect$6(TransportConnector.java:271)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:109)
at io.netty.resolver.InetSocketAddressResolver$1.operationComplete(InetSocketAddressResolver.java:62)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
at io.netty.resolver.dns.DnsNameResolver.tryFailure(DnsNameResolver.java:936)
at io.netty.resolver.dns.DnsNameResolver.access$500(DnsNameResolver.java:90)
at io.netty.resolver.dns.DnsNameResolver$5.operationComplete(DnsNameResolver.java:956)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1021)
at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(DnsResolveContext.java:966)
at io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:414)
at io.netty.resolver.dns.DnsResolveContext.access$600(DnsResolveContext.java:63)
at io.netty.resolver.dns.DnsResolveContext$2.operationComplete(DnsResolveContext.java:463)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
at io.netty.resolver.dns.DnsQueryContext.tryFailure(DnsQueryContext.java:225)
at io.netty.resolver.dns.DnsQueryContext$4.run(DnsQueryContext.java:177)
at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.UnknownHostException: failed to resolve 'xxxxxxxx' after 6 queries
at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1013)
... 22 common frames omitted
Caused by: io.netty.resolver.dns.DnsNameResolverTimeoutException: [/x.x.x.x:53] query via UDP timed out after 5000 milliseconds (no stack trace available)

2021-01-21 12:01:25.199 WARN 9396 --- [ctor-http-nio-4] s.o.c.webclient.iolog.WiretapConnector : No match for Log-Request-Id=[4ca57941]
2021-01-21 12:01:25.199 DEBUG 9396 --- [ctor-http-nio-4] somelogger : DefaultDnsQuestion(xxxxxxxx. IN AAAA) query never written and failed

io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name servers returned an answer
at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)
`

On Docker container using adoptopenjdk:11.0.7_10-jre-hotspot - TEST KO for public Internet endpoint

{"@timestamp":"2021-01-21T14:16:14.922+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.vsp.svc.cluster.local. IN A) failure","log ger_name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No answe r found and NXDOMAIN response code returned\n\tat io.netty.resolver.dns.DnsResolveContext.onResponse(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:14.922+01:00","@version":"1","message":"DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.vsp.svc.cluster.local. IN A) query never written and failed","lo gger_name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name servers returned an answer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.013+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.vsp.svc.cluster.local. IN CNAME) failure", "logger_name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No a nswer found and NXDOMAIN response code returned\n\tat io.netty.resolver.dns.DnsResolveContext.onResponse(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.013+01:00","@version":"1","message":"DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.vsp.svc.cluster.local. IN CNAME) query never written and failed" ,"logger_name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name servers returned an answer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.014+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.svc.cluster.local. IN A) failure","logger_ name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No answer fo und and NXDOMAIN response code returned\n\tat io.netty.resolver.dns.DnsResolveContext.onResponse(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.014+01:00","@version":"1","message":"DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.svc.cluster.local. IN A) query never written and failed","logger _name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name ser vers returned an answer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.015+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.svc.cluster.local. IN CNAME) failure","log ger_name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No answe r found and NXDOMAIN response code returned\n\tat io.netty.resolver.dns.DnsResolveContext.onResponse(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.015+01:00","@version":"1","message":"DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.svc.cluster.local. IN CNAME) query never written and failed","lo gger_name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name servers returned an answer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.015+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.cluster.local. IN A) failure","logger_name ":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No answer found and NXDOMAIN response code returned\n\tat io.netty.resolver.dns.DnsResolveContext.onResponse(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.015+01:00","@version":"1","message":"DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.cluster.local. IN A) query never written and failed","logger_nam e":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name servers returned an answer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.016+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.cluster.local. IN CNAME) failure","logger_ name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No answer fo und and NXDOMAIN response code returned\n\tat io.netty.resolver.dns.DnsResolveContext.onResponse(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.016+01:00","@version":"1","message":"DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.cluster.local. IN CNAME) query never written and failed","logger _name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name ser vers returned an answer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.017+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.socrate.vsct.fr. IN A) failure","logger_na me":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No answer foun d and NXDOMAIN response code returned\n\tat io.netty.resolver.dns.DnsResolveContext.onResponse(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.017+01:00","@version":"1","message":"DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.socrate.vsct.fr. IN A) query never written and failed","logger_n ame":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name serve rs returned an answer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.017+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.socrate.vsct.fr. IN CNAME) failure","logge r_name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No answer found and NXDOMAIN response code returned\n\tat io.netty.resolver.dns.DnsResolveContext.onResponse(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.018+01:00","@version":"1","message":"DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.socrate.vsct.fr. IN CNAME) query never written and failed","logg er_name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name s ervers returned an answer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.018+01:00","@version":"1","message":"DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com.socrate.vsct.fr. IN CNAME) query never written and failed","logg er_name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name s ervers returned an answer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.019+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com. IN A) no answer ServFail(2)","logger_name ":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000} {"@timestamp":"2021-01-21T14:16:15.019+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com. IN A) failure","logger_name":"somelogger" ,"thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name servers returned an an swer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.020+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com. IN CNAME) no answer ServFail(2)","logger_ name":"somelogger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000} {"@timestamp":"2021-01-21T14:16:15.020+01:00","@version":"1","message":"from /x.x.x.x:53 : DefaultDnsQuestion(api.ouibus-acc.cloud.sqills.com. IN CNAME) failure","logger_name":"somelog ger","thread_name":"reactor-http-epoll-2","level":"DEBUG","level_value":10000,"stack_trace":"io.netty.resolver.dns.DnsResolveContext$DnsResolveContextException: No name servers returned a n answer\n\tat io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(..)(Unknown Source)\n"} {"@timestamp":"2021-01-21T14:16:15.113+01:00","@version":"1","message":"","logger_name":"LOGGER_STACK","thread_name":"reactor-http-epoll-2","level":"ERROR","level_value":40000,"avro":{"ti mestamp8601": "2021-01-21T14:16:15.113+0100", "correlationId": "", "requestId": "", "version": "1.148.0-SNAPSHOT", "instance": "cart-services-cart-2895-default-dns", "logMessage": "Error with HTTP request", "exceptionToString": "org.springframework.web.reactive.function.client.WebClientRequestException: failed to resolve 'api.ouibus-acc.cloud.sqills.com' after 2 queries ; nested exception is java.net.UnknownHostException: failed to resolve 'api.ouibus-acc.cloud.sqills.com' after 2 queries", "stack1stLine": "org.springframework.web.reactive.function.client .ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:137)", "causeException": "java.net.UnknownHostException: failed to resolve 'api.ouibus-acc.cloud.s qills.com' after 2 queries ", "causeStack1stLine": "io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1013)", "fullStack": "org.springframework.web.reactive.fun ction.client.WebClientRequestException: failed to resolve 'api.ouibus-acc.cloud.sqills.com' after 2 queries ; nested exception is java.net.UnknownHostException: failed to resolve 'api.oui bus-acc.cloud.sqills.com' after 2 queries \n\tat org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:1 37)\n\tat reactor.core.publisher.MonoErrorSupplied.subscribe(MonoErrorSupplied.java:70)\n\tat reactor.core.publisher.Mono.subscribe(Mono.java:4046)\n\tat reactor.core.publisher.FluxOnErro rResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)\n\tat reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221)\n\tat reactor.core.publisher.FluxPeek$PeekSubs criber.onError(FluxPeek.java:221)\n\tat reactor.core.publisher.FluxMap$MapSubscriber.onError(FluxMap.java:132)\n\tat reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:2 21)\n\tat reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93)\n\tat reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onError(MonoFlatMapMany.java:204)\n\tat rea ctor.core.publisher.SerializedSubscriber.onError(SerializedSubscriber.java:124)\n\tat reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.whenError(FluxRetryWhen.java:224)\n\tat reactor.core.publisher.FluxRetryWhen$RetryWhenOtherSubscriber.onError(FluxRetryWhen.java:273)\n\tat reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:413)\n \tat reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onNext(FluxConcatMap.java:250)\n\tat reactor.core.publisher.EmitterProcessor.drain(EmitterProcessor.java:491)\n\tat reactor.co re.publisher.EmitterProcessor.tryEmitNext(EmitterProcessor.java:299)\n\tat reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:97)\n\tat reactor.core.publisher.I nternalManySink.emitNext(InternalManySink.java:27)\n\tat reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onError(FluxRetryWhen.java:189)\n\tat reactor.core.publisher.MonoCrea te$DefaultMonoSink.error(Mon

rschlick on 21 Jan 2021

Hello @violetagg

Is it possible to have an help on the DNS resolution issue ?
I have provided the stacktraces above.

Thank you.

rschlick on 8 Mar 2021

@rschlick I'm sorry somehow I missed that.
As this issue is closed and related to a documentation extension, it is not appropriate to continue the discussion here.
Please open a new issue.

violetagg on 8 Mar 2021

@violetagg No problem. Ok I will open another issue.

rschlick on 8 Mar 2021

👀1

@rschlick thanks

violetagg on 8 Mar 2021

Issue still remains in the latest 4.1.60 RELEASE of Netty. @jamesward's solution can be used until it get fixed.

HttpClient httpClient = HttpClient.create(provider).resolver(DefaultAddressResolverGroup.INSTANCE);

hdineth on 31 Mar 2021

thank you i resolve my problem

BenyahyaF45 on 2 Apr 2021

@rschlick

No problem. Ok I will open another issue.

Have you opened a new issue?

I'm facing the same problem after upgrading my Spring Boot Admin application to Spring Boot 2.4.3 (with Netty 4.1.59 and reactor-netty 1.0.4 on board). The suggested workaround is not applicable for me as the HttpClient is instantiated somewhere inside the library and is out of my control.

Toparvion on 19 Apr 2021

👍2

@Toparvion This is the issue #1534. It revealed two issues

https://github.com/netty/netty/issues/9796, a fix is in Java >=15
https://github.com/netty/netty/issues/11142, I'm working on it

violetagg on 19 Apr 2021

👍2

@violetagg , thank you for the links!

Meanwhile I've managed to apply the suggested workaround for Spring Boot Admin (2.4.1) by adding this single bean:

@Component
@RequiredArgsConstructor(onConstructor = @__(@Autowired))
public class DnsFixingWebClientCustomizer implements InstanceWebClientCustomizer {

  private final WebClient.Builder springWebClientBuilder;

  @Override
  public void customize(InstanceWebClient.Builder builder) {
    var reactorHttpClient = HttpClient.create().resolver(DefaultAddressResolverGroup.INSTANCE);
    springWebClientBuilder.clientConnector(new ReactorClientHttpConnector(reactorHttpClient));
    builder.webClient(springWebClientBuilder);
  }
}

Definitely not the best solution ever but hopefully useful for someone else.

Toparvion on 20 Apr 2021

👍1

While version 4.1.64.Final with #11142 fix on board is not published, here's one more workaround, this time for Spring Cloud Gateway.
Just add the following bean into your Spring context:

@Component
public class DnsResolutionFixer implements HttpClientCustomizer {
  @Override
  public HttpClient customize(HttpClient httpClient) {
    return httpClient.resolver(DefaultAddressResolverGroup.INSTANCE);
  }
}

This will be automatically picked up and used in org.springframework.cloud.gateway.config.GatewayAutoConfiguration.NettyConfiguration#gatewayHttpClient (see the very last if condition).

Toparvion on 4 May 2021

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Add option for changing the max URI tags

violetagg · 8Comments

Source line numbrers are outdated in reference documentation

izeye · 3Comments

Allow to set custom web-socket maxFramePayloadLength

EdgarsM · 7Comments

Support Graceful Shutdown

violetagg · 4Comments

Deprecation notices are not clear

vchekan · 8Comments