testcontainers-java 🚀 - Intermittent couchbase failures in CI

Example build failures (not sure if these will be overwritten when we re-run tests):
https://circleci.com/gh/testcontainers/testcontainers-java/12930
https://circleci.com/gh/testcontainers/testcontainers-java/12796
https://circleci.com/gh/testcontainers/testcontainers-java/12791

rnorth on 7 May 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

stale[bot] on 5 Aug 2019

This continues to be a problem; our tests fail even if retried. This gives me some doubt that the module is actually stable enough for real use.

I think we're going to have to have to reach out to Couchbase/Couchbase community for help. In the worst case scenario, if we can't fix it, I'm inclined to remove this module.

rnorth on 13 Oct 2019

👍1

@rnorth : Just for the record, we've observed problems with couchbase testcontainers as well; https://github.com/jhipster/generator-jhipster/issues/10594

SudharakaP on 13 Oct 2019

@SudharakaP thanks for confirming, and I’m sorry to hear that.

It sounds like a couple of Couchbase experts may be willing to lend a hand - hopefully there’s a straightforward solution so that we can make this module more reliable. Fingers crossed!

_{Sent with GitHawk}

rnorth on 14 Oct 2019

👍1

Couchbase employee here, we will investigate this issue and come back to you as soon as possible.

deniswsrosa on 14 Oct 2019

❤5 👍5

@rnorth : You are welcome. :smile:

@deniswsrosa : Just for the record, I was wondering if this issue is similar to https://github.com/testcontainers/testcontainers-java/issues/715 where the startup timeout in Couchbase is 120 being inadequate. I was trying to create a custom wait strategy to test it out, but couldn't get it working yet (due to my lack of knowledge about Couchbase probably) :smile:

SudharakaP on 14 Oct 2019

@rnorth : You are welcome. 😄

@deniswsrosa : Just for the record, I was wondering if this issue is similar to #715 where the startup timeout in Couchbase is 120 being inadequate. I was trying to create a custom wait strategy to test it out, but couldn't get it working yet (due to my lack of knowledge about Couchbase probably) 😄

thanks for the heads up, I will ping this back to the team.

deniswsrosa on 15 Oct 2019

👍1

Some additional information, if it can help.
In JHipster project, we encounter the similar issue as mentionned by @SudharakaP.

In our CI, it works perfectly well with OpenJDK11 :

      echo '*** Using OpenJDK 11'
      sudo add-apt-repository ppa:openjdk-r/ppa
      sudo apt-get update
      sudo apt-get install -y openjdk-11-jdk
      sudo update-java-alternatives -s java-1.11.0-openjdk-amd64
      java -version

But, it failed randomly when using OpenJDK8:

      echo '*** Using OpenJDK 8'
      sudo add-apt-repository ppa:openjdk-r/ppa
      sudo apt-get update
      sudo apt-get install -y openjdk-8-jdk
      sudo update-java-alternatives -s java-1.8.0-openjdk-amd64
      java -version

pascalgrimaud on 15 Oct 2019

👍1

Are there any alternatives while we wait for a solution?

enbrent on 15 Oct 2019

Trying to triage this a bit. Am I correct that the initial exception reported in this ticket is different from the failing runs linked? In the linked run I see:

Caused by:
org.testcontainers.containers.ContainerLaunchException: Timed out waiting for URL to be accessible (http://localhost:32934/ui/index.html should return HTTP 200)

while this ticket reports

shouldInsertDocument - org.testcontainers.couchbase.Couchbase4_6Test
com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight.
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:686)
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handlerRemoved(AbstractGenericHandler.java:667)
    at

The first one looks like the server is not coming up in time until it stops waiting, the second one is completely different: the SDK sent a request into the cluster but while the request was in-flight on the socket the socket got closed- and as a result the client has no other option as to cancel the request.

daschl on 17 Oct 2019

👍1

We get a mix of failures, but com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight is by far the most prevalent. If we could only fix one issue, it would be this one.

It occurs randomly for some % of CI builds and local builds, but unlike other random failures it is not helped at all by retries within a build (we have a JUnit rule to retry specific failed tests that we think are 'flaky').

This makes me think that there's some statefulness at play that's causing one failure to cascade and pollute subsequent tests. I was unable to work out why when I investigated this back in May.

rnorth on 18 Oct 2019

👍1

@rnorth one thing I was thinking, since it happens some % - maybe it would work if we increase the poll wait time up from one minute? It might at least be worth a try.

The request cancellations are very likely a side effect of a problem, if the network is healthy then I suspect we are connecting to that server port and it's not ready yet (kinda indicating polling longer waiting until it comes online too).

Are you able to reproduce this locally too or do you only see it in CI?

daschl on 18 Oct 2019

👍1

@daschl : In our case we only see this on CI; but I am starting to think the error in our end might not be exactly same as in this issue. We see;

{"app_name":"travisCouchbaseNoCache","app_port":"-1","level":"WARN","logger_name":"o.t.u.RegistryAuthLocator","message":"Failure when attempting to lookup auth config (dockerImageName: couchbase:6.0.0, configFile: /home/runner/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /home/runner/.docker/config.json (No such file or directory)","thread_name":"main","timestamp":"2019-10-09T18:12:52.726Z"}
{"app_name":"travisCouchbaseNoCache","app_port":"-1","level":"INFO","logger_name":"o.t.c.w.s.HttpWaitStrategy","message":"/lucid_mcnulty: Waiting for 60 seconds for URL: http://localhost:32782/ui/index.html","thread_name":"main","timestamp":"2019-10-09T18:12:53.812Z"}
{"app_name":"travisCouchbaseNoCache","app_port":"-1","level":"INFO","logger_name":"o.t.c.w.s.HttpWaitStrategy","message":"/lucid_mcnulty: Waiting for 60 seconds for URL: http://localhost:32782/pools/default/","thread_name":"main","timestamp":"2019-10-09T18:13:17.845Z"}
{"app_name":"travisCouchbaseNoCache","app_port":"-1","level":"WARN","logger_name":"c.c.c.c.e.Endpoint","message":"[localhost:32775][KeyValueEndpoint]: Authentication Failure: Bucket not found on Select Bucket command","thread_name":"cb-io-1-2","timestamp":"2019-10-09T18:13:21.245Z"}
{"app_name":"travisCouchbaseNoCache","app_port":"-1","level":"INFO","logger_name":"o.t.c.CouchbaseQueryServiceWaitStrategy","message":"Waiting for 120 seconds for QUERY service","thread_name":"main","timestamp":"2019-10-09T18:13:21.387Z"}

before the timeout; there's no other exceptions. Here's a typical run; https://github.com/jhipster/generator-jhipster/pull/10498/checks?check_run_id=254018117

I was also wondering whether our issue is similar to https://github.com/testcontainers/testcontainers-java/issues/715 where the startup timeout in Couchbase is 120 being inadequate.

Let me know if you need me to create a new issue for this one since I don't want to pollute this thread. 😄

SudharakaP on 24 Oct 2019

~I _might_ have a fix for this in #2076. This involved an incredibly painful amount of trial and error 😭.~

Sorry, I was wrong - #2076 might improve some aspects of resilience, but there are still numerous failures when running on slower machines.

rnorth on 16 Nov 2019

❤1 🎉1

With the couchbase-resilience branch on an old machine, I'm getting the following logs (long snippet): https://gist.github.com/rnorth/f61637b4be934d3745ecb35d28961f5b

It's interesting that these failures occur waiting for a (supposedly created) bucket to appear. This failure looks different to other failures we're seeing on CI.

rnorth on 16 Nov 2019

👍1

One of the important aspects is that the Couchbase Server HTTP API is inherently asynchronous, so even if you get a response doesn't mean it is immediately available. I wonder if it makes sense to define the explicit endpoints / "stages" the container code triggers and then I can go hunting for the right polling logic to see if it is actually in a stage to move on and declare it "done".

daschl on 16 Nov 2019

👍1

FYI I submitted #2081, it seems that there was a faulty execStart that did not wait for the completion.

While working on it, I also discovered that if we remove execStart (which means that the ports will not be proxied) then the client fails with "request cancelled inflight".
@daschl I have a feeling that the error message is wrong, because if the destination port cannot be opened, it should clearly state that since there wasn't any request to send

bsideup on 18 Nov 2019

🎉1

@bsideup the way the SDK logic is implemented is that if you get a "request cancelled inflight" it actually has been written into netty already, so the socket must've been opened at some point. Otherwise the op would time out because we couldn't find a socket to write it to.

We are either cancelling if the handler is removed or we cought an exception on the channel (https://github.com/couchbase/couchbase-jvm-core/blob/master/src/main/java/com/couchbase/client/core/endpoint/AbstractGenericHandler.java#L675) in both cases, the netty channel must've been open at some point. That said, there should be netty warn logs around this timeframe in the logs which should tell us more about the explicit reason.

daschl on 18 Nov 2019

😬 reopening - we still don't have a fix yet.

I've been running a fairly stupid series of scripted tests to try and at least narrow down where the race is occurring, injecting sleep into various locations to see if it reduces the failure rate. So far (400+ runs exercising 14 different sleep locations plus @bsideup's change), none has been flawless. The most promising, a 10s sleep just before bucket creation (containerIsStarted), reduced the failure rate to a mere 4%, but it's obviously not a fix either. We seem to have a concurrency bug that we're not seeing, or more than one bug conspiring together.

I'd be happy to share the 100s of MBs of logs I've gathered during this exercise 😂. One of the more pertinent sets of logs, from the most promising test case, are below:

Click to expand...

08:31:38.376 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Starting polling with interval 2500ms
08:31:38.382 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Setting seed hosts to [localhost]
08:31:38.383 TRACE org.testcontainers.couchbase.CouchbaseContainer - Leaving createCouchbaseCluster
08:31:38.383 TRACE org.testcontainers.couchbase.CouchbaseContainer - Inserting bucket
08:31:38.384 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Node localhost
08:31:38.385 DEBUG com.couchbase.client.core.RequestHandler - Connecting Node localhost
08:31:38.385 DEBUG com.couchbase.client.core.node.Node - [localhost]: Got instructed to connect.
08:31:38.385 DEBUG com.couchbase.client.core.RequestHandler - Connect finished, registering for use.
08:31:38.385 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service CONFIG, to Node localhost
08:31:38.386 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service CONFIG
08:31:38.387 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service CONFIG to registry and connecting it.
08:31:38.387 DEBUG com.couchbase.client.core.service.Service - [localhost][ConfigService]: Got instructed to connect.
08:31:38.388 DEBUG com.couchbase.client.core.endpoint.Endpoint - Using a connectCallbackGracePeriod of 2000 on Endpoint localhost:42653
08:31:38.390 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (IDLE) from Node
08:31:38.393 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42653][ConfigEndpoint]: Connected Endpoint.
08:31:38.394 INFO  com.couchbase.client.core.node.Node - Connected to Node localhost
08:31:38.395 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (CONNECTING) to Node
08:31:38.398 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42653][ConfigEndpoint]: Channel Active.
08:31:38.440 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42653][ConfigEndpoint]: Disconnected Endpoint.
08:31:38.440 INFO  com.couchbase.client.core.node.Node - Disconnected from Node localhost
08:31:38.440 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (DISCONNECTING) from Node
08:31:38.441 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42653][ConfigEndpoint]: Channel Inactive.
08:31:38.442 DEBUG com.couchbase.client.core.endpoint.Endpoint - Using a connectCallbackGracePeriod of 2000 on Endpoint localhost:42653
08:31:38.443 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (IDLE) from Node
08:31:38.445 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42653][ConfigEndpoint]: Connected Endpoint.
08:31:38.446 INFO  com.couchbase.client.core.node.Node - Connected to Node localhost
08:31:38.446 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (CONNECTING) to Node
08:31:38.449 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42653][ConfigEndpoint]: Channel Active.
08:31:38.481 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42653][ConfigEndpoint]: Disconnected Endpoint.
08:31:38.481 INFO  com.couchbase.client.core.node.Node - Disconnected from Node localhost
08:31:38.481 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (DISCONNECTING) from Node
08:31:38.482 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42653][ConfigEndpoint]: Channel Inactive.
08:31:38.482 DEBUG com.couchbase.client.core.RequestHandler - Node localhost already registered, skipping.
08:31:38.483 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service CONFIG, to Node localhost
08:31:38.484 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service CONFIG
08:31:38.484 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service CONFIG already added, skipping.
08:31:38.484 DEBUG com.couchbase.client.core.endpoint.Endpoint - Using a connectCallbackGracePeriod of 2000 on Endpoint localhost:42653
08:31:38.485 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (IDLE) from Node
08:31:38.487 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42653][ConfigEndpoint]: Connected Endpoint.
08:31:38.488 INFO  com.couchbase.client.core.node.Node - Connected to Node localhost
08:31:38.488 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (CONNECTING) to Node
08:31:38.491 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42653][ConfigEndpoint]: Channel Active.
08:31:38.541 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42653][ConfigEndpoint]: Disconnected Endpoint.
08:31:38.541 INFO  com.couchbase.client.core.node.Node - Disconnected from Node localhost
08:31:38.542 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (DISCONNECTING) from Node
08:31:38.542 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42653][ConfigEndpoint]: Channel Inactive.
08:31:38.649 TRACE org.testcontainers.couchbase.CouchbaseContainer - Inserting bucket user
08:31:38.650 DEBUG com.couchbase.client.core.RequestHandler - Node localhost already registered, skipping.
08:31:38.650 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service CONFIG, to Node localhost
08:31:38.651 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service CONFIG
08:31:38.651 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service CONFIG already added, skipping.
08:31:38.651 DEBUG com.couchbase.client.core.endpoint.Endpoint - Using a connectCallbackGracePeriod of 2000 on Endpoint localhost:42653
08:31:38.651 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (IDLE) from Node
08:31:38.653 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42653][ConfigEndpoint]: Connected Endpoint.
08:31:38.654 INFO  com.couchbase.client.core.node.Node - Connected to Node localhost
08:31:38.654 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (CONNECTING) to Node
08:31:38.657 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42653][ConfigEndpoint]: Channel Active.
08:31:38.677 WARN  com.couchbase.client.core.endpoint.ResponseStatusConverter - Unknown ResponseStatus with Protocol HTTP: 405
08:31:38.678 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42653][ConfigEndpoint]: Disconnected Endpoint.
08:31:38.678 INFO  com.couchbase.client.core.node.Node - Disconnected from Node localhost
08:31:38.678 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (DISCONNECTING) from Node
08:31:38.678 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42653][ConfigEndpoint]: Channel Inactive.
08:31:38.679 WARN  🐳 [couchbase/server:4.6.5] - Unable to insert user 'test', maybe you are using older version
08:31:38.679 TRACE org.testcontainers.couchbase.CouchbaseContainer - Opening bucket
08:31:38.679 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Got instructed to open bucket test
08:31:38.679 DEBUG com.couchbase.client.core.config.loader.Loader - Loading Config for bucket test
08:31:38.680 DEBUG com.couchbase.client.core.config.loader.Loader - Loading Config for bucket test
08:31:38.680 DEBUG com.couchbase.client.core.RequestHandler - Node localhost already registered, skipping.
08:31:38.680 DEBUG com.couchbase.client.core.config.loader.Loader - Successfully added Node localhost
08:31:38.680 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service BINARY, to Node localhost
08:31:38.680 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service BINARY
08:31:38.681 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (IDLE) from Node
08:31:38.681 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service BINARY to registry and connecting it.
08:31:38.681 DEBUG com.couchbase.client.core.service.Service - [localhost][KeyValueService]: Got instructed to connect.
08:31:38.681 DEBUG com.couchbase.client.core.endpoint.Endpoint - Using a connectCallbackGracePeriod of 2000 on Endpoint localhost:42646
08:31:38.681 DEBUG com.couchbase.client.core.service.Service - [localhost][KeyValueService]: New number of endpoints is 1
08:31:38.681 DEBUG com.couchbase.client.core.service.Service - [localhost][KeyValueService]: Connecting Endpoint during Service connect.
08:31:38.684 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Requesting supported features: [TCPNODELAY, XATTR, SELECT_BUCKET, TRACING, SNAPPY, XERROR]
08:31:38.697 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Negotiated supported features: [TCPNODELAY]
08:31:38.767 WARN  com.couchbase.client.core.endpoint.Endpoint - [localhost:42646][KeyValueEndpoint]: Authentication Failure: Authentication Failure
08:31:38.768 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (CONNECTING) from Node
08:31:38.769 DEBUG com.couchbase.client.core.config.loader.Loader - Could not add service on localhost because of {}, removing it again.
com.couchbase.client.core.endpoint.kv.AuthenticationException: Authentication Failure
    at com.couchbase.client.core.endpoint.kv.KeyValueAuthHandler.checkIsAuthed(KeyValueAuthHandler.java:299)
    at com.couchbase.client.core.endpoint.kv.KeyValueAuthHandler.channelRead0(KeyValueAuthHandler.java:180)
    at com.couchbase.client.core.endpoint.kv.KeyValueAuthHandler.channelRead0(KeyValueAuthHandler.java:53)
    at com.couchbase.client.deps.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
    at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
    at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
    at com.couchbase.client.deps.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312)
    at com.couchbase.client.deps.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286)
    at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
    at com.couchbase.client.deps.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921)
    at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:135)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:546)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:500)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
08:31:38.770 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Channel Inactive.
08:31:38.770 INFO  com.couchbase.client.core.endpoint.Endpoint - [localhost:42646][KeyValueEndpoint]: Got notified from Channel as inactive, attempting reconnect.
08:31:38.771 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to remove Service BINARY, from Node localhost
08:31:38.771 DEBUG com.couchbase.client.core.node.Node - [localhost]: Removing Service BINARY
08:31:38.771 DEBUG com.couchbase.client.core.service.Service - [localhost][KeyValueService]: Got instructed to disconnect.
08:31:38.771 DEBUG com.couchbase.client.core.service.Service - [localhost][KeyValueService]: New number of endpoints is 1
08:31:38.773 DEBUG com.couchbase.client.core.service.Service - [localhost][KeyValueService]: Disconnecting Endpoint during Service disconnect.
08:31:38.773 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Received signal to proactively refresh (a maybe outdated) configuration.
08:31:38.773 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Ignoring outdated signal, since no buckets are open.
08:31:38.775 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Requesting supported features: [TCPNODELAY, XATTR, SELECT_BUCKET, TRACING, SNAPPY, XERROR]
08:31:38.781 DEBUG com.couchbase.client.core.RequestHandler - Node localhost already registered, skipping.
08:31:38.781 DEBUG com.couchbase.client.core.config.loader.Loader - Successfully added Node localhost
08:31:38.781 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service CONFIG, to Node localhost
08:31:38.782 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service CONFIG
08:31:38.782 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service CONFIG already added, skipping.
08:31:38.782 DEBUG com.couchbase.client.core.config.loader.Loader - Successfully enabled Service CONFIG on Node localhost
08:31:38.782 DEBUG com.couchbase.client.core.config.loader.HttpLoader - Starting to discover config through HTTP Bootstrap
08:31:38.783 DEBUG com.couchbase.client.core.endpoint.Endpoint - Using a connectCallbackGracePeriod of 2000 on Endpoint localhost:42653
08:31:38.783 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (IDLE) from Node
08:31:38.789 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42653][ConfigEndpoint]: Connected Endpoint.
08:31:38.799 INFO  com.couchbase.client.core.node.Node - Connected to Node localhost
08:31:38.799 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (CONNECTING) to Node
08:31:38.808 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42653][ConfigEndpoint]: Channel Active.
08:31:38.838 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Negotiated supported features: [TCPNODELAY]
08:31:38.882 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42653][ConfigEndpoint]: Disconnected Endpoint.
08:31:38.882 INFO  com.couchbase.client.core.node.Node - Disconnected from Node localhost
08:31:38.882 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (DISCONNECTING) from Node
08:31:38.882 DEBUG com.couchbase.client.core.config.loader.HttpLoader - Successfully got config from terse bucket remote.
08:31:38.882 DEBUG com.couchbase.client.core.config.loader.HttpLoader - Successfully loaded config through HTTP.
08:31:38.882 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42653][ConfigEndpoint]: Channel Inactive.
08:31:38.883 DEBUG com.couchbase.client.core.config.loader.Loader - Got configuration from Service, attempting to parse.
08:31:38.884 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Loaded from loader HTTP, can fetch from carrier true
08:31:38.884 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Registering bucket test for refresh at CarrierRefresher
08:31:38.884 DEBUG com.couchbase.client.core.config.refresher.Refresher - Registering Bucket test for refresh.
08:31:38.884 INFO  com.couchbase.client.core.config.ConfigurationProvider - Selected network configuration: default
08:31:38.885 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Applying new configuration DefaultCouchbaseBucketConfig{name='test', locator=VBUCKET, uri='/pools/default/buckets/test?bucket_uuid=496a364a4f366fec00ee476996d61dc8', streamingUri='/pools/default/bucketsStreaming/test?bucket_uuid=496a364a4f366fec00ee476996d61dc8', nodeInfo=[DefaultNodeInfo{, hostname=localhost, directServices={VIEW=42652, CONFIG=8091, BINARY=42646}, sslServices={VIEW=42644, CONFIG=18091, BINARY=42647}, alternateAddresses={}, configPort=0, useAlternateNetwork=null}], partitionInfo=PartitionInfo{numberOfReplicas=0, partitionHosts=[localhost:42646], partitions=[], tainted=false}, tainted=false, rev=13}
08:31:38.885 DEBUG com.couchbase.client.core.RequestHandler - Got notified of a new configuration arriving.
08:31:38.885 DEBUG com.couchbase.client.core.RequestHandler - Starting reconfiguration.
08:31:38.885 DEBUG com.couchbase.client.core.RequestHandler - Starting reconfiguration for bucket test
08:31:38.885 DEBUG com.couchbase.client.core.RequestHandler - Node localhost already registered, skipping.
08:31:38.885 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service VIEW, to Node localhost
08:31:38.885 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service VIEW
08:31:38.886 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service VIEW to registry and connecting it.
08:31:38.886 DEBUG com.couchbase.client.core.service.Service - [localhost][ViewService]: Got instructed to connect.
08:31:38.886 DEBUG com.couchbase.client.core.service.Service - [localhost][ViewService]: No endpoints needed to connect, skipping.
08:31:38.886 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service CONFIG, to Node localhost
08:31:38.886 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service CONFIG
08:31:38.886 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service CONFIG already added, skipping.
08:31:38.886 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service BINARY, to Node localhost
08:31:38.886 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service BINARY
08:31:38.886 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (IDLE) from Node
08:31:38.886 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service BINARY to registry and connecting it.
08:31:38.887 DEBUG com.couchbase.client.core.service.Service - [localhost][KeyValueService]: Got instructed to connect.
08:31:38.887 DEBUG com.couchbase.client.core.endpoint.Endpoint - Using a connectCallbackGracePeriod of 2000 on Endpoint localhost:42646
08:31:38.887 DEBUG com.couchbase.client.core.service.Service - [localhost][KeyValueService]: New number of endpoints is 1
08:31:38.888 DEBUG com.couchbase.client.core.service.Service - [localhost][KeyValueService]: Connecting Endpoint during Service connect.
08:31:38.889 INFO  com.couchbase.client.core.config.ConfigurationProvider - Opened bucket test
08:31:38.889 DEBUG com.couchbase.client.core.RequestHandler - Starting reconfiguration.
08:31:38.889 DEBUG com.couchbase.client.core.RequestHandler - Starting reconfiguration for bucket test
08:31:38.889 DEBUG com.couchbase.client.core.RequestHandler - Node localhost already registered, skipping.
08:31:38.890 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service VIEW, to Node localhost
08:31:38.891 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service VIEW
08:31:38.891 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service VIEW already added, skipping.
08:31:38.891 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service CONFIG, to Node localhost
08:31:38.891 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service CONFIG
08:31:38.891 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service CONFIG already added, skipping.
08:31:38.891 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service BINARY, to Node localhost
08:31:38.891 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service BINARY
08:31:38.891 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service BINARY already added, skipping.
08:31:38.893 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Requesting supported features: [TCPNODELAY, XATTR, SELECT_BUCKET, TRACING, SNAPPY, XERROR]
08:31:38.894 TRACE org.testcontainers.couchbase.CouchbaseContainer - Waiting until ready
08:31:38.894 INFO  org.testcontainers.couchbase.CouchbaseQueryServiceWaitStrategy - Waiting for 120 seconds for QUERY service
08:31:38.917 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Negotiated supported features: [TCPNODELAY]
08:31:38.970 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost:42646][KeyValueEndpoint]: Endpoint connect completed, but got instructed to disconnect in the meantime.
08:31:38.970 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Channel Inactive.
08:31:39.025 WARN  com.couchbase.client.core.endpoint.Endpoint - [localhost:42646][KeyValueEndpoint]: Authentication Failure: Authentication Failure
08:31:39.026 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (CONNECTING) from Node
08:31:39.026 WARN  com.couchbase.client.core.RequestHandler - Received Error during Reconfiguration.
com.couchbase.client.core.endpoint.kv.AuthenticationException: Authentication Failure
    at com.couchbase.client.core.endpoint.kv.KeyValueAuthHandler.checkIsAuthed(KeyValueAuthHandler.java:299)
    at com.couchbase.client.core.endpoint.kv.KeyValueAuthHandler.channelRead0(KeyValueAuthHandler.java:180)
    at com.couchbase.client.core.endpoint.kv.KeyValueAuthHandler.channelRead0(KeyValueAuthHandler.java:53)
    at com.couchbase.client.deps.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
    at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
    at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
    at com.couchbase.client.deps.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312)
    at com.couchbase.client.deps.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286)
    at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
    at com.couchbase.client.deps.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1304)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:921)
    at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:135)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:546)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:500)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
08:31:39.033 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Requesting supported features: [TCPNODELAY, XATTR, SELECT_BUCKET, TRACING, SNAPPY, XERROR]
08:31:39.033 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Channel Inactive.
08:31:39.033 INFO  com.couchbase.client.core.endpoint.Endpoint - [localhost:42646][KeyValueEndpoint]: Got notified from Channel as inactive, attempting reconnect.
08:31:39.033 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Received signal to proactively refresh (a maybe outdated) configuration.
08:31:39.053 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Negotiated supported features: [TCPNODELAY]
08:31:39.092 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Connected Endpoint.
08:31:39.092 INFO  com.couchbase.client.core.node.Node - Connected to Node localhost
08:31:39.092 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (CONNECTING) to Node
08:31:39.093 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Channel Active.
08:31:39.123 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Channel Inactive.
08:31:39.123 INFO  com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Got notified from Channel as inactive, attempting reconnect.
08:31:39.123 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Received signal to proactively refresh (a maybe outdated) configuration.
08:31:39.123 INFO  com.couchbase.client.core.node.Node - Disconnected from Node localhost
08:31:39.124 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (CONNECTED) from Node
08:31:39.125 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Cancelling 1 outstanding requests.
08:31:39.127 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Requesting supported features: [TCPNODELAY, XATTR, SELECT_BUCKET, TRACING, SNAPPY, XERROR]
08:31:39.130 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Could not fetch config from bucket "test" against "localhost".
com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight.
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:686)
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handlerRemoved(AbstractGenericHandler.java:667)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:626)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyDown(DefaultChannelPipeline.java:878)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyUp(DefaultChannelPipeline.java:844)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroy(DefaultChannelPipeline.java:836)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.access$700(DefaultChannelPipeline.java:44)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelUnregistered(DefaultChannelPipeline.java:1286)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:176)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:162)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:821)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:776)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
08:31:39.130 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Error while refreshing bucket config, ignoring.
com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight.
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:686)
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handlerRemoved(AbstractGenericHandler.java:667)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:626)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyDown(DefaultChannelPipeline.java:878)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyUp(DefaultChannelPipeline.java:844)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroy(DefaultChannelPipeline.java:836)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.access$700(DefaultChannelPipeline.java:44)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelUnregistered(DefaultChannelPipeline.java:1286)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:176)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:162)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:821)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:776)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
08:31:39.145 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Negotiated supported features: [TCPNODELAY]
08:31:39.189 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Connected Endpoint.
08:31:39.189 INFO  com.couchbase.client.core.node.Node - Connected to Node localhost
08:31:39.189 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (CONNECTING) to Node
08:31:39.189 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Channel Active.
08:31:39.205 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Channel Inactive.
08:31:39.206 INFO  com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Got notified from Channel as inactive, attempting reconnect.
08:31:39.206 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Received signal to proactively refresh (a maybe outdated) configuration.
08:31:39.206 INFO  com.couchbase.client.core.node.Node - Disconnected from Node localhost
08:31:39.206 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (CONNECTED) from Node
08:31:39.207 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Cancelling 1 outstanding requests.
08:31:39.208 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Could not fetch config from bucket "test" against "localhost".
com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight.
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:686)
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handlerRemoved(AbstractGenericHandler.java:667)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:626)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyDown(DefaultChannelPipeline.java:878)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyUp(DefaultChannelPipeline.java:844)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroy(DefaultChannelPipeline.java:836)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.access$700(DefaultChannelPipeline.java:44)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelUnregistered(DefaultChannelPipeline.java:1286)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:176)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:162)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:821)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:776)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
08:31:39.209 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Error while refreshing bucket config, ignoring.
com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight.
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:686)
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handlerRemoved(AbstractGenericHandler.java:667)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:626)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyDown(DefaultChannelPipeline.java:878)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyUp(DefaultChannelPipeline.java:844)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroy(DefaultChannelPipeline.java:836)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.access$700(DefaultChannelPipeline.java:44)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelUnregistered(DefaultChannelPipeline.java:1286)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:176)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:162)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:821)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:776)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
08:31:39.210 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Requesting supported features: [TCPNODELAY, XATTR, SELECT_BUCKET, TRACING, SNAPPY, XERROR]
08:31:39.226 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Negotiated supported features: [TCPNODELAY]
08:31:39.261 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Connected Endpoint.
08:31:39.262 INFO  com.couchbase.client.core.node.Node - Connected to Node localhost
08:31:39.262 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (CONNECTING) to Node
08:31:39.262 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Channel Active.
08:31:39.288 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Channel Inactive.
08:31:39.288 INFO  com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Got notified from Channel as inactive, attempting reconnect.
08:31:39.288 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Received signal to proactively refresh (a maybe outdated) configuration.
08:31:39.288 INFO  com.couchbase.client.core.node.Node - Disconnected from Node localhost
08:31:39.289 DEBUG com.couchbase.client.core.node.Node - [localhost]: Disconnected (CONNECTED) from Node
08:31:39.291 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Cancelling 1 outstanding requests.
08:31:39.294 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Requesting supported features: [TCPNODELAY, XATTR, SELECT_BUCKET, TRACING, SNAPPY, XERROR]
08:31:39.294 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Could not fetch config from bucket "test" against "localhost".
com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight.
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:686)
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handlerRemoved(AbstractGenericHandler.java:667)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:626)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyDown(DefaultChannelPipeline.java:878)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyUp(DefaultChannelPipeline.java:844)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroy(DefaultChannelPipeline.java:836)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.access$700(DefaultChannelPipeline.java:44)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelUnregistered(DefaultChannelPipeline.java:1286)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:176)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:162)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:821)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:776)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
08:31:39.294 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Error while refreshing bucket config, ignoring.
com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight.
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:686)
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handlerRemoved(AbstractGenericHandler.java:667)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:626)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyDown(DefaultChannelPipeline.java:878)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyUp(DefaultChannelPipeline.java:844)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroy(DefaultChannelPipeline.java:836)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.access$700(DefaultChannelPipeline.java:44)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelUnregistered(DefaultChannelPipeline.java:1286)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:176)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:162)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:821)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:776)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
08:31:39.307 DEBUG com.couchbase.client.core.endpoint.kv.KeyValueFeatureHandler - Negotiated supported features: [TCPNODELAY]
08:31:39.342 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Connected Endpoint.
08:31:39.343 INFO  com.couchbase.client.core.node.Node - Connected to Node localhost
08:31:39.343 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (CONNECTING) to Node
08:31:39.343 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42646][KeyValueEndpoint]: Channel Active.
08:31:39.372 DEBUG com.couchbase.client.core.config.ConfigurationProvider - New Bucket test config proposed.
08:31:39.373 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Completed refreshing config for bucket "test"
08:31:40.882 DEBUG com.couchbase.client.core.config.ConfigurationProvider - New Bucket test config proposed.
08:31:40.888 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Applying new configuration DefaultCouchbaseBucketConfig{name='test', locator=VBUCKET, uri='/pools/default/buckets/test?bucket_uuid=496a364a4f366fec00ee476996d61dc8', streamingUri='/pools/default/bucketsStreaming/test?bucket_uuid=496a364a4f366fec00ee476996d61dc8', nodeInfo=[DefaultNodeInfo{, hostname=localhost, directServices={VIEW=42652, CONFIG=8091, BINARY=42646}, sslServices={VIEW=42644, CONFIG=18091, BINARY=42647}, alternateAddresses={}, configPort=0, useAlternateNetwork=null}], partitionInfo=PartitionInfo{numberOfReplicas=0, partitionHosts=[localhost:42646], partitions=[[m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []]], tainted=false}, tainted=false, rev=16}
08:31:40.890 DEBUG com.couchbase.client.core.RequestHandler - Got notified of a new configuration arriving.
08:31:40.890 DEBUG com.couchbase.client.core.RequestHandler - Starting reconfiguration.
08:31:40.891 DEBUG com.couchbase.client.core.RequestHandler - Starting reconfiguration for bucket test
08:31:40.891 DEBUG com.couchbase.client.core.RequestHandler - Node localhost already registered, skipping.
08:31:40.891 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service VIEW, to Node localhost
08:31:40.891 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service VIEW
08:31:40.891 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service VIEW already added, skipping.
08:31:40.891 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service CONFIG, to Node localhost
08:31:40.891 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service CONFIG
08:31:40.891 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service CONFIG already added, skipping.
08:31:40.891 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service BINARY, to Node localhost
08:31:40.892 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service BINARY
08:31:40.892 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service BINARY already added, skipping.
08:31:40.892 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Completed refreshing config for bucket "test"
08:31:43.386 DEBUG com.couchbase.client.core.config.ConfigurationProvider - New Bucket test config proposed.
08:31:43.387 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Completed refreshing config for bucket "test"
08:31:45.888 DEBUG com.couchbase.client.core.config.ConfigurationProvider - New Bucket test config proposed.
08:31:45.889 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Completed refreshing config for bucket "test"
08:31:48.384 DEBUG com.couchbase.client.core.config.ConfigurationProvider - New Bucket test config proposed.
08:31:48.389 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Applying new configuration DefaultCouchbaseBucketConfig{name='test', locator=VBUCKET, uri='/pools/default/buckets/test?bucket_uuid=496a364a4f366fec00ee476996d61dc8', streamingUri='/pools/default/bucketsStreaming/test?bucket_uuid=496a364a4f366fec00ee476996d61dc8', nodeInfo=[DefaultNodeInfo{, hostname=localhost, directServices={VIEW=42652, QUERY=42651, CONFIG=8091, BINARY=42646}, sslServices={VIEW=42644, QUERY=42643, CONFIG=18091, BINARY=42647}, alternateAddresses={}, configPort=0, useAlternateNetwork=null}], partitionInfo=PartitionInfo{numberOfReplicas=0, partitionHosts=[localhost:42646], partitions=[[m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []], [m: 0, r: []]], tainted=false}, tainted=false, rev=18}
08:31:48.391 DEBUG com.couchbase.client.core.RequestHandler - Got notified of a new configuration arriving.
08:31:48.391 DEBUG com.couchbase.client.core.RequestHandler - Starting reconfiguration.
08:31:48.391 DEBUG com.couchbase.client.core.RequestHandler - Starting reconfiguration for bucket test
08:31:48.391 DEBUG com.couchbase.client.core.RequestHandler - Node localhost already registered, skipping.
08:31:48.391 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service VIEW, to Node localhost
08:31:48.391 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service VIEW
08:31:48.391 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service VIEW already added, skipping.
08:31:48.391 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service QUERY, to Node localhost
08:31:48.392 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service QUERY
08:31:48.392 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service QUERY to registry and connecting it.
08:31:48.392 DEBUG com.couchbase.client.core.service.Service - [localhost][QueryService]: Got instructed to connect.
08:31:48.392 DEBUG com.couchbase.client.core.service.Service - [localhost][QueryService]: No endpoints needed to connect, skipping.
08:31:48.392 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service CONFIG, to Node localhost
08:31:48.392 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service CONFIG
08:31:48.392 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service CONFIG already added, skipping.
08:31:48.392 DEBUG com.couchbase.client.core.RequestHandler - Got instructed to add Service BINARY, to Node localhost
08:31:48.392 DEBUG com.couchbase.client.core.node.Node - [localhost]: Adding Service BINARY
08:31:48.392 DEBUG com.couchbase.client.core.node.Node - [localhost]: Service BINARY already added, skipping.
08:31:48.393 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Completed refreshing config for bucket "test"
08:31:49.356 TRACE org.testcontainers.couchbase.CouchbaseContainer - Finished waiting
08:31:49.356 TRACE org.testcontainers.couchbase.CouchbaseContainer - Querying bucket
08:31:49.357 DEBUG com.couchbase.client.core.service.Service - [localhost][QueryService]: Need to open a new Endpoint (current size 0)
08:31:49.357 DEBUG com.couchbase.client.core.endpoint.Endpoint - Using a connectCallbackGracePeriod of 2000 on Endpoint localhost:42651
08:31:49.361 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42651][QueryEndpoint]: Connected Endpoint.
08:31:49.362 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (DEGRADED) to Node
08:31:49.364 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42651][QueryEndpoint]: Channel Active.
08:31:49.368 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42651][QueryEndpoint]: Channel Inactive.
08:31:49.368 INFO  com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42651][QueryEndpoint]: Got notified from Channel as inactive, attempting reconnect.
08:31:49.369 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Received signal to proactively refresh (a maybe outdated) configuration.
08:31:49.371 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42651][QueryEndpoint]: Cancelling 1 outstanding requests.
08:31:49.373 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42651][QueryEndpoint]: Connected Endpoint.
08:31:49.373 ERROR org.testcontainers.couchbase.CouchbaseContainer - Bucket creation failed
com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight.
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:686)
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handlerRemoved(AbstractGenericHandler.java:667)
    at com.couchbase.client.core.endpoint.query.QueryHandlerV2.handlerRemoved(QueryHandlerV2.java:208)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:626)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyDown(DefaultChannelPipeline.java:878)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyUp(DefaultChannelPipeline.java:844)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroy(DefaultChannelPipeline.java:836)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.access$700(DefaultChannelPipeline.java:44)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelUnregistered(DefaultChannelPipeline.java:1286)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:176)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:162)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:821)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:776)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
08:31:49.373 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (DEGRADED) to Node
08:31:49.374 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42651][QueryEndpoint]: Channel Active.
08:31:49.375 ERROR 🐳 [couchbase/server:4.6.5] - Could not start container
org.testcontainers.containers.ContainerLaunchException: Bucket creation failed
    at org.testcontainers.couchbase.CouchbaseContainer.containerIsStarted(CouchbaseContainer.java:416)
    at org.testcontainers.containers.GenericContainer.containerIsStarted(GenericContainer.java:598)
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:447)
    at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:305)
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:303)
    at org.testcontainers.couchbase.CouchbaseContainer.doStart(CouchbaseContainer.java:167)
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:290)
    at org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:962)
    at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
    at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
    at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
    at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
    at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
    at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
    at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
    at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight.
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:686)
    at com.couchbase.client.core.endpoint.AbstractGenericHandler.handlerRemoved(AbstractGenericHandler.java:667)
    at com.couchbase.client.core.endpoint.query.QueryHandlerV2.handlerRemoved(QueryHandlerV2.java:208)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:626)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyDown(DefaultChannelPipeline.java:878)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroyUp(DefaultChannelPipeline.java:844)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.destroy(DefaultChannelPipeline.java:836)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.access$700(DefaultChannelPipeline.java:44)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelUnregistered(DefaultChannelPipeline.java:1286)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:176)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:162)
    at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:821)
    at com.couchbase.client.deps.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:776)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
    at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464)
    at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    ... 1 common frames omitted
08:31:49.384 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42651][QueryEndpoint]: Channel Inactive.
08:31:49.384 INFO  com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42651][QueryEndpoint]: Got notified from Channel as inactive, attempting reconnect.
08:31:49.384 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Received signal to proactively refresh (a maybe outdated) configuration.
08:31:49.384 DEBUG com.couchbase.client.core.config.ConfigurationProvider - New Bucket test config proposed.
08:31:49.386 DEBUG com.couchbase.client.core.config.refresher.CarrierRefresher - Completed refreshing config for bucket "test"
08:31:49.387 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42651][QueryEndpoint]: Connected Endpoint.
08:31:49.387 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (DEGRADED) to Node
08:31:49.387 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42651][QueryEndpoint]: Channel Active.
08:31:49.399 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42651][QueryEndpoint]: Channel Inactive.
08:31:49.399 INFO  com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42651][QueryEndpoint]: Got notified from Channel as inactive, attempting reconnect.
08:31:49.400 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Received signal to proactively refresh (a maybe outdated) configuration.
08:31:49.402 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42651][QueryEndpoint]: Connected Endpoint.
08:31:49.402 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (DEGRADED) to Node
08:31:49.402 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42651][QueryEndpoint]: Channel Active.
08:31:49.421 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42651][QueryEndpoint]: Channel Inactive.
08:31:49.421 INFO  com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42651][QueryEndpoint]: Got notified from Channel as inactive, attempting reconnect.
08:31:49.421 DEBUG com.couchbase.client.core.config.ConfigurationProvider - Received signal to proactively refresh (a maybe outdated) configuration.
08:31:49.423 DEBUG com.couchbase.client.core.endpoint.Endpoint - [localhost/127.0.0.1:42651][QueryEndpoint]: Connected Endpoint.
08:31:49.423 DEBUG com.couchbase.client.core.node.Node - [localhost]: Connected (DEGRADED) to Node
08:31:49.424 DEBUG com.couchbase.client.core.endpoint.AbstractGenericHandler - [localhost/127.0.0.1:42651][QueryEndpoint]: Channel Active.

At this point I'm afraid I'm at a bit of a loss - we're obviously missing something vital, and we're using up a lot of time and energy to make little progress!

I'd be grateful for any and all suggestions, no matter how radical!

rnorth on 18 Nov 2019

👍1

@rnorth I've been experimenting with a container that does not use socat and I think I got somewhere. I'm not fully done with it yet but everything so far seem to work fine. I've got it working in a standalone lib right now but as soon as I got my POC completely working I'll do a PR to this repository - if you are curious: https://gist.github.com/daschl/e32e05e6abc31e450f67c23fe30c3826#file-couchbasecontainer-java

Comparing to the current version:

no socat, also remapping the cluster manager port
removing a config file, before start, I think this might help but need to see if it actually needed
removed java sdk 2 dependency, only http (want to make it usable for sdk2 and sdk3 testing which is on the near horizon)
randomized all the ports in the static config for couchbase and exposing them in additon to the default ones, so it should be possible in the future to spawn a cluster.

I hope I can make more progress in december and will report back.

daschl on 23 Nov 2019

👍2

@daschl thanks so much! I've been having another unsuccessful attempt to fix things this afternoon, and was just about ready to throw in the towel. Your message is extremely good timing!

For what it's worth, the current implementation has numerous places that deserve improvement - polling for creation of buckets, proper handling of non-200 HTTP response codes, longer connect timeouts, etc. The list is long. However many patches I've made, though, there are still an extraordinary number of different kinds of failures.

This is what makes me very worried about our chances of incrementally improving the reliability of this module, so a more drastic rework seems sensible.

I'll _really_ look forward to your next updates - thanks for your efforts.

rnorth on 23 Nov 2019

👍1

Update: it seems that https://github.com/testcontainers/testcontainers-java/pull/2106 stabilized our tests and now @Flaky works as intended.

bsideup on 27 Nov 2019

👍1

Just to note, although our Couchbase daily builds seem fairly stable, the intermittent Couchbase failures seems to be still there after upgrading to version 1.14.0.

https://github.com/hipster-labs/jhipster-daily-builds/runs/597246279?check_suite_focus=true

SudharakaP on 19 Apr 2020

@SudharakaP thanks for updating! Do you have more logs of such failures? /cc @daschl

bsideup on 19 Apr 2020

👍1

@bsideup : No we run that build once a day and that's the only such time it failed (out of the 6 times it ran after upgrading to 1.14.0). But if and when there's more I'll post them here.

@pascalgrimaud : Could we schedule this build to run several times a day to speed up the process of finding more logs? :smile:

SudharakaP on 19 Apr 2020

@SudharakaP : sure, feel free to change the cron and merge it :)

pascalgrimaud on 19 Apr 2020

👍1

Done; I'll post back tomorrow with hopefully more logs. :smile:

SudharakaP on 20 Apr 2020

So, here are five failures; two of which involve Maven and the other three involving Gradle;

Gradle

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/81289930

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/82411492

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/82524483

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/82802953

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/82935879

Maven

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/82443392

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/82567070

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/82761556

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/82866603

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/83078352

https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/83033289

@bsideup : Do you think this is enough samples or do you guys want more? :smile:

EDIT: I've added some more samples above. :smile:

SudharakaP on 20 Apr 2020

@SudharakaP looking at the logs, I do not see any actual errors (i.e. https://github.com/hipster-labs/jhipster-daily-builds/runs/600774837?check_suite_focus=true) .. I only see

2020-04-20 05:38:23.395  INFO 7589 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /reverent_keldysh: Waiting for 60 seconds for URL: http://localhost:32783/pools

Is that suggesting it times out on the wait?

edit: oh, one other had an actual error in it:

Caused by: java.lang.IllegalStateException: Could not create primary index for bucket jhipster: Response{protocol=http/1.1, code=500, message=Internal Server Error, url=http://localhost:32781/query/service}

daschl on 29 May 2020

👍1

@daschl : Thanks for looking into it. Feel free to let us know if you need any further information or logs. :smile:

SudharakaP on 29 May 2020

@SudharakaP btw the new release that got out today has a change in it which will make it more intelligent to wait until the query engine is up - that maybe fixes the second error from above.. can you try upgrading to the latest bugfix version and report if it still happens?

daschl on 29 May 2020

@SudharakaP also I saw in one of the gradle failures it would just print the exception type, but not the associated message. is it possible in gradle to enable that so that we also see the exception message and cause, potentially?

daschl on 29 May 2020

@SudharakaP btw the new release that got out today has a change in it which will make it more intelligent to wait until the query engine is up - that maybe fixes the second error from above.. can you try upgrading to the latest bugfix version and report if it still happens?

@daschl : Thanks for the input. We have upgraded to 1.14.3 the latest version. There wasn't any Couchbase failures since last month so I am wondering if this problem is gone now. :thinking: Let us give some more time.

@SudharakaP also I saw in one of the gradle failures it would just print the exception type, but not the associated message. is it possible in gradle to enable that so that we also see the exception message and cause, potentially?

I'll look into enabling the Gradle logs to get the detailed error messages.

SudharakaP on 30 May 2020

Just to let you know for the past couple of months there was only one Couchbase failure; https://github.com/hipster-labs/jhipster-daily-builds/runs/764200230?check_suite_focus=true. So I think the latest version has improved things a lot. Thanks a lot for your work. :smile:

SudharakaP on 26 Jun 2020

@SudharakaP I've been trying to spot the error in the log you pointed out but I could only see build warnings - can you point me to the test failure with stack trace/logs? then I can take a look :) thanks!

daschl on 26 Jun 2020

👍1

@daschl : I thought this one is a timeout. So the line is,

2020-06-12 04:15:57.190  INFO 6779 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /optimistic_lehmann: Waiting for 60 seconds for URL: http://localhost:32783/pools/default/buckets/jhipster

Notice that before two months ago (before we upgraded to the latest version) we had this kinda logs; they had minor differences; such as this one;

https://github.com/hipster-labs/jhipster-daily-builds/runs/610843718?check_suite_focus=true

which has the line,

2020-04-23 04:11:17.675  INFO 7633 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /amazing_kirch: Waiting for 60 seconds for URL: http://localhost:32783/pools

before the timeout. Although I think this has greatly improved and failures are rare; both issues seems related; so just thought of letting you know. I'll report more if I see this in the future. :thinking:

SudharakaP on 26 Jun 2020

Testcontainers-java: Intermittent couchbase failures in CI

Most helpful comment

All 37 comments

Related issues