Google-cloud-java: Channel ManagedChannelImpl{logId=589, target=vision.googleapis.com:443} was not terminated properly

[mcantrell]

In a concurrent environment, the Vision API client will occasionally log the following error:

Nov 06, 2018 8:07:35 AM io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue
SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=589, target=vision.googleapis.com:443} was not terminated properly!!! ~*~*~*
    Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
java.lang.RuntimeException: ManagedChannel allocation site
    at io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.<init>(ManagedChannelOrphanWrapper.java:103)
    at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:53)
    at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:44)
    at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:410)
    at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:206)
    at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:157)
    at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:149)
    at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:149)
    at com.google.cloud.vision.v1.stub.GrpcImageAnnotatorStub.create(GrpcImageAnnotatorStub.java:84)
    at com.google.cloud.vision.v1.stub.ImageAnnotatorStubSettings.createStub(ImageAnnotatorStubSettings.java:120)
    at com.google.cloud.vision.v1.ImageAnnotatorClient.<init>(ImageAnnotatorClient.java:136)
    at com.google.cloud.vision.v1.ImageAnnotatorClient.create(ImageAnnotatorClient.java:117)
    at com.google.cloud.vision.v1.ImageAnnotatorClient.create(ImageAnnotatorClient.java:108)
    at com.acme.vision.support.VisionTerminationErrorTest.executeOcrRequest(VisionTerminationErrorTest.java:60)
    at com.acme.vision.support.VisionTerminationErrorTest.lambda$ocrImage$0(VisionTerminationErrorTest.java:28)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

No error is actually thrown, just logged. I'm attaching a sample project to replicate the condition. Since it doesn't actually throw an exception, I cannot fail the test. You'll need to inspect the log output. You may also need to tweak the number of threads to replicate on your system.

vision-termination-error.zip

[mcantrell]

The following workaround fixes the problem:

 client.shutdown();
 client.awaitTermination(30, TimeUnit.SECONDS);
 client.close();

It feels like client.close() should probably handle this or the documentation updated to include this.

[talonx]

The workaround does not work for me. This is with the latest (1.54.0) google-cloud-pubsub lib.

[ajaaym]

This is fixed in 628. Will be available after new version of gax-java is released