Grpc-java: Exception java.lang.IllegalStateException: Could not find policy 'pick_first'

Created on 24 Mar 2019 · 17Comments · Source: grpc/grpc-java

Please answer these questions before submitting your issue.

What version of gRPC are you using?

1.18.0

When executing the Quickstart class of Google Firestore Service I get this exception.
https://cloud.google.com/firestore/docs/quickstart-servers

Mar 24, 2019 7:16:18 PM io.grpc.internal.ManagedChannelImpl$1 uncaughtException
SEVERE: [Channel<1>: (firestore.googleapis.com:443)] Uncaught exception in the SynchronizationContext. Panic!
java.lang.IllegalStateException: Could not find policy 'pick_first'. Make sure its implementation is either registered to LoadBalancerRegistry or included in META-INF/services/io.grpc.LoadBalancerProvider from your jar files.
        at io.grpc.internal.AutoConfiguredLoadBalancerFactory$AutoConfiguredLoadBalancer.<init>(AutoConfiguredLoadBalancerFactory.java:93)
        at io.grpc.internal.AutoConfiguredLoadBalancerFactory.newLoadBalancer(AutoConfiguredLoadBalancerFactory.java:64)
        at io.grpc.internal.ManagedChannelImpl.exitIdleMode(ManagedChannelImpl.java:357)
        at io.grpc.internal.ManagedChannelImpl$ChannelTransportProvider$1ExitIdleModeForTransport.run(ManagedChannelImpl.java:455)
        at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:101)
        at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:130)
        at io.grpc.internal.ManagedChannelImpl$ChannelTransportProvider.get(ManagedChannelImpl.java:459)
        at io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:241)
        at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1.start(CensusTracingModule.java:392)
        at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1.start(CensusStatsModule.java:694)
        at io.grpc.ForwardingClientCall.start(ForwardingClientCall.java:32)
        at com.google.api.gax.grpc.GrpcHeaderInterceptor$1.start(GrpcHeaderInterceptor.java:95)
        at io.grpc.stub.ClientCalls.startCall(ClientCalls.java:308)
        at io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:280)
        at io.grpc.stub.ClientCalls.futureUnaryCall(ClientCalls.java:189)
        at com.google.api.gax.grpc.GrpcDirectCallable.futureCall(GrpcDirectCallable.java:58)
        at com.google.api.gax.grpc.GrpcExceptionCallable.futureCall(GrpcExceptionCallable.java:64)
        at com.google.api.gax.tracing.TracedUnaryCallable.futureCall(TracedUnaryCallable.java:75)
        at com.google.api.gax.rpc.UnaryCallable$1.futureCall(UnaryCallable.java:126)
        at com.google.api.gax.rpc.UnaryCallable.futureCall(UnaryCallable.java:87)
        at com.google.cloud.firestore.FirestoreImpl.sendRequest(FirestoreImpl.java:413)
        at com.google.cloud.firestore.UpdateBuilder.commit(UpdateBuilder.java:621)
        at com.google.cloud.firestore.WriteBatch.commit(WriteBatch.java:41)
        at com.google.cloud.firestore.DocumentReference.set(DocumentReference.java:172)
        at Main.main(Main.java:26)

java.util.concurrent.ExecutionException: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Panic! This is a bug!
        at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:531)
        at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:492)
        at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:83)
        at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:62)
        at Main.main(Main.java:30)
Caused by: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Panic! This is a bug!
        at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:67)
        at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
        at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
        at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
        at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68)
        at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1052)
        at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:398)
        at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1030)
        at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:675)
        at com.google.common.util.concurrent.ForwardingListenableFuture.addListener(ForwardingListenableFuture.java:45)
        at com.google.api.core.ApiFutureToListenableFuture.addListener(ApiFutureToListenableFuture.java:52)
        at com.google.common.util.concurrent.Futures.addCallback(Futures.java:1033)
        at com.google.api.core.ApiFutures.addCallback(ApiFutures.java:63)
        at com.google.api.gax.grpc.GrpcExceptionCallable.futureCall(GrpcExceptionCallable.java:67)
        at com.google.api.gax.tracing.TracedUnaryCallable.futureCall(TracedUnaryCallable.java:75)
        at com.google.api.gax.rpc.UnaryCallable$1.futureCall(UnaryCallable.java:126)
        at com.google.api.gax.rpc.UnaryCallable.futureCall(UnaryCallable.java:87)
        at com.google.cloud.firestore.FirestoreImpl.sendRequest(FirestoreImpl.java:413)
        at com.google.cloud.firestore.UpdateBuilder.commit(UpdateBuilder.java:621)
        at com.google.cloud.firestore.WriteBatch.commit(WriteBatch.java:41)
        at com.google.cloud.firestore.DocumentReference.set(DocumentReference.java:172)
        at Main.main(Main.java:26)
Caused by: io.grpc.StatusRuntimeException: INTERNAL: Panic! This is a bug!
        at io.grpc.Status.asRuntimeException(Status.java:530)
        at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:482)
        at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
        at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
        at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
        at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:699)
        at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
        at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
        at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
        at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:397)
        at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:459)
        at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:546)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:467)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:584)
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Could not find policy 'pick_first'. Make sure its implementation is either registered to LoadBalancerRegistry or included in META-INF/services/io.grpc.LoadBalancerProvider from your jar files.
        at io.grpc.internal.AutoConfiguredLoadBalancerFactory$AutoConfiguredLoadBalancer.<init>(AutoConfiguredLoadBalancerFactory.java:93)
        at io.grpc.internal.AutoConfiguredLoadBalancerFactory.newLoadBalancer(AutoConfiguredLoadBalancerFactory.java:64)
        at io.grpc.internal.ManagedChannelImpl.exitIdleMode(ManagedChannelImpl.java:357)
        at io.grpc.internal.ManagedChannelImpl$ChannelTransportProvider$1ExitIdleModeForTransport.run(ManagedChannelImpl.java:455)
        at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:101)
        at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:130)
        at io.grpc.internal.ManagedChannelImpl$ChannelTransportProvider.get(ManagedChannelImpl.java:459)
        at io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:241)
        at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1.start(CensusTracingModule.java:392)
        at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1.start(CensusStatsModule.java:694)
        at io.grpc.ForwardingClientCall.start(ForwardingClientCall.java:32)
        at com.google.api.gax.grpc.GrpcHeaderInterceptor$1.start(GrpcHeaderInterceptor.java:95)
        at io.grpc.stub.ClientCalls.startCall(ClientCalls.java:308)
        at io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:280)
        at io.grpc.stub.ClientCalls.futureUnaryCall(ClientCalls.java:189)
        at com.google.api.gax.grpc.GrpcDirectCallable.futureCall(GrpcDirectCallable.java:58)
        at com.google.api.gax.grpc.GrpcExceptionCallable.futureCall(GrpcExceptionCallable.java:64)
        at com.google.api.gax.tracing.TracedUnaryCallable.futureCall(TracedUnaryCallable.java:75)
        at com.google.api.gax.rpc.UnaryCallable$1.futureCall(UnaryCallable.java:126)
        at com.google.api.gax.rpc.UnaryCallable.futureCall(UnaryCallable.java:87)
        at com.google.cloud.firestore.FirestoreImpl.sendRequest(FirestoreImpl.java:413)
        at com.google.cloud.firestore.UpdateBuilder.commit(UpdateBuilder.java:621)
        at com.google.cloud.firestore.WriteBatch.commit(WriteBatch.java:41)
        at com.google.cloud.firestore.DocumentReference.set(DocumentReference.java:172)
        at Main.main(Main.java:26)

Source

pereira-a

Most helpful comment

Turns out it was due to the Gradle Shadow plugin not merging service files by default. I just had to add this to my build.gradle and it works for me:

shadowJar {
    mergeServiceFiles()
}

mpdn on 1 Apr 2019

👍4

All 17 comments

@SkunFly can you provide how you configure loadbalancer when you create channel? did you try what error message suggest?

creamsoup on 25 Mar 2019

@SkunFly can you provide how you configure loadbalancer when you create channel? did you try what error message suggest?

I don't configure anything. I simply follow the guide and it should work. Maybe I'm missing something but I think not. If you try to replicate this you will face the same error. https://cloud.google.com/firestore/docs/quickstart-servers

I don't know if this is the correct place to open an issue, because I'm doing a guide about Firestore. But the stack trace, the exception message and the context of the error motivated me to open an issue here.

pereira-a on 25 Mar 2019

Can you run the complete quickstart example? if the issue is still exists, file against firestore? (or the example repository)
Even if this is a gRPC issue, they also need to aware of this issue. They can fix the issue or find a workaround until the proper fix is released if this is not just a tutorial issue.

creamsoup on 25 Mar 2019

Can you run the complete quickstart example? if the issue is still exists, file against firestore? (or the example repository)

When I execute the provided class "Quickstart" it gives me the above error.

Even if this is a gRPC issue, they also need to aware of this issue. They can fix the issue or find a workaround until the proper fix is released if this is not just a tutorial issue.

I already opened an issue in their platform, but at the moment they gave no response.

pereira-a on 25 Mar 2019

@SkunFly Where is your client app running? GCP or Android?

dapengzhang0 on 25 Mar 2019

@SkunFly Where is your client app running? GCP or Android?

GCP

pereira-a on 25 Mar 2019

Hitting the same issue on a local machine. Same error and I do not have a loadbalancer configured either.

mpdn on 29 Mar 2019

@noctune, are you running the same code(firestore quickstart)?

creamsoup on 30 Mar 2019

@creamsoup Nope. This is plain GRPC. I will try to see if I can find a minimal example that causes it, but it seems difficult to replicate.

mpdn on 30 Mar 2019

Turns out it was due to the Gradle Shadow plugin not merging service files by default. I just had to add this to my build.gradle and it works for me: