Grpc-java: io.grpc.StatusRuntimeException: INTERNAL: Protocol error Rst Stream

[zhaoweizhen]

android client

plugin version: com.google.protobuf:protobuf-gradle-plugin:0.8.8

protobuf {
    protoc {
        artifact = 'com.google.protobuf:protoc:3.0.0'
    }
    plugins {
        javalite {
            artifact = "com.google.protobuf:protoc-gen-javalite:3.0.0"
        }
        grpc {
            artifact = 'io.grpc:protoc-gen-grpc-java:1.22.1' 
        }
    }
    generateProtoTasks {
        all().each { task ->
            task.plugins {
                javalite {}
                grpc {
                    // Options added to --grpc_out
                    option 'lite'
                }
            }
        }
    }
}

grpc version:1.22.1

implementation 'io.grpc:grpc-okhttp:1.22.1'
implementation 'io.grpc:grpc-protobuf-lite:1.22.1'
implementation 'io.grpc:grpc-stub:1.22.1'

Current phenomenon: after establishing a connection with the server, there is no interaction, and the client will receive this error after three minutes

By tracing, I located the error here：

class: OkHttpClientTransport

  --------------- while (frameReader.nextFrame(this)) --------------

   String threadName = Thread.currentThread().getName();
      if (!GrpcUtil.IS_RESTRICTED_APPENGINE) {
        Thread.currentThread().setName("OkHttpClientTransport");
      }
      try {
        // Read until the underlying socket closes.
        while (frameReader.nextFrame(this)) { //------------here---------------
          if (keepAliveManager != null) {
            keepAliveManager.onDataReceived();
          }
        }
        // frameReader.nextFrame() returns false when the underlying read encounters an IOException,
        // it may be triggered by the socket closing, in such case, the startGoAway() will do
        // nothing, otherwise, we finish all streams since it's a real IO issue.
        startGoAway(0, ErrorCode.INTERNAL_ERROR,
            Status.UNAVAILABLE.withDescription("End of stream or IOException"));
      } catch (Throwable t) {
        // TODO(madongfly): Send the exception message to the server.
        startGoAway(
            0,
            ErrorCode.PROTOCOL_ERROR,
            Status.INTERNAL.withDescription("error in frame handler").withCause(t));
      } finally {
        try {
          frameReader.close();
        } catch (IOException ex) {
          log.log(Level.INFO, "Exception closing frame reader", ex);
        }
        listener.transportTerminated();
        if (!GrpcUtil.IS_RESTRICTED_APPENGINE) {
          // Restore the original thread name.
          Thread.currentThread().setName(threadName);
        }
      }
    }

then:
Http2

case TYPE_RST_STREAM:
readRstStream(handler, length, flags, streamId);
break;

Please tell me if you need more information
Hoping you can help me
Thanks

[voidzcy]

Are you using grpc-java server? Can you provide us server side logs? Especially around where server sends the RST_STREAM.

[zhaoweizhen]

Are you using grpc-java server? Can you provide us server side logs? Especially around where server sends the RST_STREAM.

Server client is using grpc-go .
My colleague said that the server did not actively send the RST_STREAM
It seems that the RST_STREAM data was sent by grpc

The error occurred in the following code：

errorCodeInt =1 

  private void readRstStream(
        Handler handler, int length, @SuppressWarnings("UnusedVariable") byte flags, int streamId)
        throws IOException {
      if (length != 4) throw ioException("TYPE_RST_STREAM length: %d != 4", length);
      if (streamId == 0) throw ioException("TYPE_RST_STREAM streamId == 0");
      int errorCodeInt = source.readInt();//-----------here-----------
      io.grpc.okhttp.internal.framed.ErrorCode errorCode = io.grpc.okhttp.internal.framed.ErrorCode.fromHttp2(errorCodeInt);
      if (errorCode == null) {
        throw ioException("TYPE_RST_STREAM unexpected error code: %d", errorCodeInt);
      }
      handler.rstStream(streamId, errorCode);
    }

public enum ErrorCode {
  /** Not an error! For SPDY stream resets, prefer null over NO_ERROR. */
  NO_ERROR(0, -1, 0),

  PROTOCOL_ERROR(1, 1, 1),
..........

[voidzcy]

Client side receives an unexpected RST_STREAM frame that causes the error you saw. We need to know why server side (grpc-go server) sends RST_STREAM. So please provide us information (grpc-go logs) on server side. Likely there is some error happening on server side.

[zhaoweizhen]

.

Client side receives an unexpected RST_STREAM frame that causes the error you saw. We need to know why server side (grpc-go server) sends RST_STREAM. So please provide us information (grpc-go logs) on server side. Likely there is some error happening on server side.

Thanks for your reply. The cause of receiving an unexpected RST_STREAM frame is really caused by the server client，and my colleagues have found out the reason.

The connection between the client and server is through the domain name, not the IP directly. But due to some configurations, the connection is interrupted.

[voidzcy]

Is your issue resolved?

[zhaoweizhen]

Yeah! Thanks!

[build3r]

@zhaoweizhen Can I ask how the issue was resolved?

[penguin205]

I meet as same problem as yours, Can I ask how to resolved?@zhaoweizhen
Android client: grpc + ssl

the same traceing breakPoints

case TYPE_RST_STREAM:
readRstStream(handler, length, flags, streamId);

break;

07-22 09:54:45.649 W/System.err( 8546): io.grpc.StatusRuntimeException: INTERNAL: Protocol error
07-22 09:54:45.650 W/System.err( 8546): Rst Stream
07-22 09:54:45.650 W/System.err( 8546): at io.grpc.Status.asRuntimeException(Status.java:532)
07-22 09:54:45.650 W/System.err( 8546): at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:434)
07-22 09:54:45.650 W/System.err( 8546): at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
07-22 09:54:45.650 W/System.err( 8546): at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
07-22 09:54:45.650 W/System.err( 8546): at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
07-22 09:54:45.650 W/System.err( 8546): at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:700)
07-22 09:54:45.651 W/System.err( 8546): at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
07-22 09:54:45.651 W/System.err( 8546): at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)