Grpc-go: Recv() after GracefulStop() getting rpc error "transport is closing"

[23doors]

Please answer these questions before submitting your issue.

What version of gRPC are you using?

1.7.3

What version of Go are you using (go version)?

1.9.2

What operating system (Linux, Windows, …) and version?

Tested under macos and ubuntu lts.

What did you do?

GracefulStop() for bidirectional streams that are already started sometimes results in client Recv getting error:
rpc error: code = Unavailable desc = transport is closing

This used to work in 1.5.x and 1.6.x in a different way - graceful stop was blocking until that stream was closed.

Tested 1.7.0-1.7.3 and it ends up being racy. Usually it ends up with transport is closing, but sometimes it succeeds.

[menghanl]

Could you provide a reproduction for this?

I tried with a streaming helloworld example, the behavior was as expected (GracefulStop blocks until stream is done).

[23doors]

@menghanl Sure thing. Managed to reproduce it with helloworld example.
https://github.com/23doors/grpc-graceful-stop-bug
Put steps to reproduce in readme there.

[MakMukhi]

Thanks for your repro. This is an issue in the gRPC server's transport logic; having scheduled the status for last stream in graceful stop mode the server incorrectly assumes that it can close the transport. It must wait for the status to be written out before doing that.
I'll have a fix out today.