Grpc-go: GracefulStop doesn't wait for RPC to finish

[btoews]

I don't think this is a duplicate of https://github.com/grpc/grpc-go/issues/848, so I'm opening a new issue.

I'm trying to figure out how to gracefully shutdown my GRPC server so RPCs aren't cancelled during server deployments. From reading the docs, it sounds like I want func (*Server) GracefulStop, but in practice it seems to cancel in-flight RPCs.

I'm testing this by triggering a GracefulStop() while the server is processing a long-running RPC (sleeping). This is being output to stderr

2017/03/08 14:10:24 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp :5543: getsockopt: connection refused"; Reconnecting to {:5543 <nil>}

and the client is receiving this error: rpc error: code = 13 desc = transport is closing

Am I doing something wrong, or is this a bug?

I threw together an example server/client to demonstrate this behavior: https://github.com/mastahyeti/grace

Thanks.

[btoews]

Ah. This is because the main goroutine is exiting when Serve returns. I guess the main thread should call GracefulStop() also then to wait on the RPC to finish?

I'll close this, since it doesn't seem to be a bug.

[btoews]

For posterity, here's how I fixed my example app.

[pawelkowalak]

@mastahyeti how does it fix your issue? I've just cloned your repo with the fix, and the client is still getting transport is closing.

[btoews]

It looks like the behavior was changed in https://github.com/grpc/grpc-go/pull/1485. Maybe there's a regression in how clients deal with the server closing.

[dfawley]

I think it's possible that's #1738.

[hypnoglow]

This is finally fixed in #1734