Grpc-go: Question: testing keepalive behaviour without expected error

[spzala]

Please answer these questions before submitting your issue.

What version of gRPC are you using?

Master branch - 1.16.0-dev

What version of Go are you using (go version)?

go version go1.10.3 darwin/amd64

What operating system (Linux, Windows, …) and version?

mac os 10.13, ubuntu 16.04

What did you do?

I am using the https://github.com/grpc/grpc-go/tree/master/examples/helloworld and I have slightly modified greeter_server/main.go and greeter_client/main.go to test keepalive ServerParameters and EnforcementPolicy. The changes I made can be seen here https://github.com/grpc/grpc-go/compare/master...spzala:master. I also played with different time value and various parameters like MaxConnectionIdle as commented out in the changes.

What did you expect to see?

When I run greeter_server/main.go and greeter_client/main.go, I was expecting to see some error message like too many pings/goaway or disconnect because client Time parameter value is less than EnforcementPolicy minTime. I also exported env variables like export GRPC_GO_LOG_VERBOSITY_LEVEL=2, GRPC_GO_LOG_SEVERITY_LEVEL=info, GRPC_TRACE=all,
GRPC_VERBOSITY=DEBUG But I don't see any error.

What did you see instead?

The greeter_server/main.go and greeter_client/main.go runs fine with no error (e.g. no error like too many pings from the client, closing the connection) or disconnect.

Can you please help me understand what I am missing here? how could I see an expected behavior? Thanks!

[spzala]

/cc @gyliu513

[gyliu513]

/cc @meling @menghanl can you help check? As I saw you updated this example recently. Thanks!

[menghanl]

Did you try setting PermitWithoutStream to true?
With this set to false, the client won't send ping when there's no stream (stream only exists when the RPC is in progress, so when the unary RPC finishes, there's no stream), and the server won't check against the EnforcementPolicy when there's no stream.

[spzala]

@menghanl thanks. I am OOO but I quickly tried it without any luck of seeing any expected behavior. I am not sure if I am doing anything wrong with the way I am testing but if you get a chance, can you please try the modified code I have mentioned about? I guess they are simple and pretty quick to run. Thanks again!

[menghanl]

With the follow configuration, client will get EnhanceYourCalm goaway.
Client

grpc.WithKeepaliveParams(keepalive.ClientParameters{
    Time:                time.Millisecond,
    Timeout:             time.Millisecond,
    PermitWithoutStream: true,
})

Server

grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
    MinTime:             2 * time.Second,
    PermitWithoutStream: true,
})

Besides PermitWithoutStream, Timeout on client side is also important.

The client will send a ping, wait for Timeout, and check if there's any activity (ping ack is an activity). So the client won't send a second ping before this timeout. The default timeout value is 20 seconds.

Another easy way to reproduce this is to set PermitWithoutStream: false on server side, and don't do any RPC on the client side. So whenever the server receives a ping, the ping will be considered invalid.

[spzala]

@menghanl thank you so much, it works the way you suggested. Also, I could set up PermitWithoutStream: false on server side and that works too. However, it requires to set PermitWithoutStream: true on the client side. Even if I make call from client as below snippet, if PermitWithoutStream is set to false then it won't work. So a related question, is there a way I can keep PermitWithoutStream to false and still see the goaway messages? Thanks again!!

        c := pb.NewGreeterClient(conn)
        // Contact the server and print out its response.
        name := defaultName
        ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
        defer cancel()
        for {
                r, err := c.SayHello(ctx, &pb.HelloRequest{Name: name})
                if err != nil {
                        log.Fatalf("could not greet: %v", err)
                }
                <-time.After(4*time.Second)
                log.Printf("Greeting: %s", r.Message)
        }

I also tried examples/route_guide/client with adding keepalive param and some wait time, but that also requires me to set PermitWithoutStream:true to see EnhanceYourCalm goaway.

[menghanl]

is there a way I can keep PermitWithoutStream to false and still see the goaway messages?

The purpose of PermitWithoutStream is to control whether keepalive pings are sent when there's no active streams. So there is no way to do this.

I also tried examples/route_guide/client

Which RPC did you try? GetFeatureis a unary RPC, and is similar with HelloWorld. If you tried streaming RPCs and sent requests, keepalive pings won't happen (because there's no need to ping when there's activities on the connection).

You can try to do a RouteChat, keep the stream and don't send requests.

---

For more on keepalive, please take a look at https://github.com/grpc/grpc-go/pull/2342 and see if that helps.

[spzala]

@menghanl thanks a lot!! The client do call RouteChat https://github.com/grpc/grpc-go/blob/master/examples/route_guide/client/client.go#L191 but I tried it as such without modifying. Lemme work on it and test it. I will be closing this issue soon then.

[spzala]

@menghanl With RouteChat it works, if I don't close the stream and do nothing it does produce http2.ErrCodeEnhanceYourCalm. I am good now and closing this issue. Thanks much again for your help understand this and creating https://github.com/grpc/grpc-go/pull/2342