Grpc-web: How to disable Stream timeout

[dasois]

I have open streams from server to client, which shall be idle for a long time.
Currently they get terminated exactly after 10 min:

• HTTP 200
• grpc-message: upstream connect error or disconnect/reset before headers. reset reason: connection termination
• grpc-status: 14

I found out that there is a GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS, but I did not manage to set this on the client. How can I do this?

[dasois]

I did try the solution suggested in https://github.com/grpc/grpc-web/issues/361#issuecomment-437384471 without success.

[z0mb1ek]

may be its envoy timeout?

[dasois]

in envoy config i have:
max_grpc_timeout: 0s and
stream_idle_timeout: 0s.

What else would i need?

[kwomackcodes]

Having the same issue. Have you found a solution? @dasois

[dasois]

No, apparently there is no way to do this with grpc-web.

Am 25. Juli 2019 16:45:15 MESZ schrieb KevinWomack0318 [e-mail]:

Having the same issue. Have you found a solution? @dasois

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/grpc/grpc-web/issues/557#issuecomment-515073486

--
Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

[kwomackcodes]

No, apparently there is no way to do this with grpc-web. Am 25. Juli 2019 16:45:15 MESZ schrieb KevinWomack0318 [e-mail]:
…
Having the same issue. Have you found a solution? @dasois -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #557 (comment)
-- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

Yeah I figured...something tells me (like the intuition above) that this might be an issue with Envoy as a proxy. Some else seemed to have luck here in this (still open) issue when they switched proxies. I hope they do something about this soon.

[doublerr]

FWIW, I was having a similar issue but my timeouts were exactly 1 minute. I thought it was the grpcwebproxy so I switched to envoy which had the same issue but gave me better logging. It was then clear that my issue was a deadline being hit which was betting set be the client. Envoy and the server pay attention to these deadlines. My challenge was that the error manifested differently if data had been streamed (ERR_INCOMPLETE_CHUNKED_ENCODING 200 (OK)) then if the stream was connected but no data sent (status messages). Simply upping the deadline set by the client and handling deadline timeout on the client solved my issue.

[ansraliant]

Hi @doublerr, I came across your comment as I am experiencing the same issue with envoy proxy. You said your solution was to upping the deadline set by the client and handling deadline timeout on the client could you explain a bit more how you can achieve that?
Thank you.

[doublerr]

@ansraliant - Basic example (likely a bit dated now):

var deadline = new Date();
deadline.setSeconds(deadline.getSeconds() + 900); //deadline set for 15min which envoy will respect
var stream = this.client.streamArticles(request, {
  deadline: deadline.getTime()
});

For handling the actual timeout, look into the setTimeout function to create a timer and then handle it in stream.on("err", ...

stream.on("err", function(err) {
        console.log("err: ", err);
        clearTimeout(timer);
});

[kkweon]

I'm also getting 1 minute timeout but the above did not work. In fact, setting deadline to 0 or not setting is the same as setting the maximum deadline according to the spec.

In my case, it was nginx ingress controller. nginx ingress has the default proxy_read_timeout: 60s
fixing this has resolved the issue for me.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: chat-server-ingress
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/ingress.class: 'nginx'
    nginx.ingress.kubernetes.io/proxy-read-timeout: "86400"
    nginx.ingress.kubernetes.io/proxy-write-timeout: "86400"
    nginx.ingress.kubernetes.io/rewrite-target: /$1

spec:
  tls:
    - hosts:
        - k8s.kkweon.dev
      secretName: kkweon-tls
  rules:
    - host: k8s.kkweon.dev
      http:
        paths:
          - path: /grpc/?(.*)
            backend:
              serviceName: server
              servicePort: 8080

[emil14]

@doublerr are there any way to not use setTimeout on every call?

As far as I see we'll get our callback triggered with error when deadline will be reached. So the problem is to distinguish timeout error from any other error?

If I don't care - is there timeout error or something else, I can not use setTimeout, am I right? :)