Akka-http: Support maxKeepAliveRequests on akka http server

[kstokoz]

It would be great if akka http server would provide configuration options to set maxKeepAliveRequests per connection, so that after connection served that amount of requests akka http would force connection close.

It is very useful when using load-balance, for web service, load-balance always use connection based strategy, and when new node added in, server need a chance to re-balance the load.
Currently keepalive connection will be kept for ever, so there is no new connection established, on going connection can not be re-balanced.

When running in kubernetes load balancer would not see how many nodes running in the pod (they can be potentially hidden behind single cluster ip), so scaling up would be useless as traffic would not be rebalanced.

Many http server already support such option:

• nginx: http://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_requests
• tomcat: https://tomcat.apache.org/tomcat-7.0-doc/config/http.html
• apache: https://httpd.apache.org/docs/2.4/mod/core.html#maxkeepaliverequests
• jetty (not supported): https://bugs.eclipse.org/bugs/show_bug.cgi?id=437636

I tried as suggested in jetty bug to send connection close header in my route and it does work.
I cannot control which exactly connection i will close with it as it is abstracted away by akka http and i`m not sure how sending connection close header will affect request pipelining.

It would be great if akka http would support such configuration option out of the box.

[jrudolph]

Yes, good idea.

(That said, imo load balancing decisions should be taken at the load balancer not at the upstream backend. The load balancer is free at any time to close an upstream connection and that's the usual solution for these kinds of problems. Having such a limit could still be worthwhile for force thumb clients to reconnect regularly.)

[zsedem]

Opened a PR for this
#2016

[iamthiago]

Hi guys, I just came across this situation using cachedHostConnection on kubernetes. Would Http().single be too bad for several requests? I'm trying to think in a workaround as the above PR is still on it's way to get merged.

[zsedem]

Http().singleRequest does use the same host connection pool

At my working place we use a patched version of akka-http to workaround this problem.

[iamthiago]

@zsedem so changing from cachedHostConnection to singleRequest would make no difference?!

[zsedem]

Just quoting this: https://doc.akka.io/docs/akka-http/current/client-side/request-level.html#request-level-client-side-api

The request-level API is the recommended and most convenient way of using Akka HTTP’s client-side functionality. It internally builds upon the Host-Level Client-Side API to provide you with a simple and easy-to-use way of retrieving HTTP responses from remote servers. Depending on your preference you can pick the flow-based or the future-based variant.

The request-level API is implemented on top of a connection pool that is shared inside the actor system. A consequence of using a pool is that long-running requests block a connection while running and starve other requests. Make sure not to use the request-level API for long-running requests like long-polling GET requests. Use the Connection-Level Client-Side API or an extra pool just for the long-running connection instead.

It would make difference, but it will still use a pool with default configuration, I suspect it is in some global state.