Akka-http: Add request timeout support to client-side pool APIs

[akka-ci]

Issue by mpilquist
_Tuesday Aug 02, 2016 at 14:54 GMT_
_Originally opened as https://github.com/akka/akka/issues/21098_

---

Consider the following program, which incorrectly fails to process response entities:

import $ivy.`com.typesafe.akka::akka-http-experimental:2.4.8`

import scala.concurrent.Future

import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.http.scaladsl._
import akka.http.scaladsl.client.RequestBuilding._
import akka.http.scaladsl.server._

implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
implicit val ec = system.dispatcher

// Let's create an HTTP server that responds to all requests with a very large response entity
val serverFlow = Route.handlerFlow(Directives.complete("." * 5000000))
Http().bindAndHandle(serverFlow, "0.0.0.0", 5555).flatMap { serverBinding =>
  // Now that server has started, let's make a bunch of requests in succession using the superPool API
  val pool = Http().superPool[Unit]()
  Future.traverse(0 to 20) { i =>
    val response = Source.single(Get("http://localhost:5555") -> ()).
      via(pool).
      runWith(Sink.head)
    response.map { case (tryResponse, _) =>
      // We incorrectly fail to drain the response entity here
      println(s"Got response $i")
      tryResponse
    }
  }
}

Running this program results in:

Got response 2
Got response 0
Got response 4
Got response 1

4 requests run and then, due to the client failing to process response entities, the program hangs forever. This is a bug in this code -- the response entities must be processed. However, that's not what this ticket is about.

The super pool API could be made much safer by supporting some form of (optional?) timeouts on requests. For example, if the fifth request hasn't been assigned a host connector within X seconds, fail the response with a timeout exception indicating the pool was too busy.

In presence of such a feature, the above program would complete - 4 requests would execute like they did in the above output and then 16 requests would fail with timeout exceptions, indicating the connector pool was too busy. This is a much safer default behavior.

More generally, I'd like a programmatic way to recover from such scenarios -- e.g., if many requests are timing out, throw away the super pool and instantiate a new one. I could build this feature on top of the super pool and Future API but I suspect many folks will want similar behavior.

For some background, see this Twitter stream: https://twitter.com/wildsebastian/status/758937740410888192

/cc @rkuhn @ktoso

[akka-ci]

Comment by ktoso
_Tuesday Aug 02, 2016 at 15:08 GMT_

---

Thanks a lot for the write-up!
Will dig deeper into it tomorrow.

[akka-ci]

Comment by ktoso
_Tuesday Aug 02, 2016 at 15:14 GMT_

---

Now I see what you meant overt there on twitter. So:

4 requests run and then, due to the client failing to process response entities, the program hangs forever. This is a bug in this code -- the response entities must be processed. However, that's not what this ticket is about.

Is (just cross referencing for clarity): https://github.com/akka/akka/issues/18716
So let's not talk about it, as you said.

The super pool API could be made much safer by supporting some form of (optional?) timeouts on requests. For example, if the fifth request hasn't been assigned a host connector within X seconds, fail the response with a timeout exception indicating the pool was too busy.

I see, interesting idea. Rather specialised but perhaps indeed somewhat safer hm.
We have a lot of different timeouts so let's think if adding another one really is the best thing here (though I totally see your point and benefit of this)

If anyone has opinions please chime in cc @jrudolph @2beaucoup @drewhk

[akka-ci]

Comment by drewhk
_Wednesday Aug 03, 2016 at 09:45 GMT_

---

Can this be achieved by existing combinators? Attaching a timeout stage to the entitiyBytes that are received?

[akka-ci]

Comment by schmitch
_Saturday Aug 06, 2016 at 09:42 GMT_

---

From my understanding this is about a "request timeout" not specific to the entity that is given.
So wouldn't that mean that just attaching a timeout stage to the entityBytes would still mean that the request will hang when there is a network failure? actually I guess it will hang until akka timeout is provided but I guess the issue is all about configuring that timeout and apply it to the whole request?

Something like play-ws timeout:
https://www.playframework.com/documentation/2.5.x/ScalaWS#Request-with-timeout
with the only exception that akka-http can provide a timeout when the dns hangs.

So I guess there are 3 stages where a timeout could occur:

• DNS
• Connecting to the Server
• Gathering the Data from the Server to the client

Most people wouldn't want that a request _overall_ will take too much time, so they will fail the whole and maybe retry it or circuit break it.

For instance play-ws has 3 different timeout parameters (globally):

• connectionTimeout: Duration = 2.minutes
• idleTimeout: Duration = 2.minutes
• requestTimeout: Duration = 2.minutes

plus the requestTimeout could be overwritten per request

[jrudolph]

There's also another location where time is spent: in the queue. It would be nice if a global and per-request timeout can be set that would prune a request from the queue before it is even dispatched. This would also help reducing load on a pool when requests are already slow.

[ktoso]

Bumping by public demand.

[ktoso]

I marked it as pick next so we can discuss it in depth some more.