Akka-http: HTTP message must not contain more than one Host header

[amdelamar]

I'm trying to make calls to a REST service thats returning duplicate Host headers in its responses. We were using play-ws client before, but AkkaHttp client refuses to accept these responses:

[ERROR] [04/27/2020 15:18:09.894] [sttp-akka.actor.default-dispatcher-2] [akka.actor.ActorSystemImpl(sttp)] Outgoing request stream error
akka.http.scaladsl.model.IllegalResponseException: HTTP message must not contain more than one Host header
    at akka.http.impl.engine.client.OutgoingConnectionBlueprint$PrepareResponse$$anon$3.onPush(OutgoingConnectionBlueprint.scala:191)
    at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:523)
    at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:409)
    at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:606)
    at akka.stream.impl.fusing.ActorGraphInterpreter$SimpleBoundaryEvent.execute(ActorGraphInterpreter.scala:47)
    at akka.stream.impl.fusing.ActorGraphInterpreter$SimpleBoundaryEvent.execute$(ActorGraphInterpreter.scala:43)
    at akka.stream.impl.fusing.ActorGraphInterpreter$BatchingActorInputBoundary$OnNext.execute(ActorGraphInterpreter.scala:85)
    at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:581)
    at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:749)
    at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:764)
    at akka.actor.Actor.aroundReceive(Actor.scala:539)
    at akka.actor.Actor.aroundReceive$(Actor.scala:537)
    at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:671)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:614)
    at akka.actor.ActorCell.invoke(ActorCell.scala:583)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:268)
    at akka.dispatch.Mailbox.run(Mailbox.scala:229)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:241)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

From lines: https://github.com/akka/akka-http/blob/master/akka-http-core/src/main/scala/akka/http/impl/engine/parsing/HttpMessageParser.scala#L166-L168

If I could update this external service I would, but I'm stuck with it. So this error is preventing me from using Akka Http client altogether. :(

Is there any chance we can change this behavior, to maybe log a warning and use the first value? Perhaps a config setting ignore-duplicate-headers= on/off. Or more explicitly list of headers["host"]. I like the idea behind validating these headers; it makes sense for using Akka Http server-side, but for client-side this is a problem.

[raboof]

That makes sense!

For requests we want to keep the current behavior by default (as duplicate host headers could be used to try and confuse firewalls). Since AFAICS responses shouldn't have Host headers anyway (https://tools.ietf.org/html/rfc7230#section-5.4), there perhaps the check isn't needed.

I think such an option would be fine as long as it doesn't impact performance. Would you be interested in contributing?

[jrudolph]

Maybe we don't even need an option and just disable the check for Host in the response parser?

[amdelamar]

@raboof Yeah the Host header can be manipulated I think via proxy/mitm. As for contributing, I'd have to get OSS approval from my employer before doing anything.

@jrudolph Disabling this check outright is fine too.

[ennru]

• There is currently no test asserting the detection of a duplicated host header.
• Should HTTP response parsing even ignore other duplicated headers and just read the first?

[raboof]

Should HTTP response parsing even ignore other duplicated headers and just read the first?

I think it should treat multiple Host headers the same as multiple headers of any other name (keeping them all and letting the user figure it out)

[ennru]

(keeping them all and letting the user figure it out)

The current parser errors for duplicates of Content-Length (with a different value), Content-Type (with a different value), and Host.

[raboof]

The current parser errors for duplicates of Content-Length (with a different value), Content-Type (with a different value), and Host.

Right, and the proposal of this issue is to ignore duplicate Host headers when parsing responses (since a Host header in a response doesn't really have any official meaning anyway)

[jrudolph]

The current parser errors for duplicates of Content-Length (with a different value), Content-Type (with a different value), and Host.

Right, and the proposal of this issue is to ignore duplicate Host headers when parsing responses (since a Host header in a response doesn't really have any official meaning anyway)

Agreed. In general, some headers might have a defined meaning of multiple headers of the same sort while others don't.

[ennru]

Fixed with #3158