Akka-http: Unmarshalling issue: Substream Source cannot be materialized more than once

[todor91]

Hi everyone. I am experiencing some strange behavior with akka-http and unmarshalling.

The problem is the following:
My route accepts a single JSON object(modeled by case class) and an array of JSON objects. When I query the route with array of JSON objects and chunked transfer encoding like so:

curl -XPOST -H "Content-type: application/json" -H "Transfer-Encoding: chunked" -d @multipleRequests.json 'localhost:8080/testRoute'

I get the error message:

The request content was malformed:
Substream Source cannot be materialized more than once

When I pass a single object(not an array of them) everything works fine.

Route is defined as

val route = path("testRoute") {
    post {
      entity(as[SearchRequest]) { request =>
        complete {
          "Processing single request"
        }
      } ~ entity(as[Seq[SearchRequest]]) { request =>
        complete {
          "Processing multiple requests"
        }
      }
    }
  }

And when I switch places of these two entity calls I can submit an array of objects without any problems, but then submitting a single JSON object fails with the same error message described above.

curl -XPOST -H "Content-type: application/json" -H "Transfer-Encoding: chunked" -d @singleRequest.json 'localhost:8080/testRoute'

The only way I was able to run it normally in both cases is by switching both of these entity calls with a single entity(as[JsValue]) and then doing some if-then-else logic, which is far from convenient.

Has anyone experienced anything like this?

I've created a simple reproducer: https://github.com/todor91/akka-reproducer
Inside resources folder there are files containing valid json objects.

[ktoso]

Wrap your routes in a toStrict, since then it'll materialize once.

The root is the same as: https://github.com/akka/akka-http/issues/618 so
I'm inclined to close this as duplicate.

Long story short:

• to get the entity into a unmarshalled form like you try to here is
  causing the stream (from tcp directly) to start reading
• then it turns out that it's not that entity, so it rejects
• second branch is executed, again needs to materialize, but can't since it
  was already drained and the data is not accumulated in memory (which is how
  we can say it indeed is streaming, and not just putting the entire thing
  into memory.
• so you should make sure it does (starts streaming), once and keeps the
  data, that's what the toStrict directive (or method on request) does.

Tickets about making this magically work are:

• https://github.com/akka/akka-http/issues/117
• and somewhat related: https://github.com/akka/akka-http/issues/183

--
Konrad ktoso Malawski
Akka http://akka.io @ Lightbend http://lightbend.com

On 10 January 2017 at 13:59:43, Ivan Todorović ([email protected])
wrote:

Hi everyone. I am experiencing some strange behavior with akka-http and
unmarshalling.

The problem is the following:
My route accepts a single JSON object(modeled by case class) and an array
of JSON objects. When I query the route with array of JSON objects and
chunked transfer encoding like so:

curl -XPOST -H "Content-type: application/json" -H "Transfer-Encoding:
chunked" -d @multipleRequests.json 'localhost:8080/testRoute'

I get the error message:

The request content was malformed:
Substream Source cannot be materialized more than once

When I pass a single object(not an array of them) everything works fine.

Route is defined as

val route = path("testRoute") {
post {
entity(as[SearchRequest]) { request =>
complete {
"Processing single request"
}
} ~ entity(as[Seq[SearchRequest]]) { request =>
complete {
"Processing multiple requests"
}
}
}
}

And when I switch places of these two entity calls I can submit an array of
objects without any problems, but then submitting a single JSON object
fails with the same error message described above.

curl -XPOST -H "Content-type: application/json" -H "Transfer-Encoding:
chunked" -d @singleRequest.json 'localhost:8080/testRoute'

The only way I was able to run it normally in both cases is by switching
both of these entity calls with a single entity(as[JsValue]) and then doing
some if-then-else logic, which is far from convenient.

Has anyone experienced anything like this?

I've created a simple reproducer: https://github.com/todor91/akka-reproducer
Inside resources folder there are files containing valid json objects.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/akka/akka-http/issues/745, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHYkzsVJB-RPGo_l16jo17bv7JRuJEYks5rQ4C9gaJpZM4LfYe_
.

[todor91]

Yeah, that seems to fix the problem.
Thanks for the explanation.