Hyper: Converting Hyper::Body to Vec<u8>

[janus]

Please how do I get the below issue resolved. I have tried to use concat2 and try_concat but to no avail serde_json::from_slice expects bytes.

 use std::future::Future;
 use hyper::Client;
 use hyper_tls::HttpsConnector;
 use serde_json;
 async {
          let mut request = hyper::Request::builder().method(.....);
     let https = HttpsConnector::new().expect("TLS initialization failed");
     let client =  Client::builder().build::<_, hyper::Body>(https);
     let response = client.request(request).await;
     response.unwrap().into_body()
     let value: serde_json::Value = serde_json::from_slice(&body).unwrap();
};

error[E0308]: mismatched types
--> src/actions/request.rs:235:67
|
235 | let value: serde_json::Value = serde_json::from_slice(&body).unwrap();
| ^^^^^ expected slice, found struct hyper::Body
|
= note: expected type &[u8]
found type &hyper::Body

[jakedeichert]

Not sure if this is what you might be looking for?

req.into_body().concat2().map(move |chunk| {
    let body_bytes = chunk.iter().cloned().collect::<Vec<u8>>();
    // then use with serde...
});

[fundon]

use futures::stream::TryStreamExt;
request.body_mut().try_concat().await.unwrap().to_vec()

[NeoLegends]

Maybe you'd want to be using reqwest instead? It builds on top of hyper and can directly parse responses as JSON, for example.

[janus]

use futures::stream::TryStreamExt;
request.body_mut().try_concat().await.unwrap().to_vec()

@fundon
I got this ;

error[E0599]: no method named `try_concat` found for type `hyper::Response<hyper::Body>` in the current scope
   --> src/actions/request.rs:232:20
    |
232 |           response.try_concat().await.unwrap().to_vec()
    |                    ^^^^^^^^^^ method not found in `hyper::Response<hyper::Body>`
    |
    = note: the method `try_concat` exists but the following trait bounds were not satisfied:
            `&hyper::Response<hyper::Body> : futures_util::try_stream::TryStreamExt`
            `&mut hyper::Response<hyper::Body> : futures_util::try_stream::TryStreamExt`
            `hyper::Response<hyper::Body> : futures_util::try_stream::TryStreamExt`

What do I do next?

[janus]

Maybe you'd want to be using reqwest instead? It builds on top of hyper and can directly parse responses as JSON, for example.

I am upgrading to async await ... So I cant change crate.

[NeoLegends]

The next version of reqwest will ship with async/await support, the alpha releases already feature it and are quite usable.

[janus]

https://github.com/hyperium/hyper/blob/master/examples/echo.rs
https://github.com/hyperium/hyper/blob/master/examples/client_json.rs

I checked out the above, and it is now obvious to me that there is something wrong with my code.
Could it be coming from, let response = client.request(request).await;

[janus]

My Cargo.toml
[dependencies]
clap = "2.32.0"
colored = "1.7.0"
csv = "1.0.5"
hyper = "0.13.0-alpha.4"
hyper-tls = "0.4.0-alpha.4"
regex = "1.1.2"
serde_json = "1.0.39"

time = "0.1.42"
yaml-rust = "0.4.3"
linked-hash-map = "0.5.2"

futures = "0.3.1"

tokio = "0.1.15"
tokio-scoped = "0.1.0"

futures-timer = "1.0.3"

rustc --version rustc 1.40.0-nightly (50f8aadd7 2019-11-07)

I am beginning to think that what I am going through could be from version mismatch

[fundon]

@janus You missed this body_mut(), try it

use futures::stream::TryStreamExt;
response.body_mut().try_concat().await.unwrap().to_vec()

[janus]

@janus You missed this body_mut(), try it

use futures::stream::TryStreamExt;
response.body_mut().try_concat().await.unwrap().to_vec()

error[E0599]: no method named try_concat found for type &mut hyper::Body in the current scope
--> src/actions/request.rs:231:29
|
231 | response.body_mut().try_concat().await.unwrap().to_vec()
| ^^^^^^^^^^ method not found in &mut hyper::Body
|
= note: the method try_concat exists but the following trait bounds were not satisfied:
&&mut hyper::Body : futures_util::stream::try_stream::TryStreamExt
&hyper::Body : futures_util::stream::try_stream::TryStreamExt
&mut &mut hyper::Body : futures_util::stream::try_stream::TryStreamExt
&mut hyper::Body : futures_util::stream::try_stream::TryStreamExt
hyper::Body : futures_util::stream::try_stream::TryStreamExt
= help: items from traits can only be used if the trait is in scope
= note: the following trait is implemented but not in scope; perhaps add a use for it:
use futures_util::try_stream::TryStreamExt;

error: aborting due to previous error

I have this the use futures::stream::TryStreamExt;

This issue I am having may be related to hyper v0.13.0-alpha.4

https://github.com/hyperium/hyper/tree/v0.13.0-alpha.4/examples.
Please @fundon could your try https://github.com/hyperium/hyper/blob/v0.13.0-alpha.4/examples/client_json.rs ? I would like to see your Cargo.toml file.

[fundon]

@janus Enable the unstable-stream feature.

hyper = { version = "0.13.0-alpha.4", features = ["unstable-stream"] }

[janus]

@janus You missed this body_mut(), try it

use futures::stream::TryStreamExt;
response.body_mut().try_concat().await.unwrap().to_vec()

error[E0599]: no method named try_concat found for type &mut hyper::Body in the current scope
--> src/actions/request.rs:231:29
|
231 | response.body_mut().try_concat().await.unwrap().to_vec()
| ^^^^^^^^^^ method not found in &mut hyper::Body
|
= note: the method try_concat exists but the following trait bounds were not satisfied:
&&mut hyper::Body : futures_util::stream::try_stream::TryStreamExt
&hyper::Body : futures_util::stream::try_stream::TryStreamExt
&mut &mut hyper::Body : futures_util::stream::try_stream::TryStreamExt
&mut hyper::Body : futures_util::stream::try_stream::TryStreamExt
hyper::Body : futures_util::stream::try_stream::TryStreamExt
= help: items from traits can only be used if the trait is in scope
= note: the following trait is implemented but not in scope; perhaps add a use for it:
use futures_util::try_stream::TryStreamExt;

error: aborting due to previous error

I have this the use futures::stream::TryStreamExt;

@janus Enable the unstable-stream feature.

hyper = { version = "0.13.0-alpha.4", features = ["unstable-stream"] }

I am aware of this and it was enabled in the last build it still failed. It was because I enabled it that I have this output:
= note: the following trait is implemented but not in scope; perhaps add a use for it: use futures_util::try_stream::TryStreamExt;

[janus]

@fundon
Please try one of the listed examples and see for yourself.

[fundon]

I just add to_vec() to the body. Everything is ok.

async fn fetch_json(url: hyper::Uri) -> Result<Vec<User>> {
    let client = Client::new();

    // Fetch the url...
    let res = client.get(url).await?;
    // asynchronously concatenate chunks of the body
    let body = res.into_body().try_concat().await?.to_vec();
    // try to parse as json with serde_json
    let users = serde_json::from_slice(&body)?;

    Ok(users)
}

$ cargo run --example client_json --features="runtime unstable-stream"

users: [
    User {
        id: 1,
        name: "Leanne Graham",
    },
    User {
        id: 2,
        name: "Ervin Howell",
    },
    User {
        id: 3,
        name: "Clementine Bauch",
    },
    User {
        id: 4,
        name: "Patricia Lebsack",
    },
    User {
        id: 5,
        name: "Chelsey Dietrich",
    },
    User {
        id: 6,
        name: "Mrs. Dennis Schulist",
    },
    User {
        id: 7,
        name: "Kurtis Weissnat",
    },
    User {
        id: 8,
        name: "Nicholas Runolfsdottir V",
    },
    User {
        id: 9,
        name: "Glenna Reichert",
    },
    User {
        id: 10,
        name: "Clementina DuBuque",
    },
]
sum of ids: 55

@janus Please show ur real example.

[fundon]

Use

    let mut res = client.get(url).await?;
    let body = res.body_mut().try_concat().await?.to_vec()

instead of

let res = client.get(url).await?
let body = res.into_body().try_concat().await?;

It's ok too.

[LucioFranco]

Hyper depends on futures 0.3.0-alpha.19 not futures 0.3 so the stream traits do not line up.

[fundon]

So, Must hyper = { version = "0.13.0-alpha.4", features = ["unstable-stream"] } + futures-preview = "=0.3.0-alpha.19". dont use futures 0.3 on current version.

[klausi]

Same problem, the compiler simultaneously recommends futures_util::stream::try_stream::TryStreamExt; and cannot use it:

error[E0432]: unresolved import `futures_util::try_stream`
 --> src/lib.rs:9:19
  |
9 | use futures_util::try_stream::TryStreamExt;
  |                   ^^^^^^^^^^ could not find `try_stream` in `futures_util`


error[E0599]: no method named `try_concat` found for type `&mut hyper::body::body::Body` in the current scope
   --> src/lib.rs:120:62
    |
120 |                         let body_bytes = response.body_mut().try_concat();
    |                                                              ^^^^^^^^^^ method not found in `&mut hyper::body::body::Body`
    |
    = help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope, perhaps add a `use` for it:
    |
1   | use futures_util::stream::try_stream::TryStreamExt;
    |

error: aborting due to 2 previous errors

Code is in https://github.com/klausi/rustnish/tree/hyper-await

Can anyone post a combination of cargo futures dependencies that is working right now? Would be super useful to be able to read bytes from a response body :-)

[orballo]

I am using only:

futures-preview = '0.3.0-alpha.19'

[dependencies.hyper]
version = '0.13.0-alpha.4'
features = ['unstable-stream']

and

use futures::stream::TryStreamExt;

And works just fine.

[leshow]

Ran into this also, you have to do as @orballo says, turn on the feature and use the preview version of futures

[hwchen]

Not turning on the unstable-stream feature has bitten me a few times already, I hope streams can be stabilized soon. https://github.com/hyperium/hyper/issues/1920