Json: Consider adding async methods

[VictorKoenders]

With the upcoming stabilization of async/await, it would be nice to add a method that would allow developers to leverage the async ecosystem.

The goal seems to be to make futures (now futures-preview) the go-to library that everyone builds upon*.

• The trait to read from a stream is futures::io::AsyncRead.
• The trait to write to a stream is futures::io::AsyncWrite.

Seeing as:

• async/await is not to be stabilized before 1.39,
• and not everyone might want to use this feature,
• and the minimum required version of serde would have to be raised
  I think it would be wise to put this behind a feature flag, e.g. async or async-std

This would add methods to (names are bikesheddable):

• serde_json::: from_async_reader, to_async_writer and to_async_writer_pretty
• serde_json::Serializer::: new_async, pretty_async, with_async_formatter
• serde_json::Deserializer::: new_async, from_async_reader

*) Judging from async_std::fs::File which implements async_std::io::read which is a re-export of futures::io::AsyncRead

[0xpr03]

The question is what exactly you want serde to do here. You can already use futures together with serde.
The thing you're probably looking for is streaming json support. I'm not sure how well this works at all with serde ser-/deserialization. So you probably always have to first collect the whole buffer of data and then de-/serialize it. The only thing serde could do streaming is verification of the JSON format.

[pickfire]

@VictorKoenders This is not just about adding to serde_json, I believe all the other serde_* requires this as well. I am wondering if from_reader could read from AsyncRead.

[belak]

So, the main problem with this is that the entire definition of a serializer/deserializer would need to be made async because for this to work because the deserializers read from the input assuming it's a buffer. In order for this to work, all serde traits would need to be async (or have async variants) which wouldn't be a trivial undertaking so they could call async functions inside them.

[dtolnay]

I think I would prefer not to build this into serde_json, but I would be interested to see someone else develop an async-first JSON library.

[hjfreyer]

An alternative to choosing between a "async" or "sync" model would be to remove the "inversion of control" from the library. That is, currently you construct a Deserializer by giving it a Read, and then the Deserializer is responsible for calling the Read, meaning it is sensitive to any changes to the Read API. It's understandable to have said, "well the Read API is never gonna change", but then again, here we are :P.

The alternative would be to have the Deserializer trait have a next_char(u8) method. The caller feeds in one character at a time, and the Deserializer manages a state machine internally, indicating at each character whether the string so far is a complete value, or incomplete, or invalid. This would be a totally self-contained and functional API, so the caller is totally in control of the flow. It's simple to build both the current API and an async API on top of such an implementation, and perhaps other APIs that callers could dream up in the future.

[Note: rather than accepting one character at a time, you'd probably want to have a next_chars(&[u8]) -> usize method that allows short writes like std::io::Write, for efficiency's sake.]

Alas, it looks like that would probably require changes to serde itself. Perhaps as async_std becomes more pervasive, serde will have to do a major revision, and maybe a change to a functional interface like this would be warranted.

[hjfreyer]

Hah, it seems there's already a proposal for that in the serde repo: https://github.com/serde-rs/serde/issues/768. I'll cross-post there.