Tokio: Improve codec's documentation

[Darksonn]

The documentation is currently rather poor for new users.

[carlosb1]

what is the status of this issue? Is it closed in the the other issue (number 45)?. could it be good to have some guidelines about what it is necessary for this documentation

[Darksonn]

This is still very relevant. Basically I am looking for an improvement such that I no longer have to link people to this when they ask for help. The following things would be useful:

1. A general explanation of the methods that are used for Decoder. People very often don't understand how (or even that) they are supposed to handle incomplete frames.
2. A similar explanation for Encoder. This one is simpler.
3. A full example of a Decoder.
4. A full example of an Encoder.

[Darksonn]

The documentation should also mention the StreamReader and ReaderStream types that are also in tokio-util, since StreamReader is a special case of an decoder with a BytesCodec.

[carlosb1]

Ok, I started with an example that it works.

[Darksonn]

I would prefer an example that actually deals with the kind of data that has well defined boundaries, since otherwise it doesn't demonstrate when to use Ok(None), and how to manipulate the data remaining in the BytesMut. For example, the stream could be a sequence consisting of 4 byte length markers followed by that number of bytes.

[carlosb1]

No problem, it was only a template. Anyway, I can use another parser like serde-json or I can get ideas from LengthDelimitedCodec . Furthermore, I can keep sending my doubts via discord if it is easier
.

[Darksonn]

Sure, just ask any questions in the discord.

[carlosb1]

I added here a tutorial and here the code. I can do the PR if it is enough or we can discuss about more changes.

[Darksonn]

There are some problems with the decoder examples in your tutorial. The main thing I need the tutorial to explain is Framed does _not_ know where chunks start and end. That's the job of the decoder! The data in buf may not yet be a full JSON object, and in that case it should return Ok(None) to wait for more (additionally, buf.len() will never be zero). There may also be more data than a single JSON object, and it needs to handle that by only removing part of the data in the buffer.

I would prefer if we start with a much smaller piece of text we can put on docs.rs to explain this. We can add a section to the website after we have improved the text on docs.rs itself.

[carlosb1]

I see that you has another PR for this, if you need some help. I open to contribute!