Tokio: Where's tokio-process?

[andreycizov]

This: https://github.com/alexcrichton/tokio-process repo is archived and directs me here saying it was integrated in tokio.

Does tokio still have this functionality?

[LucioFranco]

@andreycizov Hi, it has gotten merged into tokio-net.

[aymericbeaumet]

The documentation can now be found at tokio_net::process.

This is what you should put in your _Cargo.toml_:

[dependencies]
tokio-net = { version = "0.2.0-alpha.5", features = ["process"] }

[nathan-at-least]

As a relative noob to rust, it took me about 90 minutes of effort to find this fix. I think the best fix would involve improving the api doc gen toolchain (so not this repo). However, given the current doc toolchain's behavior, maybe the tokio doc strings can be improved to help people like me.

Here was my process:

Background:

• I'm familiar with async programming frameworks from Python twisted and some javascript promise-like stuff.
• rust is my new favorite language, so I'm spending all my available skills improvement time focusing on it.
• I'm used to the rust API doc UI (I frequently use and love rustup doc or cargo doc --open).
• I am aware of, but not experienced with, tokio and want to learn it.
• I am vaguely aware that there's been a major backwards incompatible migration process, or multiple, in the tokio world, so I know to double check every example to figure out which era it's from.
• I wanted to write a utility that runs multiple child process, multiplexes their output and presents it all on stdout, with each line prefixed by the pid / stdout / stderr details of the child process source, so I thought it would be a perfect learning opportunity to practice my rust + tokio skillset.

Process:

1. Googled for "tokio process", quickly found https://github.com/alexcrichton/tokio-process and saw the disclaimer in the README just as this ticket's OP.
2. Clicked the tokio link in that repo's README.
3. Clicked the API Docs link in that tokio repo's README near the top after the first section.
4. Typed S to search, and typed process, then clicked the only hit for tokio::net::process which links to https://docs.rs/tokio/0.2.0-alpha.6/tokio/net/process/index.html
5. I see at the top of the page this API doc is for tokio-0.2.0-alpha.6.
6. I put tokio = 0.2.0-alpha.6 into my project's dependencies.
7. I run cargo doc --open.
8. In the local API doc UI, I search for process and it's nowhere to be found.
9. Get lost, flail around, try different versions of tokio, directly depending on tokio_net, etc…
10. Finally find this ticket, see the suggestion in the previous comment, update my project (except I use alpha.6 rather than alpha.5) and now cargo doc --open let's me search for and find tokio_net::process.

So how can users (especially noobs) discover that they need the process feature? I don't see that in any documentation. I understand that features are such a standard part of the toolchain that experienced rust devs can probably dig into source code or other resources to figure out which features exist. But what about noobs?

Two fixes:

a. Manually tweak the tokio_net::process API docs to explicitly say "This module is only available when the process feature is enabled. To enable that feature for your project, do …".
b. Improve the rust doc generation toolchain to automatically add solution a. whenever it can detect modules are feature-dependent.

Obviously, a. could be resolved within this repo and b. will require modifying other tools. I'm not yet sure which repos are best for approach b, but if anyone points me to them I'll go pester those devs. >;-)

Also, I realize open source docs are super hard, so the next available time for this project, I intend to submit a patch for solution a. myself.

[carllerche]

Thanks for the report. We will address this before the final. I will try to write up a proposal in the next week or so.

[carllerche]

For one, we should have tokio::process in the main crate. I think that is probably the biggest problem right now.

[carllerche]

@ipetkov are you able to include a re-export?

[nathan-at-least]

Hm, that sounds like one lingering confusion I have: https://docs.rs/tokio/0.2.0-alpha.6/tokio/net/process/index.html documents tokio::net::process whereas my local docs refer to tokio_net::process. I don't understand the discrepency (yet). Relevant snippet:

[dependencies]
tokio = "0.2.0-alpha.6"
tokio-net = { version = "0.2.0-alpha.6", features = ["process"] }

Are both dependencies necessary?

Also a rust noob question: if my binary depends on A and A depends on B, can I use B::foo in my project, or _must_ I explicitly declare B as a dependency?

[carllerche]

We should avoid re-exporting process and signal from tokio::net and just have tokio::{process, signal}.

We do need to go around and clean all this up.

[ipetkov]

@carllerche yes I can move the reexports under tokio::{signal, process} instead of tokio::net::{signal, process}. This might lead to some different confusion for those who see the exports available under the tokio_net crate, but I figure we'll have this dust settle before the official 0.2 release. Looking forward to your proposal!

Also a rust noob question: if my binary depends on A and A depends on B, can I use B::foo in my project, or _must_ I explicitly declare B as a dependency?

@nathan-at-least you must explicitly declare B as a dependency, which allows cargo to figure out what version fits both yours and A's requirements. If you are already depending on the tokio crate, you can enable it's process feature (which in turn will depend on tokio_net and turn on its process feature):

[dependencies]
tokio = { version = "=0.2.0-alpha.6", features = ["process"] }

[carllerche]

@ipetkov is process enabled by default for tokio? Ideally it should be.

[ipetkov]

@carllerche it is now in #1643!

[ipetkov]

The reexports have landed in master

[LucioFranco]

Can we close this then?

[nathan-at-least]

Wow, this has been an extremely productive and positive experience! Thanks for the quick response, work towards a resolution, and answering my basic questions. I'm motivated to keep contributing to tokio as I go. <3

@ipetkov your suggested Cargo.toml dependency now works for me and opens local docs with the usability I expect.

So my current understanding is that tokio::net is reexported by tokio out of tokio_net, but this also is an implementation detail that application developers needn't know, since tokio is designed to be the application's primary interface for all of tokio. Furthermore an upcoming release of tokio will expose tokio::process without involving the tokio::net namespace. Is all of that understanding correct?

[ipetkov]

@nathan-at-least

So my current understanding is that tokio::net is reexported by tokio out of tokio_net, but this also is an implementation detail that application developers needn't know, since tokio is designed to be the application's primary interface for all of tokio.

Yes, if you are an application author you should depend on tokio directly and turn on whatever non-default features you may need (we will be revisiting what features are on by default as we get closer to publishing 0.2 after the alphas settle). The other tokio-* crates are more geared towards library authors that can depend only on the features they need rather than pulling in all of tokio.

Furthermore an upcoming release of tokio will expose tokio::process without involving the tokio::net namespace. Is all of that understanding correct?

Yes we've landed a change in master where the process and signal APIs are exported from tokio::{process, signal} (the definitions still live in the tokio_net crate, though we may be revisiting crate organization before finalizing the 0.2 release).

I'm not sure if these changes have landed in the latest alpha, but if they haven't, they will soon! (My recommendation is to depend directly on the tokio crate if you are writing an application. We will not be removing any API exports from it in the near future, and right now all the public process APIs are exported there)

[ipetkov]

I'm going to close this issue out as it looks like we've reached any immediate resolutions for now.

We will be revisiting any API exports/crate organization/documentation before the 0.2 release so we can track that progress separately.

If anyone has any other questions, feel free to reopen this or a new issue!

[carllerche]

Soon, everything will just exist in tokio and we will be able to avoid this.