Rfcs: Proposed refactoring to implement core::io

Created on 25 Dec 2017 · 10Comments · Source: rust-lang/rfcs

Label: T-libs

There have of course been several efforts to refactor std::io to make it usable without std as core::io or similar.

Some links to prior art from a few hours of searching:

From what I can see all efforts are currently stalled, but I'd like to make progress on using std::io in a no_std environment. My use case is a crate I'm writing that uses std::io::{Read, Write}, and I want it to build with std and when no_std.

As mentioned in https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301, there are circular dependencies between std::io and the rest of std.

I have an idea that I've not seen mentioned: use conditional compilation in the std::io module to enable use without std. Steps required in more detail:

Move rust/src/libstd/io/*.rs to a new directory rust/src/libio_template
Add a new rust/src/libstd/io/mod.rs with just an include!("../../libio_template/mod.rs").
Add new crate rust/src/libio_nostd, which will also include! libio_template.
Add cfg feature "libio_for_std" to both libstd and libio_nostd, with it enabled and disabled by default respectively.
Patch libio_template to compile in libstd and libio_nostd.
All done. no_std consumers can use libio_nostd, libstd API should be identical, future std::io patches must be correct both in libstd and libio_nostd.

The ideal solution, as proposed in https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301, is probably a careful refactoring to decouple std, but that seems like a lot of work, which has already stalled once. A templated approach allows quick progress and can be eradicated progressively as more parts of std::io are refactored out of std.

I'm willing to work on a prototype branch or more formal RFC if there is interest from the libs team.

A-input-output A-no_std A-traits-libstd T-libs

Source

fluffysquirrels

👍16

Most helpful comment

One problem I see with the include! approach is that std::io::Read and core::io::Read would be different traits, making APIs using either incompatible with each other.

TimNN on 28 Dec 2017

👍6

All 10 comments

One problem I see with the include! approach is that std::io::Read and core::io::Read would be different traits, making APIs using either incompatible with each other.

TimNN on 28 Dec 2017

👍6

Yes, good point! They'd be different traits, but they have to be.
std::io::Error may store a heap-allocated Box, which is a great
feature that has been stabilized, but is impossible on a microcontroller
with no heap.

It would be easy to add wrappers that map between std::io and core::io,
losing some data on errors converted to core::io::Error, of course. Then
code designed mostly for no_std can simply use core::io and still
ergonomically interoperate with std::io code.

With this templated approach library authors can easily opt-in to writing
for both std and no_std environments by creating their own feature flags to
swap between std::io or core::io in their API. That's the use case I'd
imagined.

On 28 Dec 2017 8:46 a.m., "Tim Neumann" notifications@github.com wrote:

One problem I see with the include! approach is that std::io::Read and
core::io::Read would be different traits, making APIs using either
incompatible with each other.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang/rfcs/issues/2262#issuecomment-354252471,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAm2kp4rmbmr7xdEPt-NCj-PNjPnrCR0ks5tE1VegaJpZM4RMDvQ
.

fluffysquirrels on 28 Dec 2017

I like the blanket impl I propose here https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301/3 to connect the two traits.

Ericson2314 on 7 Mar 2018

👍1

@Ericson2314 Yes! I Read that thread with great interest.

I agree the platform abstraction layer sounds like a great approach. I didn't feel up to the task of starting that work and was looking for a quicker fix that could be improved later.

__Edit: misread your post.__

Yes, it should be possible to rewrite std::io as a concrete layer on top of a generic core::io layer with some wrapper structs to convert between the two, and make it all low-cost. I would be interested in prototyping that approach instead of what I proposed here.

fluffysquirrels on 17 Apr 2018

🎉1

There’s also methods like Read::read_to_vec that mention Vec which is defined in the alloc crate.