Cargo: Run doc tests on README

[huonw]

cargo test could run (the equivalent of) rustdoc --test README.md, since people often have a basic example in it.

[tomjakubowski]

I imagine people might have many markdown files which are documentation and contain examples; it would be nice to be able to specify additional .md files to run rustdoc --test on in Cargo.toml.

[alexcrichton]

Sounds like a good idea! We could at least start out with a README and expand it with a manifest key later on.

[alexcrichton]

Some concerns were raised in #445 which are applicable to this as well.

[tomaka]

The question is: how will README interact with the Cargo registry?

Currently the examples are in the README file mainly because it's easy to read when you discover a library on github.

But if the registry automatically provides the rustdoc output of each package, then I think that the basic examples should simply be in the crate root documentation, just like the standard regex and green crates do.

[skade]

I'd like to have this feature. Broken README code is worse then broken code in the crate docs, as this is the first place where people go to. Outdated READMEs are even worse of an issue, a thing tested READMEs elegantly circumvent.

Also, I prefer writing README code, as crate documentation is a bit cumbersume and hard to read in source, given the amount of comment characters it needs.

Rust code can be highlighted on github and ignored by using:

```{.rust .ignore}

(this currently is parsed correctly, but not properly picked up. {.ignore .rust} works. I'll file a bug on rustdoc after validating)

As a middleground, my suggestion is to allow listing additional documents to test with rustdoc in the Manifest and generate a Cargo.toml including "README.md" in that list per default. This raises awareness of the feature, but ultimately leaves the user in control.

[skade]

I'd like to push this one, I just ran into precisely the problem that my README examples were the only ones broken.

[pzol]

+1

[pythonesque]

I would love this.

Concerns can be addressed by making the feature opt-in (and maybe also with attribute metadata, but I don't know markdown well enough to know if that's a thing).

[moonglum]

:+1:

[s-panferov]

Always forget to update README.md in my projects :+1:

[huonw]

cc https://github.com/brson/rust-skeptic

[carols10cents]

potentially relevant: https://github.com/rust-lang/rfcs/pull/1990

[abonander]

The external docs include feature (RFC 1990) is definitely the best way to go about this, IMO. You could include your README.md as the docs for a hidden item and then rustdoc should be able to automatically test the examples. There's a bit of engineering effort left to make it stabilizable though.

[abonander]

In case anyone missed it, I've got a proof-of-concept project for using the external_doc feature to test code examples in README.md: https://github.com/abonander/readme-doctest-poc

This can be done conditionally under a Cargo feature so it doesn't require nightly to test.

cc https://github.com/rust-lang/rust/issues/44732#issuecomment-456673619

[joshtriplett]

I'd like to revive this. I've seen projects do CI on their README by manually running rustdoc test README.md and specifying their crate as an --extern.

[shepmaster]

I've been happily using doc-comment to do this.

[ibraheemdev]

For anyone who doesn't want to add another dependency, this little macro works perfectly:

#[cfg(doctest)]
mod test_readme {
  macro_rules! external_doc_test {
    ($x:expr) => {
        #[doc = $x]
        extern {}
    };
  }

  external_doc_test!(include_str!("../README.md"));
}