Rusoto: Meta: Ideas for consolidating and improving documentation

[srijs]

Hi! This is just me thinking aloud on Rusoto documentation and looking for a bit of feedback on my thoughts. I'll try to add to this over the next days, but would appreciate some early feedback!

Current State

Discoverability

Currently, Rusoto documentation lives in many different places, which can be confusing for newcomers:

• docs.rs automatically publishes reference-level documentation for the rusoto API, but it's lacking discoverability in the sense of being able to easily browse between different rusoto crates
• https://rusoto.github.io/rusoto/ is a reference-level documentation for the rusoto API, hosted on github pages, that gets pushed on every integration into master
• rusoto.org is a guide-level documentation published via git books (repo)
• https://matthewkmayer.github.io/blag/public/post/rusoto-rds-mk2/ is a guide-level documentation covering rusoto_core and rusoto_rds

Updates

Some of the documentation is written manually and at risk of getting stale or out-dated.

For example, the Rusoto gitbook includes a section on the supported AWS services, which has to be kept in sync manually.

Also, there is no automated testing for example code being published in the source code documentation, the git book guides, or the guide on @matthewkmayer's blog. This means that due to breaking changes code could stop compiling without visibility into the issue.

Ideas for Improvement

• [ ] Think about possible entry points for newcomers (via docs.rs, via crates.io, via github), and make sure that it's possible to find the relevant documentation (user is pointed at crates.io, how can we funnel them into the getting started guide?)

• [ ] Investigate a "main" starting point (rusoto.org?) which provides an overview over Rusoto and links to all other resources (similar to tokio.rs)

• [ ] Investigate enabling testing for example code in source code documentation (probably needs AWS credentials set-up in Travis)

• [ ] Investigate auto-generating the list of crates on the "Supported AWS Services" page

• [ ] Investigate adding support to docs.rs to provide discoverability (maybe somehow tie-in with Cargo workspaces?)

• [ ] Investigate using rust-skeptic or literate programming (see tango) for guide-level docs so that code examples in the guides can be type checked or even properly tested

[Mythra]

Thanks for the great write up on this. This is totally something we've needed for awhile (as each you mentioned pretty much has been talked about in an existing issue: #754 , #747 , #685 , #670, #669, #318 ).

I do like having it all in one place though. I think a good first step would be getting on docs.rs, and aws creds in travis (something we need to lower build times for because travis can get creds now).

The rest sound like good ideas, but I think those can wait a bit. That's just my opinion though, I'm curious what would you like to see get done first?

[matthewkmayer]

Yes, thanks for bringing this all into one spot!

We've worked at ensuring the rusoto_core crates.io page has the appropriate links: Homepage goes to https://rusoto.org and documentation goes to api docs on github pages .

I've recently updated the rusoto.org source to remove stability tags from services in order to make automating the supported services list easier. However, we haven't had much action in that area yet.

The biggest bang for our buck is probably having our sample code run. We have some locked-down AWS creds in Travis for SCCache ( https://github.com/rusoto/rusoto/pull/833 ). Being able to ensure the DynamoDB sample in our README runs would be 👍 .

If I could wave a magic wand and have one thing, a comprehensive https://rusoto.org site would make me really happy. 😄

[srijs]

Just to create some visibility here, @jonhoo has recorded a great live-coding video where he's using Rusoto to spin up EC2 spot instances.

Watching the part where Jon was integrating Rusoto made me cringe a lot (to be clear, entirely _not_ his fault) due to all the problems he ran into with our documentation.

The YouTube link to jump to the beginning of the Rusoto part is here: https://www.youtube.com/watch?v=Zdudg5TV9i4&feature=youtu.be&t=2190

The video is worth a watch and I think does an excellent job at shining an even brighter light at some of the pain points with our current documentation setup, and should help us understand better where we can improve.

I'll try to find the time later to provide a written summary of the roadblocks from the video.

[matthewkmayer]

This is the first time I've seen someone find our outdated gitbook published to github pages. I've made https://github.com/rusoto/rusoto.github.io/issues/80 to track fixing that to redirect to rusoto.org which is the correct, up-to-date site.

Thanks for the link to that video!

[matthewkmayer]

I'm strongly considering relying on docs.rs for published docs. It's accurate for our published crates and lets of skip doc generation in Travis, which is a big contributor to our slow build times on the stable Rust on Linux build. We should verify they look as expected first since we do some massaging of docs from botocore.

[srijs]

Sounds good to me!

This will also remove the problem of the published documentation being generated from the master branch instead of the latest release.

I still have that branch where I attempt to do conversion of the weird AWS documentation format into proper Markdown, which might improve the formatting and help us get rid of the hoedown hack.

Maybe we could also consider contributing a feature to docs.rs to group certain crates (maybe based on workspace?) to improve the discoverability, but I'm not sure how that could look like yet.