Rust: Move the doc directory to separate repo

[arijit79]

The doc directory contains documentation for various tools and also submodules for various other learning resources. I think it should be moved out to be on its own separate repo. This would help documentation writers like us to clone only the docs without cloning the entire compiler sources. Plus it has become a quite large directory with many files

[est31]

Conversely, when you clone the rust repo wanting to change the compiler or standard library, cloning all the doc related submodules takes a lot of time.

[arijit79]

Exactly. Its a waste of bandwidth and storage when I just need to change 1 or 2 words in the docs but need to clone the compiler sources

[arijit79]

I am willing to help to move the docs. But just need a confirmation that whether any member of the Rust team will work on that or shall I myself start it

[petrochenkov]

Majority of subdirectories in src/doc are already submodules living in separate repos (^*) (see https://github.com/rust-lang/rust/tree/master/src/doc), only compiler-internal docs edited together with the compiler are living in tree.

^*which is pretty annoying because you cannot simultaneously update the docs when making changes in the compiler or library, so I personally stopped contributing to docs when they were moved out.

when I just need to change 1 or 2 words in the docs

FWIW, this can be done through GitHub interface (the "Edit this file" button) without cloning the repo.

[arijit79]

FWIW, this can be done through GitHub interface (the "Edit this file" button) without cloning the repo.

I had already tried to do what you have suggested. I had a PR where I needed to merge 6 commits but then the assignees and the reviewers said that I needed to squash the commits. And for doing that I needed to clone the fork repo

EDIT: As for the submodules, there are many directories inside the doc folder like rustdoc which are not submodules

[arijit79]

Hey guys. I just wanted to ask whether I should work on to fix this issue

[steveklabnik]

This has gone back and forth historically; there are pros and cons of each approach. When the docs were in the repo, people were concerned about bors queue length. So we moved them out, but that made coordinated changes difficult.

Any changes here would involve changing the build + deployment story, so @rust-lang/release should be at least consulted if there's a decision to change any of this.

[Mark-Simulacrum]

No, release team doesn't really care - infra might, but also not really (more or less not important whether these are submodules or not).

I think the next change to make here is probably to go through these repositories and try to move some of the larger images or artifacts to static.r-l.o or similar, there's really no reason to have them in those repositories.

[arijit79]

What is static.r.l.o. Could you please explain that to me

As far as doc directory is concerned, I mostly care about the docs of various tools such as rustdoc (and rustc maybe, if its not involved in any Ci testing process) as well as the man pages

[petrochenkov]

If some doc subdirectory is moved to a separate repo, then it should probably use a git subtree like clippy does.
(At least from rust-lang/rust side we can declare the clippy subtree experiment to be a great success at this point.)

[tmandry]

Agreed with @petrochenkov, I think we should pursue a subtree approach (#70651 has more details). This gives us the best of both worlds for contributors: people who only care about docs get to clone only that repo, and people working in the compiler get to make coordinated changes easily.

[camelid]

What is static.r.l.o. Could you please explain that to me

My understanding is that it hosts build artifacts etc. for Rust. For example, when you download a new version of Rust it is coming from static.rust-lang.org (abbreviated as static.r-l.o).

[steveklabnik]

@petrochenkov @tmandry I don't know a ton about subtree, but would be interested in hearing more about it; maybe there's some examples of how it's worked or you could ping me on zulip? that does sound like the best of both worlds, but I didn't realize it was possible.

[tmandry]

@steveklabnik This section in the guide has the commands we use for clippy. I'm not too familiar with it but @oli-obk and @Manishearth probably have the most hands-on experience.

[oli-obk]

Unfortunately at this time we still need to use an unreleased version of git (details in https://github.com/rust-lang/rust-clippy/issues/5565), because the rust git repo just has too many commits and merge commits.

Edit: Note that this git version is only necessary for people synchronizing the repositories, not for working with either of them at all.

[arijit79]

If the subtree approach gives best of both worlds, then I am fine with that

[Manishearth]

Note that if we are doing subtrees, some team needs to commit to owning the synchronization, or we need to introduce automation for it.

[arijit79]

An automation would be much better

[Manishearth]

Someone needs to build it and commit to running it.