Rls: Easy to use indexing of std libs

[nrc]

This means building and distributing save-analysis info for std libs and making RLS find and use that info. The tricky part if that it must exactly match the installed libraries. I expect this will need Rustup or Cargo integration.

• [x] produce save-analysis info for nightly builds (https://github.com/rust-lang/rust/pull/37411)
• [x] make that data available on r-l.o
• [x] download the data corresponding to current libs using rustup (or distribute with libs by default)
• [x] RLS to control the whole process without user interaction (or just an OK for the downloads or something)
• [x] heuristic for discovering the rustup save-analysis for std lib based on target

I don't think we necessarily need all of this to work for the alpha release, but we it should be easier than it is now (we can't require users to build their own compiler and libs) and we should have a good plan for the long term solution.

[matklad]

we should have a good plan for the long term solution.

Why you can't just use an .rlib file? If it contains enough information for compiling code, it must be also enough for rls. IntelliJ IDEs use the same abstraction (PSI) for working with both .java and compiled .class files. So you can just add a compiled binary library to your project and get completion and other features. You can also attach a source code to the binary library so that "go to definition" works properly.

[nrc]

The metadata in the rlib is technically what we need but it is hard to read (we'd basically need to link the compiler to decode it) and is an implementation detail and thus can change at any moment. The save-analysis metadata is basically the same information but with a layer of abstraction, thus easy to read and fundamentally stable.

[matklad]

we'd basically need to link the compiler to decode it

Hm, you already are using an in-process rustc?

and is an implementation detail and thus can change at any moment.

If it is more or less stable in practice, then perhaps it is possible to write a translator from rlib to save-analysis format. I don't know the amount of work here, but it will be the most convenient form for the users, because they won't need to download/install additional stuff. Custom build stdlibs would also work, which is a nice little benefit, as well as an ability to work with hypothetical closed source binary only crates.

[nrc]

Hm, you already are using an in-process rustc?

Only when we don't have the save-analysis data already

If it is more or less stable in practice

It is not

then perhaps it is possible to write a translator from rlib to save-analysis format. I don't know the amount of work here, but it will be the most convenient form for the users, because they won't need to download/install additional stuff

I don't think there is much more convenience for users - they have to wait for a translation step vs waiting for a download - either is just a wait, everything else should be automatic. Long term, the save-analysis data should be distrbuted with the libs, so there would not even be an extra download.

Custom build stdlibs would also work

If custom building then you can just output save-analysis, no issue there

[nrc]

planning:

• remaining for alpha: change the search path for std save-analysis data, finish instructions in README.md
• for alpha2: rustup integration
• for beta: total automation

[nrc]

Changing milestone since we've now done everything intended for the alpha

[nrc]

Working on rustup distribution of data. Hopefully, we don't actually need to modify rustup at all, just provide the right manifests. Unfortunately, we need to do that for the current (Buildbot) and future (Travis) systems.

https://github.com/rust-lang/rust-buildbot/pull/150 should address the current system.

When https://github.com/rust-lang/rust/pull/39284 lands, we'll need to add a line for the analysis data.

[Arzte]

@nrc Both of the mentioned pull requests have merged.

[nrc]

Changes for the new system: https://github.com/rust-lang/rust/pull/40026

[nrc]

Rustup now downloads the data and rls-analysis manages it. We are done for alpha 2 (modulo any bugs discovered).

Remaining:

• make the RLS handle fetching the data, rather than needing the user to interact with rustup.
• cross-compilation - we currently require the host and target triples to match exactly. To address this we need to pass the target triple from the rls to rls-analysis. I think the best way to get the target triple is by parsing the calls from cargo to rustc. We might need to add API to Cargo or rustc though.

[jntrnr]

Rustup can now fetch and install the rls, rust-analysis, and rust-src. Good enough for beta.