Rust-analyzer: Implement downloading of binary github releases from vscode extension

[Veetaha]

The task is to download the rust-analyzer language server from the latest stable (do we have unstable releases?) Github release and store it on the users filesystem.

The initial discussion on this task was started under this PR.

I'll copy it here to continue the discussion under this issue:

One super-important task would be to implement downloading of binary github releases. Basically, porting this things: https://github.com/fwcd/vscode-kotlin-ide/blob/master/src/serverDownloader.ts

We provide separate artifacts for each plarform: https://github.com/rust-analyzer/rust-analyzer/releases
We now only need to download the right one and store it into ExtensionContext.storagePath

Okay, got it. Say I am not a systems programming pro, but I suppose even within one OS there might be differences in binary interfaces of different processor architectures, or do you think having 3 binary artifacts would be enough to cover all users?

By default, rust targets a pretty old CPU, so this shouldn't be the problem. Like, there are folks with desktop linux on arm, and they'll have to compile from source, but for the overwhelming majority of users this should just work, modulo bugs

[Veetaha]

@matklad you mentioned ExtensionContext.storagePath. Is there a reason not ot use globalStoragePath instead? If you are not sure, let me investigate this then...

[matklad]

right, I think it should be globalStoragePath actually

On Sun, 2 Feb 2020 at 21:04, Veetaha [e-mail] wrote:

@matklad https://github.com/matklad you mentioned
ExtensionContext.storagePath. Is there a reason not ot use
globalStoragePath instead?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rust-analyzer/rust-analyzer/issues/2988?email_source=notifications&email_token=AANB3M6LTBZILMH5OE6Q6P3RA4RNFA5CNFSM4KO3XP52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKR7SZA#issuecomment-581171556,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANB3M7EBFUJRLK4CQESWBTRA4RNFANCNFSM4KO3XP5Q
.

[DJMcNab]

We should have a setting to use the latest 'stable' or your installed one, detecting whichever is newer (is there a GitHub API we could use for this?)

[matklad]

We should not implement auto-update in the first version of the feature.
Just an explicit command "download the latest realse" is all we need. We
do, however, need a three-state setting for ra-lsp-server binary:

• bundled
• cargo installed one
• explicit PATH

Though, perhaps they can be folded just to one setting, where null or "" is
1, ra_lsp_server is 2 and "/my/path/to/server" is 3

On Wed, 5 Feb 2020 at 11:38, Daniel McNab [e-mail] wrote:

We should have a setting to use the latest 'stable' or your installed one,
detecting whichever is newer (is there a GitHub API we could use for this?)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rust-analyzer/rust-analyzer/issues/2988?email_source=notifications&email_token=AANB3M2NWLTYRWJEWRUVIQ3RBKJMBA5CNFSM4KO3XP52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK26JXQ#issuecomment-582345950,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANB3M2YJEECDWDFZS45F63RBKJMBANCNFSM4KO3XP5Q
.

[Veetaha]

I wish we had something like rustup or nvm (node version manager) but for rust-analyzer. This way the task would be trivial...
Alas, I see the following way we implement this:

• Rust Analyzer: Download lastest language server version vscode command (i.e. something you can invoke from the command palette).
• "languageServerPath" extension configuration option that when absent defaults to ra_lsp_server from $PATH. However, I propose not to treat empty string specially (this is en error to specify empty string because it denotes an empty path).
• When the extension is loaded it checks ra_lsp_server binary availability at "languageServerPath" by invoking it with --version. If the binary is not found we issue a notification
  "Language server binary for rust-analyzer was not found. Do you want to download it now?" and it provides a handly Download now button that initiates the download.
• Not to be implemented in first PR, just future roadmap: each time the extension is activated and the internet connection is available we check for the latest extension version on GitHub and throw a pop-up info message for the user to notify if she can update it. Beware that we cannot just download the new ra-lsp-server binary, because the extension client code can also be updated in the new release. The user would need to update the extension from the VSCode marketplace anyway.

Since cargo xtask install already puts ra_lsp_serever to .cargo/bin which should be present in $PATH there is no need for three-state.

PS: @matklad Am I correct that we are targeting VSCode marketplace as a deploy destination?

[lnicola]

Not to be implemented in first PR, just future roadmap: each time the extension is activated and the internet connection is available we check for the latest extension version on GitHub and throw a pop-up info message for the user to notify if she can update it. Beware that we cannot just download the new ra-lsp-server binary, because the extension client code can also be updated in the new release. The user would need to update the extension from the VSCode marketplace anyway.

I think the C++ extension dowloads the server automatically and asks the user to reload the window, but I'm not sure how they synchronize it with the extension updates.

[matklad]

With the proposal, the default is effectively "use rust-analyzer from ~/.cargo/bin". I strongly feel that the default should be "use bundled rust-analyzer". Without any options, we should suggest downloading the thing even there is a ra_lsp_server in PATH.

Reasoning:

• this makes one-time upgrade to binary releases easy, no one will be stuck with old rust-analyzer
• this should reduce the number of issues users report due to using outdated rust-analyzer, as, by default, we control the version of the binary. That is, this prevents

-- rust-anlzyer is broken for X
-- which version do you use?
-- the latest
-- puzzled
-- oh, I've realised the extension was picking ra_lsp_server from ~/.local/bin, I've put it there a year ago and forgot, sorry for the noise.

[Veetaha]

@matklad
Just to clarify, bundled ra_lsp_server is the binary which our extension downloads and puts into globalStoragePath, isn't it?

this makes one-time upgrade to binary releases easy, no one will be stuck with old rust-analyzer

Upgrade to binary releases? Do we download .vsix artifact and reinstall the extension alongside downloading the binary from GitHub?

And so if I understood you right:

• The default for "languageServerPath" would be globalStoragePath
• "languageServerPath": "ra_lsp_server" means ra_lsp_server available from $PATH env var
• "languageServerPath": "/explicit/absolute/or/relative/path" means ra_lsp_server at that concrete path.

And can you please confirm the following:

Am I correct that we are targeting VSCode marketplace as a deploy destination?

Otherwise I am not sure that downloading .vsix artifact from GitHub releases would play well with this, not mentioning that we don't use semver for our releases, but this is a separate topic anyhow ...

[matklad]

Do we download .vsix artifact and reinstall the extension alongside downloading the binary from GitHub?

Nope, we just publish the extension to the marketplace.

And so if I understood you right:

Yes

Am I correct that we are targeting VSCode marketplace as a deploy destination?

Yes, we only need to download the server iteself, the extension will be published to the marketplace.

[Veetaha]

Okay, my main concern is to prevent having the version of .vsix and ra_lsp_server out of sync. For this we need to have information about TypeScript extension version to download the corresponding binary version from GitHub releases. Do we have any information about the current extension version available from its code, or does VSCode provide the API to reflectively query the version of the extension?

[matklad]

We should not worry about it at this point. We should always just download
the latest version.

On Thu, 6 Feb 2020 at 14:20, Veetaha [e-mail] wrote:

Okay, my main concern is to prevent having the version of .vsix and
ra_lsp_server out of sync. For this we need to have information about
TypeScript extension version to download the corresponding binary version
from GitHub releases. Do we have any information about the current
extension version available from its code, or does VSCode provide the API
to reflectively query the version of the extension?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rust-analyzer/rust-analyzer/issues/2988?email_source=notifications&email_token=AANB3M6BOOI4QIUSNAJG6MDRBQFA7A5CNFSM4KO3XP52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK7GDSY#issuecomment-582902219,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANB3M3DSQ7QDDHTSWNWVY3RBQFA7ANCNFSM4KO3XP5Q
.

[Veetaha]

Okay, so we just keep it out of scope of the first implementation (read first PR), because this will need to be added to the development todo list anyway, right ?

[matklad]

More generally, we explicitly don't care about backwards/forwards
compatibility at the moment, and only guarantee that the latest rustc,
rust-analyzer, VS Code and VS Code plugin work together.

Ensuring proper compatibility story is hard, we are not stable enough to
make investing time into this worthwhile.

On Thu, 6 Feb 2020 at 14:36, Veetaha [e-mail] wrote:

Okay, so we just keep it out of scope of the first implementation (read
first PR), because this would need to be added to the development todo list
anyway, right ?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rust-analyzer/rust-analyzer/issues/2988?email_source=notifications&email_token=AANB3M6UXCOCCWHQ4QD75T3RBQG4HA5CNFSM4KO3XP52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK7HUUA#issuecomment-582908496,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANB3MZFSOOAHNUHSQTMPILRBQG4HANCNFSM4KO3XP5Q
.

[Veetaha]

FYI: already working on this ...

[Veetaha]

See the MVP at #3053

[Veetaha]

3053 landed, the initial implementation asks the user to press the button to download the binary if it doesn't see the lsp server available in globalStroragePath. Closing this and opening a new one for adding "Download latest language server" command.

[SomeoneToIgnore]

How do I build and install the extension from source now?

After I run cargo xtask install on the fresh master, and open the project, no RA server is started: no pop-up about the loaded modules appears anymore, document formatting does not work either.
I get the pop-up with the proposal to download and install the binary, but that's what I don't want, I've already run cargo xtask install that should have installed the binary for me.

[Veetaha]

@SomeoneToIgnore
Can you please open command palette with Ctrl + Shift + P, write Toggle Developer Tools and press enter to open the Developer Tools, then open Console tab and send its contents here?
Also please give the contents of Help > About window in VSCode?

[SomeoneToIgnore]

Nothing suspicious in the console:

Also please give the contents of Help > About window in VSCode?

I use VSCodium on macOs, so it's VSCodium -> About VSCodium for me:

Version: 1.42.0
Commit: ae08d5460b5a45169385ff3fd44208f431992451
Date: 2020-02-07T11:29:49.095Z
Electron: 6.1.6
Chrome: 76.0.3809.146
Node.js: 12.4.0
V8: 7.6.303.31-electron.0
OS: Darwin x64 19.3.0

Before you ask: VSCodium is almost the same VSCode, build from the MS GitHub repo and does not include the propietary tracking code that MS bundles with normal VSCode.
The cargo xtask workflow was forking for me before for it.

[Veetaha]

Oh, sorry, I need better refresh the docs...
Now you just need to explicitly specify the path to ra_lsp_server in "rust-analyzer.raLspServerPath" setting.
cargo xtask install should add it to ~/.cargo/bin/ra_lsp_server that should be in your $PATH.
This means just adding "rust-analyzer.raLspServerPath": "ra_lsp_server" must help!

[SomeoneToIgnore]

Nothing suspicious in the console:

Oh, now I've noticed the undefined version for RA extension.

Thanks for the tip, I'll try that out.

[Veetaha]

No undefined is alright, since runing ${globalStoragePath}/ra_lsp_server-mac --version didn't output anything and that's why it proposed you to download the binary

[SomeoneToIgnore]

This means just adding "rust-analyzer.raLspServerPath": "ra_lsp_server" must help!

Great, adding "rust-analyzer.raLspServerPath": "ra_lsp_server", into settings.json did the trick, thank you.
Let's update the docs to avoid confusing other poor fellas :)

[Veetaha]

Already working on this, thanks for the input!

[Veetaha]

@SomeoneToIgnore can you please correct me if I am wrong. cargo install puts binaries to ~/.cargo/bin on mac too, right? I also need to figure out this path for Windows...

[lnicola]

Yes for Mac AFAIK. On Windows I think it's under %APPDATA%.

[SomeoneToIgnore]

This is the default installation dir, but that can be overridden with CARGO_HOME variable, so I don't think it's correct to rely on this in general.