Lsp: [RFC] Automatic updates/installation of language servers

[jwortmann]

I just stumbled over this conversation in the Package Control repo, and I wonder if it would make sense to have a built-in update mechanism in the base LSP package for the servers. There are a few LSP-* packages, but they have to be maintained manually and according to this comment the currently used strategy for updates seems far from optimal. From a quick look, most of the LSP-* packages contain pretty much the same plugin code and just apply the commands/settings that already are described in the documentation. I think all the configuration settings specified in the LSP-* packages could likewise easily go under the default_clients settings shipped with LSP - if not already present there. If I'm missing something here, please feel free to correct me @predragnikolic. I think previous installation problems were mostly caused by unclear documentation or because some servers like for Julia or PowerShell require a bit more extensive settings / run command to start.

I personally would welcome an update mechanism for the servers I use, but I don't want to create and maintain additional LSP-* packages for them. Since most servers are installed via npm, or other runtime environments with a package manager like Python/pip, it is only a single command to run an update for the server. The same basically applies to the installation as well. And the installation/update commands could be introduced as new setting keys then, e.g.:

{
    "clients": {
        "vscode-html": {
            "command": ["html-languageserver", "--stdio"],
            "install": ["npm", "install", "-g", "vscode-html-languageserver-bin"],
            "update": ["npm", "update", "-g", "vscode-html-languageserver-bin"],
            "languageId": "html",
            "scopes": ["text.html.basic"],
            "syntaxes": ["Packages/HTML/HTML.sublime-syntax"]
        },
        "pyls": {
            "command": ["pyls"],
            "install": ["pip", "install", "python-language-server"],
            "update": ["pip", "install", "--upgrade", "python-language-server"],
            "languageId": "python",
            "scopes": ["source.python"],
            "syntaxes": ["Packages/Python/Python.sublime-syntax"]
        }
    }
}

LSP could then provide additional commands "LSP: Install language server", "LSP: Update language server" or "LSP: Update all language servers" that could be run from the command palette. This way users could provide own commands for servers that have no LSP-* package yet (most servers actually). I'm not sure currently if or how this would work best for servers provided as binaries or installed from a specific REPL like Julia, but I think this can't be solved with LSP-* packages either. (An option for updating the Julia language server could be to create a small Julia script file with the update instructions and then run that file "update": ["julia", "path/to/update_server.jl"])
Edit: The Julia server could be updated with "update": ["julia", "-e", "'import Pkg; Pkg.update(\"LanguageServer\"); Pkg.update(\"SymbolServer\")'"].

Automatic updates could perhaps be implemented via another general settings key "auto_upgrade_servers": true/false and then run the update command every 24 hours or so and save the time of the last update check.

Is this something worth to think about / implement?

[ayoub-benali]

I like the idea. More generally I think servers should be part of this package as well since
most of the LSP-* packages only takes care of the server installation and add some custom features.

Also for metal LSP package, updates are handled via a settings value here, which is delegated to the artifact fetcher (coursier).

[jfcherng]

I wonder how many time (if in serial) / resources (if in parallel) it will take to check if there is a update or not, if I have many LSP-* packages installed because I consider npm on Windows is slow...

[simonklb]

This feels like feature creep to me and a bad idea in general. People and OSes have different ways of managing packages on the system, let's keep it that way.

If you choose to implement this please make it opt-in as I want to control how and when packages are installed and updated on my system and don't want any nasty surprises in the future.

[predragnikolic]

@simonklb I don't see it as a bad idea.

it will be opt-in. No one will force you to install a Language handler.

You can always add language servers by adding them in the LSP-settings client config option. And you will be in total control when you want to update the server or not.

[jwortmann]

Of course it would be optional and automatic updates should be disabled by default. Perhaps we should at first implement only manual commands for installation and updates. I guess that shouldn't be too hard to implement, but I would like to gather some more opinions or pros/cons about this first.

I would see it as a one-step simplification for server updates from within Sublime Text. Currently, what I have to do with a few servers installed if I want to keep them up to date is to remember or look up the updating commands of npm, pip, etc. and the exact server names and then manually call them from the command line. With a single command "LSP: update all servers" the LSP package could go through all servers that are currently enabled in the settings file and run the specified commands for these servers in the background for me. Okay, I probably could also put the update commands into a batch-script and run that script once a while, but this isn't really user-friendly to maintain when I enable/diable or add new servers and it isn't cross-platform.

Some implementation questions would be if it is possible to update when the server is currently running, user feedback for success/errors and how to handle installation and updates for servers which are only provided as a binary file for download. A drawback could indeed be bad performance for npm etc.

[simonklb]

The way I view this plugin is that it should only act as the component that turns Sublime into a LSP enabled editor, that's it.

I wasn't perhaps clear on the feature creep part. As soon as you start supporting automatic installations and updates of the server applications users are going to start rely on this feature of the plugin. This means increased burden on maintainers, issues that might not be tied to the plugin directly and so forth. You already see this when language servers are buggy and people think it's this plugin that is working poorly.

However, I do see the point that this might be beneficial for new users that just want a plug and play behavior.

I'll leave this up to the maintainers to decide. Just wanted to weigh in with my 2 cents on the matter.

[rchl]

In my opinion, the base LSP package should not be concerned about languageserver-specific options and configuration and that's why I like having separate LSP-* packages for individual servers that can be delegated to people that care and use those servers and can keep them up to date.

The part that is bad about (some) of those LSP-* packages is the fact that they ship in unpacked form and install often very big dependencies in the form of node_modules.

The good thing about LSP-* packages are:

• Mentioned as a con above, but the good side of having a local copy of the server is that it's controlled by the package what version of server is installed and when it's updated. There is less chance of things randomly breaking as opposed to having globally installed dependency where updating it could break compatibility with its settings.
  For example: eslint server had various compatibility-breaking changes that required updating language server options. If LSP-eslint controls dependency, the breaking change can be handled by updating LSP-eslint to a new version at the same time when updating eslint server dependency.
• Updates can be released independently of LSP and are handled automatically by Package Control.
• It works around the problem of npm/node not being in the system PATH and Sublime not being able to start the server. A common problem that affects many people, especially on Mac.

So I don't agree that this stuff should move to LSP and servers should be installed globally. And as said, I also don't think the current solution is ideal (due to massive node_modules directory), but I can't think of an ideal solution now.

Ideally, the servers would be installed in some directory that is considered a "temp" or "cache" directory in user's profile, and LSP-* packages would control and update those servers.

[FichteFoll]

Ideally, the servers would be installed in some directory that is considered a "temp" or "cache" directory in user's profile, and LSP-* packages would control and update those servers.

That's pretty much exactly what sublime.cache_path() provides. However, I believe we could do better as a community to document and suggest that solution because so far I hardly see anyone mention it outside of involved dev work on the Discord or when reviewing packages to be added to PC.

[rchl]

So then, taking LSP-eslint as an example which:

• ships with an eslint-server npm package (eslint-server needs to be included because it's not published to npm - it's part of vscode-eslint package normally.)
• requires npm install to be run inside eslint-server to install dependencies

We could probably copy eslint-server's package files using sublime.load_binary_resource into cache directory and run npm install there. Copying files using that API is probably the trickiest part - not sure there is API to list files and directories inside a package...

[FichteFoll]

You can use sublime_lib.ResourcePath.copytree for that. Unfortunately the docs are broken atm, but you can read the docstrings directly instead.

Edit: docs fixed btw

[AmjadHD]

I think the clients section of the settings should be removed and instead rely exclusively on language handlers, for the majority of server configurations and for the time being a minimal language handler for each is all it is required, later on these can be extended.
This makes LSP focused on being a framework more like vscode.

[rchl]

LSP-* packages are now shipping packaged and are installing servers in the cache directory so the only downside of them is gone.

I think we agree that base LSP package should not be concerned with server-specific configuration, updates or custom message handlers so I believe we can close this issue.