Lsp-mode: Pyright support

[zeronone]

Is it possible to add Pyright support? https://github.com/microsoft/pyright

[seagle0128]

Pyright provides some features that overlap with functionality provided by the standard VS Code Python extension: “hover”, type completion, definitions, references, rename symbols, etc. You may see duplicate results if Pyright is installed alongside the Python extension. You can disable the duplicate language service functionality in the Python extension by setting python.languageServer to None. If you want to disable language service features in Pyright, set pyright.disableLanguageServices to true.

I don't think it's a good idea to use both.

[zeronone]

For reference newly announced Pylance is built on top of Pyright: https://devblogs.microsoft.com/python/announcing-pylance-fast-feature-rich-language-support-for-python-in-visual-studio-code/

[danlamanna]

It seems like the appropriate course of action then is for some emacs package (maybe this one) to support pylance which would utilise pyright.

[danlamanna]

Related :( - https://github.com/microsoft/pylance-release/issues/4

[zeronone]

Pylance = Pyright + IntelliCode AI models (not open-source)

But Pyright is itself has a Language Server and open-source.

[seagle0128]

I am wondering what's the advantage of pyright, comparing to mspyls. mspyls is powered by C#, and has better performance I think.

[yyoncho]

... IntelliCode AI models (not open-source)`

@zeronone we can use this even if it is not open-source as long as the license allows that. Do you know more about that?

[yyoncho]

I moved the issue in lsp-mode repo since current repo provides support for the C# server and both doesn't seem to be related(correct me if I am wrong)

[ericdallo]

Any problem supporting both mspyls and this new one? And we keep the best as the default? I think this gives more options to LSP users

[zeronone]

@zeronone we can use this even if it is not open-source as long as the license allows that. Do you know more about that?

Sorry, I don't know much about it either.

Regarding pylance, I gives much better completions than mypyls thanks to type-inference. Although quite rough, I could make it work with lsp-mode.

# install pylance extension
code --install-extension ms-python.vscode-pylance

# add a pylance executable
cat  <<EOF > ~/bin/pylance.sh
#!/usr/bin/env bash
set -euo pipefail

node $HOME/.vscode/extensions/ms-python.vscode-pylance-2020.6.1/server/server.bundle.js --stdio
EOF

(defun +register|pylance ()
  (setq lsp-pylance-ms-executable "~/bin/pylance.sh")

  (lsp-register-client
   (make-lsp-client
    :new-connection (lsp-stdio-connection (lambda () lsp-pylance-ms-executable)
                                          (lambda () (f-exists? lsp-pylance-ms-executable)))
    :major-modes '(python-mode)
    :server-id 'mspylance
    :priority 3
    :initialized-fn (lambda (workspace)
                      (with-lsp-workspace workspace
                        (lsp--set-configuration (lsp-configuration-section "python")))))))

[jfcherng]

The LICENSE says

a) General. You may install and use any number of copies of the software only with Microsoft Visual Studio, Visual Studio for Mac, Visual Studio Code, Azure DevOps, Team Foundation Server, and successor Microsoft products and services (collectively, the “Visual Studio Products and Services”) to develop and test your applications.

So it's not legit to use it outside of MS products?

I don't think MS will sue (for personal use) but just wonder the legality.

[yyoncho]

@jfcherng just like vscode-cpp - they are saying "it is not easy to go open source" but at the same time the license forbids anything but vscode... (this is for pylance).

[seagle0128]

If pylance (not pyright) has better performance and functionalities, I vote to implement a new client. Of course, if the license is Okay.

[leungbk]

Any problem supporting both mspyls and this new one? And we keep the best as the default? I think this gives more options to LSP users

Long term, MS plan to deprecate mspyls.

Regarding pylance, I gives much better completions than mypyls thanks to type-inference. Although quite rough, I could make it work with lsp-mode.

# install pylance extension
code --install-extension ms-python.vscode-pylance

# add a pylance executable
cat  <<EOF > ~/bin/pylance.sh
#!/usr/bin/env bash
set -euo pipefail

node $HOME/.vscode/extensions/ms-python.vscode-pylance-2020.6.1/server/server.bundle.js --stdio
EOF

(defun +register|pylance ()
  (setq lsp-pylance-ms-executable "~/bin/pylance.sh")

  (lsp-register-client
   (make-lsp-client
    :new-connection (lsp-stdio-connection (lambda () lsp-pylance-ms-executable)
                                          (lambda () (f-exists? lsp-pylance-ms-executable)))
    :major-modes '(python-mode)
    :server-id 'mspylance
    :priority 3
    :initialized-fn (lambda (workspace)
                      (with-lsp-workspace workspace
                        (lsp--set-configuration (lsp-configuration-section "python")))))))

@zeronone I tried this with Pyright instead of Pylance and it worked well. There has been some interest in this already (https://github.com/microsoft/pyright/issues/99). It would be great if you could PR an executable upstream.

In the Emacs client, you would need to add notification handlers to ignore or simply log pyright/beginProgress, reportProgress, and endProgress. There might be some other things I'm not aware of.

[seagle0128]

I tried pyright and pyplance, both work well. I can implement the new client if I have time. But I don't know which one is the best option. Thoughts?

[yyoncho]

Pylance is probably better but it is restricted to vscode/ms products so probably pyright is the only option.

[seagle0128]

pyright is just a type checker. The vscode extension supports LSP, so can be leveraged. But I see this.

Q: What is the long-term plan for Pyright?
A: Pyright is now an officially-supported Microsoft type checker for Python. It will continue to be developed and maintained as an open-source project under its original MIT license terms. The Pyright extension for VSCode is a reference implementation and is not guaranteed to be fully functional or maintained long-term.

[danlamanna]

pyright is just a type checker. The vscode extension supports LSP, so can be leveraged.

No, it appears to include a language server implementation.

Q: What is the difference between Pyright and Pylance?
A: Pyright is an open-source Python type checker and language server. Pylance leverages Pyright’s functionality with additional features, some of which are not open-sourced.

[seagle0128]

@danielmartin Do you have how to run pyright command line as a language server?

[leungbk]

Do you have how to run pyright command line as a language server?

Slight modification to @zeronone's solution, without a separate shell executable

  (setq lsp-pyright-server-cmd '("node"
                                 "/home/brian/.vscode/extensions/ms-pyright.pyright-1.1.50/server/server.bundle.js"
                                 "--stdio"))

  (lsp-register-client
   (make-lsp-client
    :new-connection (lsp-stdio-connection
                     (lambda () lsp-pyright-server-cmd)
                     (lambda ()
                       (and (cl-second lsp-pyright-server-cmd)
                            (file-exists-p (cl-second lsp-pyright-server-cmd)))))
    :major-modes '(python-mode)
    :server-id 'mspyright
    :priority 3
    :initialized-fn (lambda (workspace)
                      (with-lsp-workspace workspace
                        (lsp--set-configuration (lsp-configuration-section "python"))))
    :notification-handlers (lsp-ht ("pyright/beginProgress" 'ignore)
                                   ("pyright/reportProgress" 'ignore)
                                   ("pyright/endProgress" 'ignore))))

It works pretty well.

[yyoncho]

@leungbk this looks good, also there is NPM package so

npm install -g pyright

would work.

[leungbk]

@zeronone It seems like your solution is pretty good as is. Whenever you get a chance, it would be great if you could submit it here as a PR.

[zeronone]

Here are the cons of each.

• pyright (installed from NPM)
  
  
  
  • Just provides a CLI for type-checker (the LSP part is not available in the CLI)
  
  
• pyright (installed from code --install-extension)
  
  
  
  • Provides the LSP server CLI command but add dependency on VSCode
  
  
  • Going to be deprecated.
  
  
• pylance (installed from code --install-extension)
  
  
  
  • Not open-source
  
  
  • Adds dependency on VSCode
  
  

In the snippet above, I directly used pylance VSCode extension executable, but I don't think it is good to add a dependency on VSCode for lsp-mode users. If it is okay, we probably need to add a new deps-provider here: https://github.com/emacs-lsp/lsp-mode/blob/master/lsp-mode.el#L7523 but I not versed in elisp or lsp-mode :(

[leungbk]

In the snippet above, I directly used pylance VSCode extension executable, but I don't think it is good to add a dependency on VSCode for lsp-mode users. If it is okay, we probably need to add a new deps-provider here: https://github.com/emacs-lsp/lsp-mode/blob/master/lsp-mode.el#L7523 but I not versed in elisp or lsp-mode :(

You don't actually need VS Code.

git clone https://github.com/microsoft/pyright
cd pyright
npm i
npm run build:serverProd

And then

(setq lsp-pyright-server-cmd `("node" "~/pyright/client/server/server.bundle.js" "--stdio"))

and the rest is the same as above.

Going to be deprecated.

There's no guarantee that it will be, but if it does get deprecated, there is nothing stopping someone from forking it.

In comparison, Microsoft's C# implementation of their Python server is 1) definitely getting deprecated 2) buggier (this can be verified by opening a large Python project in Linux) 3) lower code quality and 4) less featureful.

[seagle0128]

npm run build:serverProd

It's same as vscode extension, just building it locally.

The Pyright extension for VSCode is a reference implementation and is not guaranteed to be fully functional or maintained long-term.

The pyright extension is not full functional and will be deprecated in future. I am concerned about it although someone may fork it.

[leungbk]

It's same as vscode extension, just building it locally.

One of his two objections was that VS Code would become a dependency, which isn't the case.

The Pyright extension for VSCode is a reference implementation and is not guaranteed to be fully functional or maintained long-term.

The pyright extension is not full functional and will be deprecated in future. I am concerned about it although someone may fork it.

It's certainly a concern. But MS's C# implementation is at least as likely to become deprecated (https://devblogs.microsoft.com/python/announcing-pylance-fast-feature-rich-language-support-for-python-in-visual-studio-code/):

Our long-term plan is to transition our Microsoft Python Language Server users over to Pylance and eventually deprecate and remove the old language server as a supported option.

The short- and long-term prospects for Pyright seem better than those of the C# implementation. Even if it will never be as fully featured as Pylance, Pyright at present has over the MS C# Python server

• no memory leak on Linux (that I've noticed)
• multi-root workspaces
• call hierarchy
• cleaner code

Pyright also resolves a much larger percentage of its Issues (> 99% for Pyright vs < 80% for C# Py).

[zeronone]

Yes, we can build it locally but it is not that nice.

I created a PR so that pyright-langserver is linked upon installing pyright from NPM. https://github.com/microsoft/pyright/pull/810

[mjlbach]

Does some additional setting need to be passed to discover virtualenv modules? I'm only seeing completion for built-ins (like os, sys, etc are fine). Edit: I think python.pythonPath needs to be set as well (see: https://github.com/microsoft/pyright/issues/702#issuecomment-636248126)

[zeronone]

@mjlbach I am not 100% sure, but I guess for libraries you can either add type stub file or set the following maybe (from: https://github.com/microsoft/pyright/blob/master/docs/settings.md)

pyright.useLibraryCodeForTypes [boolean]: Determines whether pyright reads, parses and analyzes library code to extract type information in the absence of type stub files. This can add significant overhead and may result in poor-quality type information. The default value for this option is false.

[mjlbach]

@zeronone Thanks, I've tried adding that (for reference, I'm also working on adding support for pyright to nvim's native lsp) and now get some virtualenv completion.

Something else I noticed is that using the pyright-langserver cli option results in different completion options than compiling from source and directly calling server.bundle.js

[zeronone]

@mjlbach I found a small problem with the pyright-langserver CLI, I made a PR. Hopefully it fixes the issue.
https://github.com/microsoft/pyright/pull/835

[zeronone]

Created a PR: https://github.com/emacs-lsp/lsp-mode/pull/1921

[leungbk]

Now that https://github.com/emacs-lsp/lsp-pyright is up, it seems like this can be closed.