Pylance-release: Go to symbol in Workspace very slow and lists non-workspace objects

[robintw]

Environment data

• Language Server version: 2020.6.1
• OS and version: OS X 10.13.6
• Python version (& distribution if applicable, e.g. Anaconda): Anaconda Python 3.7

Expected behaviour

Running Go To Symbol In Workspace (Cmd-T) will allow me to search quickly for a symbol in my workspace.

Actual behaviour

When I press Cmd-T and start typing, I get a progress bar/activity indicator along the bottom of the text entry field. It sits there processing for quite a while, and then eventually gives me a list of symbols. This is _significantly_ slower than when using the standard MS Python Language Server. The first time I run it after starting VS Code it takes multiple minutes to run, subsequent times it takes a few seconds: the standard MS Python Language Server is almost instant.

When the list of symbols is displayed, it shows me all sorts of strange symbols from code inside the .vscode folder, and code in some of the Python packages installed in my conda environment. With the standard MS Python Language server it only showed me symbols from within my workspace.

Logs

I've put the log output in a Gist here as it is very long.

It seems like it is searching all across my entire Python environment, rather than just in my workspace...that would explain why I'm getting so many extra results, and it is taking so long.

[jakebailey]

Symbol searching in Pylance is based on the analysis, but the symbol searching in MPLSv1 was a separate index in parallel that didn't involve the analysis (but would get things "wrong" as it has no context).

So there are a few things (@robintw if you could answer the questions in the notes below, I'd appreciate it):

• It may be slower because it may have to open files to extract symbols, but it shouldn't need to do a full analysis (just "binding" names to the symbol table). Do you have a repo somewhere we can test that shows this behavior?
• It shouldn't show things in your conda environment, it should be limiting to the workspace, so that appears to be a bug.
• As for .vscode, does that folder contain python code, or just JSON? If I recall correctly, VS Code's symbol search is done for all languages at once, so it may be searching symbols provided by say, the JSON language server.

[heejaechang]

should "go to symbol" index libraries as well?

[ThiefMaster]

should "go to symbol" index libraries as well?

Probably something for a separate issue. but this is the ONE PyCharm feature I'm missing a lot: Being able to quickly jump to library symbols! By default I just want my application symbols, but I'd like to have a way to browse library symbols as well, e.g. if I want to view the code of some library function which I know exists, but which is not currently being imported in my code.

[jakebailey]

The LSP spec has workspace/symbol which traditionally only searches within the workspace/project: https://microsoft.github.io/language-server-protocol/specification#workspace_symbol

Searching libraries might need an LSP change to make it more clear that this is done (or we optionally do it anyway with some toggle).

[ThiefMaster]

Without being able to toggle it (ideally not a preference but something lightweight that can be easily done while searching - in PyCharm you hit the shortcut to search symbols again for example) it would be quite annoying and possibly very spammy.

[ThiefMaster]

Since this is unrelated to this bug report I opened a new issue with my suggestion to add this as a proper feature: #35
I guess the comments about this here could be collapsed (and maybe the one about the LSP protocol added to the new issue?) :)

[robintw]

Thanks for the prompt responses on this!

In response to @jakebailey's questions:

Note: All of the results below were obtained from a newly-installed copy of VS Code Insiders, with the Python extension and Pylance installed.

Cloning https://github.com/robintw/Py6S and adding it as a workspace folder, opening a .py file (to load the extension) and then pressing Cmd-T and searching for extract takes around 12 seconds with nothing displayed, and then displays the following:

This shows a few extract... methods from my workspace, but before them it shows a load of methods from the site-packages folder of my Anaconda environment, and at the bottom some from the ~\.vscode-insiders folder where there are obviously various bits of Python code located.

I hope that answers most of your questions - let me know if you need any more information.

[jakebailey]

Ah, I see what you mean by .vscode folders, the actual extension folders (versus say .vscode/settings.json in a workspace).

We're returning results from our copy of the typeshed stubs, which isn't intended and likely the same issue as conda. These should be getting filtered to workspace only. Ensuring we only search the workspace would likely also help the performance.

[lucaluca]

I am having the same problem, but I don't use conda. The first search I make into Quick Open binds and parses not only every file in my workspace, but every file in my site-packages folder and what appears to be all the source code for Pylance, and includes these in the results. Relative to my workspace root, the site-packages folder is at ../lib/site-packages so it shouldn't be traversing up the tree like that. The search takes about 5 minutes on the first search and 10-20 seconds on every search afterwards. My log output looks like what @robintw provided. My only changes to the default Pylance configuration are to uncheck "Auto search paths" and add an extra path where my modules are located within my workspace.

[jakebailey]

This feature is being worked on, and we will be limiting workspace symbols to only things in the workspace (that aren't also say, in a venv) pending an overhaul of how symbols work.

[vkramanuj]

Quick fix while they're working this out: I found that disabling "Python Analysis: Use Library Code for Types" greatly speeds up search and no longer includes site-packages functions.

[lucaluca]

@vkramanuj This worked for me! Now it only includes the pylance code but not everything else in site-packages. Thank you!

[jakebailey]

Note that doing this means we are no longer scanning library code for info, and any library that doesn't have stubs will no longer work. This might work fine for some libraries, but I would expect it to not work so well for many others.

[lucaluca]

Understood. I think getting it to work better on my own code outweighs the loss of functionality in other libraries for me, but can see that it wouldn't work for others.

[jakebailey]

This should be fixed in the next release, which pulls in a fix to only search / return results from user code.

[jakebailey]

This issue has been fixed in version 2020.8.2, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/master/CHANGELOG.md#202082-20-august-2020