Pylance-release: Breaks open symbol by name

[janosh]

Environment data

• Language Server version: v2020.7.2 (just installed today)
• OS and version: macOS 10.15.6
• Python version: 3.6.10

Opening symbols by name (cmd + T) stopped working in my workspace after installing Pylance. VS Code no longer finds anything, instead displaying the spinner below the search field indefinitely.

[jakebailey]

If you enable trace logging (see https://github.com/microsoft/pylance-release/blob/master/TROUBLESHOOTING.md#filing-an-issue), do you see that Pylance is parsing and binding files while the search is pending?

[janosh]

Yes and it eventually throws an error:

...
[FG] parsing: /usr/local/Caskroom/miniconda/base/envs/default/lib/python3.6/site-packages/pandas/_libs/join.cpython-36m-darwin.so (2494ms)
[FG] binding: /usr/local/Caskroom/miniconda/base/envs/default/lib/python3.6/site-packages/pandas/_libs/join.cpython-36m-darwin.so (0ms)
(node:1539) UnhandledPromiseRejectionWarning: Error: request cancelled
    at Object.t.throwIfCancellationRequested (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:383748)
    at h._addSymbolInformationForScope (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:488594)
    at h.visitModule (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:488203)
    at h.visitNode (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:111574)
    at h.walk (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:108636)
    at h.findSymbols (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:488141)
    at Function.addSymbolsForDocument (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:490501)
    at t.SourceFile.addSymbolsForDocument (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:166489)
    at /Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:127661
    at Object.runWithCancellationToken (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:321544)
(node:1539) UnhandledPromiseRejectionWarning: Error: request cancelled
    at Object.t.throwIfCancellationRequested (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:383748)
    at h._addSymbolInformationForScope (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:488594)
    at h.visitModule (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:488203)
    at h.visitNode (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:111574)
    at h.walk (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:108636)
    at h.findSymbols (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:488141)
    at Function.addSymbolsForDocument (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:490501)
    at t.SourceFile.addSymbolsForDocument (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:166489)
    at /Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:127661
    at Object.runWithCancellationToken (/Users/janosh/.vscode/extensions/ms-python.vscode-pylance-2020.7.2/server/pyright.bundle.js:1:321544)
(node:1539) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 98)
(node:1539) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 98)

[erictraut]

Thanks for the logs. One thing that immediately jumps out at me is the fact that Pyright is attempting to parse and bind ".so" files. It obviously shouldn't be doing that. That's an easy fix, but it's not clear that's the real problem here.

Another potential problem is the UnhandledPromiseRejectionWarning. I would expect the vscode-languageserver library to be handling this, since it accepts the promise for the "find symbols" request, and it handles cancellation exceptions. Or am I making a bad assumption there, @jakebailey?

[jakebailey]

I think there's some oddity happening with _runEvaluatorWithCancellationToken / runWithCancellationToken where the exception is getting stopped rather than making it all the way back out to the LSP library. From my glance, it's doing the right thing by rethrowing, but there might be something we missed about how node handles cancellation exceptions when in an async/promise context.

[jakebailey]

But I'd agree that the perf problem is more likely to be impacted by the reading of non Python files than anything. That exception (from my experience) happens at the very end because VS Code has given up on receiving an answer, hence why it's a cancellation.

[heejaechang]

they might have wrong filespec that include *.so?

[erictraut]

I've introduced a fix in Pyright to not include native libraries in the list of tracked source files, so they will no longer be parsed.

[jakebailey]

I'm thinking this is fixed, but wasn't marked as such. Let us know if that's not the case.

34 is the general tracking issue for this workspace symbol system being slow (and that's being worked on).

[janosh]

@jakebailey Yes, working now. Thanks for the update.

[janosh]

@jakebailey While it's working, it's extremely slow. Even in a relatively small workspace, looking up a symbol takes several seconds.

[erictraut]

It's slow because it's looking in all imported libraries for the symbol. After additional discussion, we've decided this is not the correct behavior, so I've changed the code to look only in user files in the workspace. It's now _much_ faster (almost instantaneous for small workspaces). This change will be included in the next release of Pylance. We're also working on a lightweight indexer that will make this feature even faster, but this work isn't yet complete.

[janosh]

Ah, there was a misunderstanding then. I had thought the fix was already out.