Pylance-release: memory and cpu leak when version 2020.9.0 installed

[byehack]

ms-python version: 2020.8.106424

[jakebailey]

I'm not sure which issue template you used, but the bug template asks for logs and a bit more info, otherwise we don't have anything to go off of here. https://github.com/microsoft/pylance-release/blob/master/TROUBLESHOOTING.md#filing-an-issue

[uraniun]

I've experienced the same issue, but the main reason why this happened for me was Local History extension. It stores diff of file changes as separate files. During indexing on startup JS was out of memory, possibly because of the large number of files.

My env:

• Language Server version: v2020.9.0
• OS and version: Ubuntu 20.04
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.8.2

Language Server log snippet:

[Info  - 11:33:12 AM] Pylance language server 2020.9.0 (pyright 60eb2cca) starting
[Info  - 11:33:12 AM] Server root directory: /home/yurii/.vscode/extensions/ms-python.vscode-pylance-2020.9.0/dist
[Error - 11:33:12 AM] stubPath typings is not a valid directory.
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
[Info  - 11:35:06 AM] Connection to server got closed. Server will restart.
[Error - 11:35:06 AM] Request textDocument/codeAction failed.
Error: Connection got disposed.
    at Object.dispose (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:54:83344)
    at Object.dispose (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:300914)
    at _.handleConnectionClosed (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:301126)
    at _.handleConnectionClosed (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:54:34852)
    at t (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:299235)
    at invoke (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:461163)
    at o.fire (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:461924)
    at Q (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:54:72909)
    at invoke (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:461163)
    at o.fire (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:461924)
    at l.fireClose (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:54:63386)
    at ChildProcess.<anonymous> (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:203016)
    at ChildProcess.emit (events.js:208:15)
    at ChildProcess.EventEmitter.emit (domain.js:476:20)
    at maybeClose (internal/child_process.js:1021:16)
    at Socket.<anonymous> (internal/child_process.js:430:11)
    at Socket.emit (events.js:203:13)
    at Socket.EventEmitter.emit (domain.js:476:20)
    at Pipe.<anonymous> (net.js:588:12)
[Error - 11:35:06 AM] Request textDocument/codeAction failed.
Error: Connection got disposed.
    at Object.dispose (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:54:83344)
    at Object.dispose (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:300914)
    at _.handleConnectionClosed (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:301126)
    at _.handleConnectionClosed (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:54:34852)
    at t (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:299235)
    at invoke (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:461163)
    at o.fire (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:461924)
    at Q (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:54:72909)
    at invoke (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:461163)
    at o.fire (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:461924)
    at l.fireClose (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:54:63386)
    at ChildProcess.<anonymous> (/home/yurii/.vscode/extensions/ms-python.python-2020.8.106424/out/client/extension.js:1:203016)
    at ChildProcess.emit (events.js:208:15)
    at ChildProcess.EventEmitter.emit (domain.js:476:20)
    at maybeClose (internal/child_process.js:1021:16)
    at Socket.<anonymous> (internal/child_process.js:430:11)
    at Socket.emit (events.js:203:13)
    at Socket.EventEmitter.emit (domain.js:476:20)
    at Pipe.<anonymous> (net.js:588:12)
[Info  - 11:35:07 AM] Pylance language server 2020.9.0 (pyright 60eb2cca) starting
[Info  - 11:35:07 AM] Server root directory: /home/yurii/.vscode/extensions/ms-python.vscode-pylance-2020.9.0/dist
[Error - 11:35:07 AM] stubPath typings is not a valid directory.
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
[Info  - 11:36:59 AM] Connection to server got closed. Server will restart.

[jakebailey]

The most recent update added an indexer, which would scan all of the files in your workspace to find symbols. It's likely your workspace is so big that we run out of memory; the indexer does consume a bit until it "finishes" and then drops the unnecessary data.

Be aware that a leak would imply we would consume this memory for all time and never release, even over a very long period; that's not what's intended, but if the original poster meant a general increase in memory at startup (or this OOM scenario with many files), that makes sense.

I'm not sure why the version install menu doesn't work; other users have been able to do it.

[jakebailey]

That extension stores files in a specific folder, perhaps we need to be providing a way to explicitly ignore a set of files/folders, in addition to the overall "improve the performance" goal.

[judej]

@uraniun, a possible workaround for now would be to store the history outside the workspace. The docs for the Local History indicates that you can do this. Would you please try this and let us know if this fixes the issue?

[uraniun]

@judej , yes, moving .history folder out of the workspace fixes indexing issue for me. How about excluding .vscode folder from indexing? I've stored a .history folder there before

[byehack]

[Info  - ۲۲:۳۵:۳۹] Pylance language server 2020.9.0 (pyright 60eb2cca) starting
[Info  - ۲۲:۳۵:۴۷] Server root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(2) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(2) started
[Info  - ۲۲:۳۵:۴۷] Background analysis(3) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(3) started
[Info  - ۲۲:۳۵:۴۷] Background analysis(4) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(4) started
[Info  - ۲۲:۳۵:۴۷] Background analysis(1) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(1) started
[Info  - ۲۲:۳۵:۴۷] Background analysis(7) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(7) started
[Info  - ۲۲:۳۵:۴۷] Background analysis(8) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(6) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(8) started
[Info  - ۲۲:۳۵:۴۷] Background analysis(6) started
[Info  - ۲۲:۳۵:۴۷] Background analysis(5) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(5) started
[Info  - ۲۲:۳۵:۴۷] Background analysis(9) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(9) started
[Info  - ۲۲:۳۵:۴۷] Background analysis(10) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۴۷] Background analysis(10) started
[Info  - ۲۲:۳۵:۵۲] Background analysis(11) root directory: c:\Users\Tester\.vscode\extensions\ms-python.vscode-pylance-2020.9.0\dist
[Info  - ۲۲:۳۵:۵۲] Background analysis(11) started
[Error - ۲۲:۳۵:۵۲] stubPath typings is not a valid directory.
[Error - ۲۲:۳۵:۵۲] stubPath typings is not a valid directory.
[Error - ۲۲:۳۶:۰۰] stubPath typings is not a valid directory.
[Error - ۲۲:۳۶:۰۱] stubPath typings is not a valid directory.
[Error - ۲۲:۳۶:۰۲] stubPath typings is not a valid directory.
[Error - ۲۲:۳۶:۰۲] stubPath typings is not a valid directory.
[Error - ۲۲:۳۶:۰۴] stubPath typings is not a valid directory.
[Error - ۲۲:۳۶:۰۴] stubPath typings is not a valid directory.
[Error - ۲۲:۳۶:۰۵] stubPath typings is not a valid directory.
[Error - ۲۲:۳۶:۰۵] stubPath typings is not a valid directory.
[Error - ۲۲:۳۶:۰۵] stubPath typings is not a valid directory.

[byehack]

current working solution for me is downgrade pylance to 2020.8.3

[pedrobaeza]

I'm having the same memory consumption, with 12 GB consumed until my 16 GB memory is exhausted. If this is the symbol construction, it should be definitively optimized. Not sure how can I provide useful data (I don't think logs serve here) for debugging this.

[dmulcahey]

I'm also experiencing this as well. LMK what I can grab for logs to help out. Paralyzes both a core i5 and a core i9 16" MBP and makes VSCode practically unusable.

[kielnino]

Same here. As soon as I start VSCode, all four cores are fully utilized and the 8 GB memory is filled until the system freezes.

[pvojnisek]

current working solution for me is downgrade pylance to 2020.8.3

The same for me. Reinstalling the previous version worked well. First I had to figure it out how to make it. Here is some help:

1. Uninstall Pylance from vs code

2. Download version: 2020.8.3

3. Rename the extension of the downloaded file from VSIXPackage to VSIX

4. In vs code Ctrl+Shift+P: Install from VSIX

5. Select the downloaded file and install it.

6. Enjoy..

[vmasrani]

You should be able to downgrade using the "Manage" button on the pylance extension as well:

[thieman]

Confirming this issue in a Python repository with ~1M lines of code. Seeing the same behavior as above:

As soon as I start VSCode, all four cores are fully utilized and the 8 GB memory is filled until the system freezes.

Needed to downgrade.

[lig]

I can confirm enormous CPU consumption with 2020.9.0 on a relatively big workspace. Full 8 cores for several minutes after the vs code start. This basically results in a complete local DOS. Waiting for it to finish indexing works but it looks like it would be more satisfying to start indexing on-demand on a folder by folder basis after a file is opened in the folder.

[lig]

And this happens on each start. At least, the data could be cached, I guess.

[AllanDaemon]

Same here. Going to downgrade

[huguesv]

This issue has been fixed in version 2020.9.4, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/master/CHANGELOG.md#202094-10-september-2020