Vscode-cpptools: cpp-tools extension consumes 6GB RAM

[viktorku]

I am observing very high memory usages from the cpp-tools extension. Judging from this htop overview, it keeps spawning new processes and even if I close VS Code not all processes are killed (had to kill them all manually after which 6GB memory was released).

[hiding threads for brevity]

I have Ubuntu 14.04 with:

Version 1.17.1
Commit 1e9d36539b0ae51ac09b9d4673ebea4e447e5353
Date 2017-10-10T14:21:18.489Z
Shell 1.7.7
Renderer 58.0.3029.110
Node 7.9.0
Architecture x64

Extensions:

PeterJausovec.vscode-docker
austin.code-gnu-global
bbenoist.Doxygen
chimon2000.polymer-snippets
dbaeumer.vscode-eslint
devCAT.lua-debug
donjayamanne.python
esbenp.prettier-vscode
hars.CppSnippets
joaompinto.asciidoctor-vscode
joelday.docthis
keyring.Lua
marvinhagemeister.theme-afterglow-remastered
matthewferreira.cppcheck
mitaki28.vscode-clang
ms-vscode.azure-account
ms-vscode.cpptools
ms-vscode.sublime-keybindings
ms-vscode.Theme-TomorrowKit
msjsdiag.debugger-for-chrome
polymer.polymer-ide
trixnz.vscode-lua
twxs.cmake
vector-of-bool.cmake-tools
xaver.clang-format
zhuangtongfa.Material-theme

This started happening definitely after I upgraded to 1.17. Previously, I didn't use the new 'Default' engine for IntelliSense as it was slow and not always correct. Now I can see it parsing for a longer time than before...

[bobbrow]

We are releasing an update this week that has improved the process management in linux/mac. I will ask you to try again after this update and let us know if you continue to see this issue.

[bobbrow]

0.14.0 is available for installation. If you continue you see abandoned processes, please let me know. I am aware of one remaining issue that I haven't solved yet, but the extension should be much better about cleaning up the IntelliSense child processes now and I haven't been seen a case in recent days where closing VS Code doesn't also clean up the extension and its dependencies (previously it was very common to see this).

[viktorku]

Thanks @bobbrow. I will test it these days and let you know 👍

[viktorku]

The issues seem to be fixed.

[viktorku]

@bobbrow This happened again:

I noticed vscode became very laggy, then closed it, but saw all these processes/threads dangling. Had to kill them manually. There was about 3-4 GB memory released.

_[EDIT 13/12/17]_ It keeps happening quite regularly, every once in a while it becomes laggy and consumes lots of memory, and if I close it there are several threads still alive. Even after killing the parent process some do get killed but some persist...

[xrEngine512]

I confirm this issue on Ubuntu 16.04.
I think for me it happens usually when parsing hangs for some reason (when there's always red flame in status bar).
If I close or reload window in that state, there's always dangling cpp extension's processes, that never exits.
I found out when I started using vscode on my laptop that I'm not powering down, only suspend, and at some point there was so many instances of cpptools, that intellisense refused to work, giving me errors like (Compiler exited with error - No IL available).
VSCode version: 1.23.1
CPPTools version: 0.17.4

[davydden]

I also see a big memory consumption from Microsoft.VSCode.CPP.IntelliSense.Msvc.darwin (6 processes, total about 4.5Gb. All opened projects using Cmake with
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON.

VSCode versions: 1.25.1
CPPTools: 1.17.7

[sean-mcmanus]

@davydden High memory usage for the IntelliSense process is "by design" currently. With certain translation units, the process could go over 2 GB (but most seem to use < 500 MB). We have an internal work item tracking reducing the memory usage issue. Setting the "intelliSenseEngine" to "Tag Parser" will disable the IntelliSense-based features if the memory usage is too big of a problem.

[davydden]

thanks @sean-mcmanus . I will give it a try. I presume one could try setting "C_Cpp.intelliSenseEngine": "Tag Parser", for those folders in the workspace which contain large C++ projects (i.e. I don't have to enable fuzzy search for the whole workspace)?

[sean-mcmanus]

@davydden Yeah, the setting should be settable per-folder in a multi-folder workspace.

[hungys]

+1

C/C++ extension consumed over 2GB of RAM for one of my largest code base, and the computer became very laggy when the issue happened. Actually I didn't aware of this issue few versions ago, is there any roadmap for the fix?

[sean-mcmanus]

@hungys We have always had a memory issue with the IntelliSense processes (I'm not aware of any memory regressions, unless you're hitting a case where previously the process would crash before using memory). It's an issue we're aware of -- the roadmap/timeline for investigating/fixing this hasn't been decided upon yet.

[hungys]

@sean-mcmanus After switching to Tag Parser, the peak memory usage of the extension process can be still up to 1.4GB. Is this "by design" or caused by memory leak?

[sean-mcmanus]

@hungys It sounds like it could be a bug, but I don't know of a repro.

[ruffianeo]

I see the same on my windows 10 x64 machine. 4GB and 2 rogue extension processes in my case. (I only have 4GB RAM and as such it bogs down my whole machine.. Also had to manually kill the c++ extension processes after closing VSCode. Also worth noting is that they seem to "oscillate" in their memory consumption. The whole thing was periodically moving up from 800MB to the 4GB and back without intervention. Maybe garbage collection issues?

Update: Here some more details on how my tiny project looks like:

I include some modern STL headers (optional, tuple), some long time existing STL files (like vector, string, functional etc). Then I include windows.h and gdiplus.h (which produces static analyzer warnings btw.). I link with user32.lib and gdiplus.lib and gdi32.lib. The problems started once I added the latter 2 libs and respective headers. I build on the command line and not within VSCode. Debugging works great, btw.

[hungys]

+1
Still having the same issue with "Tag Parser" mode.
The extension consumed up to 2.5GB RAM and cause my machine to hang...

[sean-mcmanus]

@ruffianeo You can set the intelliSenseMode to "Tag Parser" to stop launching the IntelliSense processes which use a lot of memory -- we're working on reducing the memory usage of that process. We don't use garbage collection, but we close IntelliSense processes after a memory limit is hit.

@hungys We need more repro info on how the Microsoft.VSCode.CPP.Extension process is using a lot of memory. I have never seen that before.

[hungys]

@sean-mcmanus It typically occurred during first scan of a large project. According to the parsing progress icon of the extension, our code base contains 20000+ files for parsing, and after letting it process for the whole weekend, now the memory usage is less than 100MB for this folder. But my point is that, the memory usage can be up to 2.xGB during parsing, is this by design? Have your team ever try to tune the performance on any large code base?

[sean-mcmanus]

@hungys Thanks a lot for providing more info -- I got a repro with chromium. I've filed bug https://github.com/Microsoft/vscode-cpptools/issues/2737 . The original issue was about the IntelliSense process using a lot of memory. Strange, seems like the bug doesn't repro with all projects.

UPDATE: Our next 0.21.0-insider build should have the fix.

[sean-mcmanus]

We fixed a memory leak with the main process when parsing a lot of files with 0.21.0-insiders (auto installed if "C_Cpp.updateChannel": "Insiders" is set, or manually installed from the offline vsix). Let us know if you find any problems with it.

We're currently working on fixing the large memory usage for the IntelliSense processes for our December release.

[sean-mcmanus]

UPDATE: Oops, sorry for the spam -- I forgot this was actually fixed with the earlier 0.21.0-insiders :).

The memory leak when opening a new, large workspace should be fixed now with our latest Insiders release: https://github.com/Microsoft/vscode-cpptools/releases . If you're using VS Code Insiders or have http.proxySupport not set to "off", you might not get the 0.21.0-insiders2 update automatically with C_Cpp.updateChannel set to "Insiders".

The large memory usage of the IntelliSense processes is still being worked on -- so far we've reduced the memory usage by about 75% for Windows, but we still have to port the Windows system calls to Linux/Mac (hopefully for the January release).

[senhalil]

Hello @sean-mcmanus, is this fix ported to Linux? I am having memory issues due to IntelliSense while using an external library (google/or-tools).

[sean-mcmanus]

@senhalil What is the memory problem? I can't tell from your screenshot.

[senhalil]

@sean-mcmanus sorry for not being clear. By issue I meant high memory usage but I came to think that maybe it is not that high. Still when intellisense is running on two projects, it is enough to freeze the cursor.

[sean-mcmanus]

@senhalil Yeah, it looks like your machine is using lots of memory and has lots of IntelliSense processes running. We have code that is supposed to close IntelliSense processes if memory usage goes above 1.5 GB. Do you know why so many IntelliSense processes are staying around? This could happen if you have multiple workspace folders open.

[sean-mcmanus]

@senhalil Does setting the C_Cpp.intelliSenseCacheSize to 0 fix the memory issue for you? Another user reported a memory issue from that https://github.com/Microsoft/vscode-cpptools/issues/3326.

[senhalil]

Do you know why so many IntelliSense processes are staying around? This could happen if you have multiple workspace folders open.
Yeah, I had two workspaces open however there are three IntelliSense processes for some reason. I am not sure why.

I tried using tag-parser but it didn't change the memory usage tremendously to be honest but it decreased the quality of the feedback.

The project had many errors due to a library version change. Not sure if this makes IntelliSense go crazy.

@senhalil Does setting the C_Cpp.intelliSenseCacheSize to 0 fix the memory issue for you? Another user reported a memory issue from that #3326.

Going to try this if I have memory issue again.

[jabaa]

I'm working on a system with 8 GB memory. The current project includes

Poco/1.9.4@pocoproject/stable
boost/1.69.0@conan/stable
fmt/5.3.0@bincrafters/stable

with Conan and builds with CMake. Intellisense reserves enough resources to freeze the whole system for over 30 minutes. After this time I can start to try to kill. It's still difficult to login into a new terminal because the login process often takes more than 60 seconds and is aborted.

[jabaa]

It seems that the new update 0.26.0-insiders3 fixed the problem. My machine doesn't freeze anymore and I can work with that project again.

[Colengms]

We think this was addressed (in 0.26.2). If anyone is still experiencing a similar problem, please open a new issue.