Solargraph: Pegged at 100% CPU

[AndrewRayCode]

The solargraph server is pegged at 100% CPU always for me. I'm using the vscode-ruby plugin

I also regularly get these errors spat out from the "Ruby Language Server":

[Error - 6:19:10 PM] Request textDocument/definition failed.
  Message: [TypeError] no implicit conversion of nil into String
  Code: -32603 

This is on a very large codebase, so it's probably impossible for me to narrow this down to a specific case.

[blackst0ne]

My solagraph instance eats 100% CPU and a lot of RAM which makes it unusable.

[castwide]

I'm in the middle of two sets of changes that should have a significant impact on performance and memory.

The first set of changes involve the ApiMap and are currently in the master branch:

• It avoids remapping the workspace for edits that do not change the API surface.
• Updates require less YARD parsing during runtime.
• It caches more data, resulting in more simple key-value lookups vs. intensive queries.

The second set of changes involve the YardMap. Solargraph uses YARD docs to provide data from the Ruby core, the stdlib, and any gems required by the codebase. I've known for a while that YARD queries were a good candidate for optimization, but when I profiled them recently, they turned out to be even less efficient than I expected. I've figured out a few ways to make them faster, most notably by storing the pins generated by the YardMap directly in the ApiMap store. Testing against the rails/rails repo, completion items are noticeably faster and the server process uses nearly 70% less memory.

I expect to publish these updates in gem version 0.25.0.

@AndrewRayCode Do you get those errors after running Go to Definition? I can't think of any other reason the client would send a textDocument/definition request. Are there any more detailed error messages in the developer console?

[castwide]

All the ApiMap and YardMap enhancements are pushed to master.

[blackst0ne]

@castwide what's the ETA of the next release with these improvements? :slightly_smiling_face:

[castwide]

@blackst0ne Most likely tomorrow.

[castwide]

Gem version 0.25.0 is released. I tested it on 2 machines: a desktop with 32GB RAM and 8x4GHz CPU, and a laptop with 4 GB RAM and 2x2.6 GHz CPU. Both showed notable improvements in performance and memory usage.

Opening castwide/solargraph in VS Code on the desktop, the language server took about 10 seconds to initialize. After that, most operations were noticeably faster. Completion requests were instantaneous. So was Go to Definition, for both workspace symbols (e.g., Solargraph::Source.load_string) and symbols from external gems (e.g., YARD::Docstring.parser). The server hit about 20% CPU during initialization and stayed under 4% during active editing (mostly under 1% with occasional spikes in the 3-4% range). Memory usage hovered around 240MB.

On the laptop, initialization took about 25 seconds and stayed between 30-60% CPU usage. Afterwards, CPU stayed under 5% and memory was around 240MB.

For comparison, running castwide/solargraph with gem version 0.24 used up to 8 times as much CPU during editing (10%-15% on the desktop and up to 40% on the laptop) and about 430MB RAM.

I also tried running rails/rails on the desktop. Initializing the language server took about 45 seconds. The first completion request lagged for about 10 seconds, but after that they were close to instantaneous (probably < 250ms). CPU usage hovered around 5-10% during active editing and memory usage was around 520MB, while gem 0.24 clocked at 10-15% CPU and over 2GB RAM.

Some relevant statistics about each workspace:

• castwide/solargraph
  
  
  
  • 125 mapped files
  
  
  • 8,420 mapped lines of code
  
  
  • 30,743 mapped pins (including core, stdlib, and gems)
  
  
• rails/rails
  
  
  
  • 1,122 mapped files
  
  
  • 133,320 mapped lines of code
  
  
  • 66,160 mapped pins
  
  

A quick explanation of the changes affecting performance:

• The YardMap contributes pins directly to the ApiMap instead of requiring repeated YARD queries.
• Sources avoid remapping the workspace for document edits that do not change the API surface.
• The ApiMap caches more queries.
• Pins are capable of reusing unchanged docstrings across edits without reparsing them.

[castwide]

Gem version 0.27.0 includes more improvements that should reduce CPU usage. Mapping large workspaces can still take a while, but reindexing after the initial map is much faster. My tests with rails/rails yielded good results for completion requests.

[skyeagle]

I have installed version 0.28.2 and still experiencing the same issue from time to time. How could I help to resolve it?

[castwide]

@skyeagle One thing you could try is customizing which files are included in maps with a .solargraph.yml file. The default settings look like this (run solargraph config to generate a default file):

include:
- "**/*.rb"
exclude:
- spec/**/*
- test/**/*

That configuration includes all .rb files except the spec and test folders. There might be other folders that are safe to exclude like bundle/**/* or vendor/**/*. You could also go the other way and limit included files to, e.g., lib/**/*.rb and/or app/**/*.rb. (Note that if you open an excluded file in an editor, it'll still get mapped as long as it's open.)