Rust-analyzer: RA uses quite a lot of RAM

[suhr]

Maybe 4GB is a very small amount of RAM in the modern world but nonetheless it would be nice to be able to use RA on such a machine.

[matklad]

Yeah, that is a known problem. We have implemented basic GC, so a steady-state RAM usage should not be that high. However, we have not yet added the limit for maximal memory usage, and during startup it can spike pretty high.

[daboross]

Do you know any workarounds / configuration to help this? Or is there any open issues in the project that I could contribute to to help reduce memory usage?

Asking because this is currently the main thing blocing my using rust-analyzer on a day-to-day basis. It provides a great experience, but I can't really afford the ~2.7gb ram per project. My main workflow usually involves opening 3+ editor sessions and using firefox for documentation, and 8gb ram can't handle 3 rust-analyzer instances and a browser.

[matklad]

@daboross good you've asked! Could you try https://github.com/rust-analyzer/rust-analyzer/pull/1382 ? It should cap the memory usage of RA a bit!

My main workflow usually involves opening 3+ editor sessions and using firefox for documentation, and 8gb ram can't handle 3 rust-analyzer instances and a browser.

Long-term, there should be only a single rust-analyzer instance for any number of projects, but that's far away at the moment.

[lnicola]

Long-term, there should be only a single rust-analyzer instance for any number of projects, but that's far away at the moment.

@daboross If you can use a Cargo workspace, you'll only get one RA instance.

[daboross]

@daboross good you've asked! Could you try #1382 ? It should cap the memory usage of RA a bit!

Thank you! Using #1382, I get ~0.9gb ram usage - a huge improvement!

@daboross If you can use a Cargo workspace, you'll only get one RA instance.

That's good to know! I guess I can reorganize things a bit, or maybe trick cargo into thinking the crates are in a workspace without committing to it in git.

[divinerapier]

Please help me. I use the latest version. Rust-analyzer uses more than 100GB memory.

[zacps]

I observed 9-10GB memory usage by rust-analyzer when running on WSL (vscode). This was no more than 5 minutes after opening the workspace and after no more than ~4 edits.

Commit: 778f50b1477ad3c33cd6220f71bac7b83a66176a

---

After restarting I observed it grow from around 180MB to 893MB while loading the workspace (340 packages).

I recorded this graph of the private bytes of ra_lsp_server covering about 30mins of the edit-check cycle.

This could be better, but clearly isn't the behaviour I observed in the first case.

If I manage to reproduce this again is there any particular information that would help debugging?

[lnicola]

@zacps Try updating, see https://github.com/rust-analyzer/rust-analyzer/issues/2052.

[zacps]

After update it grew to 1GB loading the workspace. Then 2.4GB after 15-20min of editing.

I haven't yet reproduced the 10GB case above.

Commit: c02f1165ca4099ea6c3706a670513f7904630615

[Robbepop]

On this workspace rust-analyzer memory consumption grows from approx. 2GB to approx. 8GB after about an hour of intense usage. It is a rather medium sized workspace. I can see similar behaviour as @zacps in that memory consumption continues to grow. It seems like the memory leak is caused mainly by editing the code.

Editing code in the linked workspace generally causes rust-analyzer to perform quite a bit of computations for up to 40 seconds with many memory spikes that are cleaned up again after some seconds.

I am using 813b725957efe64b06316f524419328b6c152f73 which is a commit after c02f116.

[Logarithmus]

Yeah, that is a known problem. We have implemented basic GC, so a steady-state RAM usage should not be that high. However, we have not yet added the limit for maximal memory usage, and during startup it can spike pretty high.

Man, are you serious?! I started using Rust because it has no GC, provides memory safety and you can use it without a HUGE memory-hogging tooling (yes, I'm speaking about IntelliJ IDEA). I'm pretty happy with rls, sometimes it stops working, but restart helps. And memory usage rarely goes higher than 500 MB (when project has no big depencies). As far as I understand, in future rls will be deprecated in favor of rust-analyzer. And what we seeing here is that Rust's tooling is going to be more memory hungry than Java's?! Why even introduce garbage collection in rust-analyzer when we already have bad experience with it in JVM ecosystem?
Please, make something about this.

[kiljacken]

@Logarithmus I am unsure if you're being sarcastic, so I'll give you a serious answer:

TL:DR; rust-analyzer caches a lot of stuff to go fast. The "GC" mentioned is just clearing some of the caches.

Rust-analyzer doesn't have a garbage collector. We just borrow the naming. To be able to answer queries about the loaded program quickly we keep a lot of data in memory about the structure and semantics of the code. This include things like:

• Syntax trees
• Definitions of types
• Inferred types for expressions
• etc.

The different things depend on each other in a large web we manage using a library called salsa which is meant to do this kind of "on-demand" computing. It automatically manages the dependency network and caching of results along the way to keep it fast if nothing has changed.

Salsa however (as I understand it) doesn't attempt eagerly free up memory for results that have been invalidated/gone out-of-date. So when we "garbage collect" we are simply telling salsa to go clean up values that aren't valid anymore so we can save some memory.

I hope that clears things up and please don't hesitate to ask if some of the above is confusing.

EDIT: We are also constantly attempting to keep memory usage low, but it's a balancing act between optimization and actually getting features working that improve the IDE experience.

[Logarithmus]

@kiljacken Thank you for such a rapid reply! I wasn't sarcastic and thought that this project implements something like Chromium's GC. What's the current state of memory usage in rust-analyzer? Last time I used it a couple of months ago. Are there any efforts to optimize it? Or maybe it's more of salsa problem?

I'm going to try making some RAM optimizations in RA, when will have some free time.

Also, why did you start development of rust-analyzer instead of improving rls? Are there some architectural limitations in the latter? I'm asking because it looks like reinventing the wheel.

[lnicola]

See https://github.com/nikomatsakis/rfcs/blob/rust-analyzer/text/0000-rust-analyzer.md#motivation.

[kiljacken]

@Logarithmus I myself is a bit out of the loop, but I don't think there's been anything major memory wise recently. There's been some interesting work on an alternate syntax tree representation over in rust-analyzer/rowan#58 (rowan is the syntax tree representation we use).

Syntax trees used to be the major offender until we added a limit on how much can be cached. I'm not sure if @matklad wants to pipe in on what is currently the main memory hog.

The new release blog post has a good recap of rls vs rust-analyzer, also the RFC for focusing on rust-analyzer as a replacement for rls.

[RalfJung]

In the rustc worksapce, RA quickly uses more than 2GB of RAM. On my 16GB RAM machine I had to actually reduce the parallel jobs for stage 2 builds to 4 as otherwise, librustc_middle fails to compile with OOM (granted, mostly that's because librustc_middle takes enormous amounts of RAM to builds with 8-way parallelism, but without RA that's not a problem on my machine).

[lfdominguez]

RA is using huge RAM on my 8GB PC, already is using 18% of my RAM (1.42 GB) for a simple workspace on vscode of a prometheus exporter, why?

Now if i restart vscode the consume returns to low... this RA doesnt have any old indexes cleaner?? or if is a thrid party library pass the issue to that library...

[lnicola]

There's a "Rust Analyzer: Memory Status" command that you can use to drop some caches and another one that restarts the language server. But if you're using https://github.com/tikv/rust-prometheus/, that's about 1 353 009 lines of code, excluding the standard library (another 350 406 lines). That's quite a lot.

Of course, rust-analyzer could use less memory itself. Some recent optimizations have helped a bit, and more will probably come in the future.

[djmittens]

Hi guys 👋 , i am running RA on manjaro linux and vscode on a vm with 9 GB of space, simply following the rogue like tutorial, my vm runs out of memory completely only running vscode at a cadence of about 2 hours. Please help, its borderline unusable :(

[lnicola]

What is the memory usage of the server? There's a command to show a memory usage report, can you paste both that and the output from ps?

Note that if you're running cargo check on save, you might run out memory on clean builds if your VM has a large number of CPUs.

[jeromegn]

Not sure if this is the same issue, but memory usage grows "constantly" (as I make changes and save) on my project.

Resident memory usage is at 8064M right now. This will sometimes consume all my available memory + swap and make my system unresponsive. I need to restart the server to get memory back, before it happens.

Here's the output of the Memory Usage command:

Per-query memory usage:
   778mb TraitImplsInDepsQuery
   391mb ItemTreeQuery
   268mb ImplDatumQuery
   256mb BodyQuery
   193mb InferQueryQuery
   178mb CrateDefMapQueryQuery
   119mb MacroArgTextQuery
   103mb BodyWithSourceMapQuery
    91mb TraitSolveQuery (purge)
    90mb FileTextQuery (purge)
    83mb TraitSolveQuery (deps)
    75mb ImplDatumQuery (purge)
    41mb ImplDatumQuery (deps)
    40mb ImplDataQuery
    35mb FunctionDataQuery
    32mb InternEagerExpansionQuery
    26mb ExprScopesQuery
    26mb GenericParamsQuery
    21mb InferQueryQuery (deps)
    21mb AttrsQuery
    20mb MacroExpandQuery (purge)
    19mb MacroDefQuery
    19mb AstIdMapQuery (purge)
    19mb GenericParamsQuery (purge)
    19mb ItemTreeQuery (purge)
    18mb ParseMacroQuery (purge)
    17mb InternMacroQuery
    16mb MacroArgTextQuery (purge)
    15mb AssociatedTyValueQuery
    15mb AstIdMapQuery
    15mb InternMacroQuery (purge)
    14mb ImplSelfTyQuery (purge)
    13mb GenericPredicatesQuery (purge)
    13mb ImplTraitQuery (purge)
    11mb FnDefDatumQuery
    11mb ImplDataQuery (purge)
     9mb InternFunctionQuery
     8mb ImplTraitQuery
     6mb ParseMacroQuery (deps)
     6mb InternFunctionQuery (purge)
     6mb AssociatedTyValueQuery (purge)
     6mb GenericPredicatesQuery
     5mb ImportMapQuery
     5mb TraitSolveQuery
     5mb InternConstQuery
     5mb TraitImplsInCrateQuery
     5mb InternConstQuery (purge)
     5mb ImplTraitQuery (deps)
     5mb FunctionDataQuery (purge)
     5mb StructDataQuery
     5mb ImplSelfTyQuery (deps)
     5mb MacroExpandQuery (deps)
     5mb ImplDataQuery (deps)
     5mb InternImplQuery (purge)
     5mb TypeAliasDataQuery
     4mb CallableItemSignatureQuery
     4mb MacroArgTextQuery (deps)
     4mb ItemTreeQuery (deps)
     4mb InternImplQuery
     4mb GenericParamsQuery (deps)
     4mb AttrsQuery (deps)
     4mb TyQuery (purge)
     4mb FileSourceRootQuery
     4mb ValueTyQuery (purge)
  3712kb TraitDatumQuery
  3709kb GenericPredicatesQuery (deps)
  3339kb AstIdMapQuery (deps)
  3204kb SourceRootCratesQuery (deps)
  2645kb SourceRootQuery (purge)
  2624kb BodyWithSourceMapQuery (deps)
  2607kb StructDatumQuery
  2600kb GenericDefaultsQuery (purge)
  2530kb CallableItemSignatureQuery (purge)
  2481kb StructDatumQuery (purge)
  2238kb AssociatedTyValueQuery (deps)
  2234kb CrateDefMapQueryQuery (deps)
  2146kb EnumDataQuery
  2099kb FieldTypesQuery
  2065kb CrateGraphQuery (purge)
  1995kb TraitDatumQuery (purge)
  1962kb BodyQuery (purge)
  1955kb BodyWithSourceMapQuery (purge)
  1950kb InferQueryQuery (purge)
  1940kb ExprScopesQuery (purge)
  1862kb FnDefDatumQuery (purge)
  1696kb InternTypeAliasQuery (purge)
  1563kb InternTypeAliasQuery
  1513kb TypeAliasDataQuery (purge)
  1479kb CallableItemSignatureQuery (deps)
  1417kb TyQuery
  1409kb FunctionDataQuery (deps)
  1164kb TyQuery (deps)
  1071kb LangItemQuery (purge)
   998kb ValueTyQuery
   953kb ParseMacroQuery
   947kb TraitImplsInCrateQuery (deps)
   946kb ModuleLangItemsQuery (deps)
   936kb FileSourceRootQuery (purge)
   934kb InternStructQuery
   911kb ValueTyQuery (deps)
   903kb ModuleLangItemsQuery (purge)
   901kb ConstDataQuery
   835kb FieldTypesQuery (purge)
   784kb InternStructQuery (purge)
   767kb FileSymbolsQuery
   760kb FnDefDatumQuery (deps)
   752kb TraitDatumQuery (deps)
   712kb GenericDefaultsQuery
   690kb MacroExpandQuery
   680kb GenericDefaultsQuery (deps)
   644kb InternTypeParamIdQuery (purge)
   636kb StructDatumQuery (deps)
   624kb InternCallableDefQuery (purge)
   535kb ConstDataQuery (purge)
   497kb StructDataQuery (purge)
   476kb InherentImplsInCrateQuery
   459kb ParseQuery
   446kb FieldTypesQuery (deps)
   406kb InherentImplsInCrateQuery (deps)
   394kb TraitDataQuery
   391kb TypeAliasDataQuery (deps)
   388kb InternEagerExpansionQuery (purge)
   375kb GenericPredicatesForParamQuery
   372kb ImplSelfTyQuery
   368kb BodyQuery (deps)
   366kb ExprScopesQuery (deps)
   345kb InternClosureQuery (purge)
   320kb MacroDefQuery (purge)
   299kb GenericPredicatesForParamQuery (purge)
   237kb DocumentationQuery
   234kb CrateDefMapQueryQuery (purge)
   223kb AssociatedTyDataQuery
   214kb LineIndexQuery
   206kb LangItemQuery (deps)
   205kb StaticDataQuery
   202kb ReturnTypeImplTraitsQuery
   191kb TraitImplsInDepsQuery (deps)
   184kb TraitDataQuery (purge)
   161kb DocumentationQuery (purge)
   156kb ConstDataQuery (deps)
   145kb StructDataQuery (deps)
   135kb EnumDataQuery (purge)
   132kb GenericPredicatesForParamQuery (deps)
   128kb InternStaticQuery
   127kb InternEnumQuery
   122kb TraitImplsInCrateQuery (purge)
   116kb CrateLangItemsQuery (purge)
   114kb TraitImplsInDepsQuery (purge)
   106kb TraitDataQuery (deps)
    99kb InternTraitQuery
    99kb StaticDataQuery (purge)
    98kb InternEnumQuery (purge)
    98kb InternStaticQuery (purge)
    98kb InternTraitQuery (purge)
    81kb InternImplTraitIdQuery (purge)
    67kb CrateLangItemsQuery (deps)
    53kb AssociatedTyDataQuery (purge)
    51kb MacroDefQuery (deps)
    51kb SourceRootCratesQuery (purge)
    47kb CrateLangItemsQuery
    47kb InherentImplsInCrateQuery (purge)
    43kb ReturnTypeImplTraitsQuery (purge)
    40kb EnumDataQuery (deps)
    38kb SourceRootCratesQuery
    37kb ReturnTypeImplTraitsQuery (deps)
    36kb AssociatedTyDataQuery (deps)
    29kb DocumentationQuery (deps)
    26kb StaticDataQuery (deps)
    14kb LineIndexQuery (purge)
    12kb ImportMapQuery (purge)
    12kb FileSymbolsQuery (purge)
    10kb UnionDataQuery
     9kb ImportMapQuery (deps)
     9kb ModuleLangItemsQuery
   3184b InternUnionQuery (purge)
   2848b LibraryRootsQuery (purge)
   2816b InternUnionQuery
   2656b LineIndexQuery (deps)
   2048b FileSymbolsQuery (deps)
   1696b UnionDataQuery (purge)
    432b UnionDataQuery (deps)
    304b LocalRootsQuery (purge)
      0b FileSourceRootQuery (deps)
      0b SourceRootQuery
      0b SourceRootQuery (deps)
      0b LangItemQuery
      0b AssociatedTyDataQuery
      0b AssociatedTyDataQuery (deps)
      0b AssociatedTyDataQuery (purge)
      0b InternCallableDefQuery
      0b InternCallableDefQuery (deps)
      0b InternTypeParamIdQuery
      0b InternTypeParamIdQuery (deps)
      0b InternImplTraitIdQuery
      0b InternImplTraitIdQuery (deps)
      0b InternClosureQuery
      0b InternClosureQuery (deps)
      0b LibrarySymbolsQuery
      0b LibrarySymbolsQuery (deps)
      0b LibrarySymbolsQuery (purge)
      0b LocalRootsQuery
      0b LocalRootsQuery (deps)
      0b LibraryRootsQuery
      0b LibraryRootsQuery (deps)
      0b InternMacroQuery (deps)
      0b InternEagerExpansionQuery (deps)
      0b InternFunctionQuery (deps)
      0b InternStructQuery (deps)
      0b InternUnionQuery (deps)
      0b InternEnumQuery (deps)
      0b InternConstQuery (deps)
      0b InternStaticQuery (deps)
      0b InternTraitQuery (deps)
      0b InternTypeAliasQuery (deps)
      0b InternImplQuery (deps)
      0b InternTypeParamIdQuery
      0b InternTypeParamIdQuery (deps)
      0b InternTypeParamIdQuery (purge)
 -1360kb ParseQuery (purge)
 -3021kb CrateGraphQuery
 -3621kb ParseQuery (deps)
    -4mb FileTextQuery
    -4mb FileTextQuery (deps)
    -6mb CrateGraphQuery (deps)
 -4071mb AttrsQuery (purge)

(note: database has been cleared)

Running this command does not reduce memory usage.

[lnicola]

 -4071mb AttrsQuery (purge)

Does that mean that it increases by 4 GB after the purge? :confused:

[jeromegn]

@lnicola I checked the memory before running the command and after and it wasn't much different (maybe 100MB lower after running the command)

Edit: oh, do you mean a "purge" would've happened at some point and make my memory grow by 4GB?

[bjorn3]

When invoking the "memory usage" command, it should free the memory of all queries one at a time. It then reports the amount of freed memory as coming from said query. In this case for some reason the memory usage supposedly increased by 4GB when attempting to free the memory for AttrsQuery.

[lnicola]

Either the numbers are inaccurate, or the memory freed up when you ran "Memory Usage" got used immediately for AttrsQuery so the total remained about the same.

[bjorn3]

Could this be caused by #6153 causing the cache to be primed while executing the "memory usage" command?

[jeromegn]

Unfortunately the project is closed-source, so I can't really show you much code.

This seems to be happening since the last 2 stable releases. I will downgrade to 3 releases ago to confirm.

[jeromegn]

Can confirm, downgrading to this release: https://github.com/rust-analyzer/rust-analyzer/releases/tag/2020-10-12 does not have the leak. The output of the Memory Usage command does not have negative numbers either.

[jeromegn]

Should I create a new issue for this?

[lnicola]

@jeromegn #6441 might have helped a little so you might want to test again, but it's probably fine to file a new issue since we have a suspicion about what caused it.

[jonas-schievink]

Could this be caused by #6153 causing the cache to be primed while executing the "memory usage" command?

The cache only gets primed again when something is typed, so don't do that while memory usage is being computed.

6441 should not affect memory usage, it only throttles how frequently the progress is reported.

I've also seen occasional memory usage spikes in the order of 6-7 GB, and purging the database only reduced it by maybe 500 MB. I'm not sure where the rest comes from, it might be the allocator behaving weirdly, or some memory leak.

Note that the query memory usage reported in https://github.com/rust-analyzer/rust-analyzer/issues/1252#issuecomment-717885482 totals to 3-4 GB, while the actual memory usage is 8 GB, so something odd is going on.

[jeromegn]

@jeromegn #6441 might have helped a little so you might want to test again, but it's probably fine to file a new issue since we have a suspicion about what caused it.

It _appears_ to have helped. Been running yesterday's nightly for a while and have not seen a huge rise in memory usage.

[kjeremy]

Maybe we should bake this into our profiling: https://github.com/nnethercote/dhat-rs