Roslyn: DiagnosticIncrementalAnalyzer allocates large amounts of memory just finding/figuring out analyzers

[davkean]

In the SDK version of https://github.com/dotnet/project-system/issues/4078, https://developercommunity.visualstudio.com/content/problem/354199/memory-consumption-with-sdk-projects.html, the following is responsible for ~12% of the trace (330 MB). GC is at 18.9% of the total CPU.

Digging in - there seems to be lots of overhead in just getting figuring out what analyzers to run, for example, there's 40 MB in just creating type names:

[sharwell]

This should be an easy fix. We don't need to construct the type name until after we determine the set of supported languages is not empty. Currently, all type names in all assemblies listed as analyzers get expanded. After the change, only type names for types registered as analyzers would be expanded.

[davkean]

Bear in mind that type name is just one case that I pulled, I did not dig further into the other cases.

[heejaechang]

this part shouldn't be different between SDK vs classic projects so the big difference should come from different part though?

but regardless, any reduction of allocations is good if possible +1!

[iharwell]

I'll take a shot at the type name issue.

[davkean]

@sharwell Can you please look at the trace to look at the other data being produced? The type name was just one example and just producing 40MB out of the 300MB produced on this path.

[sharwell]

@davkean I looked again and didn't see any other low hanging fruit on this path which would result in an observable improvement. The remaining work would be bucketed under #26778.

[davkean]

So analyzers allocating 12% of opening a solution is expected?

[sharwell]

@davkean The last time I measured the analyzer path, the overall allocations were around 50-60GiB of data. The 40MiB improvement was kept for its simplicity, but generally we'd be looking at a target of 1GiB for a small improvement here. When viewing a complete snapshot including the analysis process, 12% would actually be surprisingly low for me for overall allocations within the analyzer driver. If you are interested in capturing a larger snapshot to make further improvements to the scenario, I would recommend starting with the GC Only scenario on this page.