Roslyn: Error during build related to CSharp.Core.Targets

[jonathandbyrd]

Issue Description

We had a fully functioning build of a .net core 3.1 Durable Function Solution with .net testing projects. After upgrading to Visual Studio 16.8.2, we are now getting build errors during compile from csc.exe. Downgrading back to a 16.7.x (tested on 16.7.7) builds successfully again. All team members are experiencing same behavior

Small snippet of Visual Studio's Build Log

1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\Roslyn\Microsoft.CSharp.Core.targets(71,5): error :

1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\Roslyn\Microsoft.CSharp.Core.targets(71,5): error : Process is terminated due to StackOverflowException.

Steps to Reproduce

Visual Studio Enterprise v. 16.7.7
.Net Core 3.1 SDK
Solution Contains

• Durable Function Project
• Azure Function API Project
• MSTest Project

Run Build/F5
Success

Upgrade Visual Studio to 16.8.2(+)
Build/F5
Error as noted above

Expected Behavior

Solution/Project should build

Actual Behavior

After upgrading, Solution/Project is failing to build

Ask us questions

Is there anything related to Azure Functions, Tools, Durable Functions, etc., that would not be playing nice with .net 5 sdk as we assume this is the issue. Also we have not tried to "update" any projects etc. to support .net 5, right now its just running the build via VS.

Researching this issue, it seems to crop up every few years in the CSharp.Core.Targets files, it will just have a different line number, but same/similar code in question, where its failing

[rainersigwald]

That line is just the Csc compiler task invocation, so this sounds like a C# compiler bug that's causing a stack overflow. Moving to roslyn.

[jaredpar]

@jonathandbyrd can you share out the binlog that was used to build here?

The binlog should have the stack trace of the compiler crash here and that should help us triage what is going on here

[jonathandbyrd]

Update, team member discovered that disabling "code analysis" allows it work. uploading binlog now
msbuild.binlog.log

[jaredpar]

@jonathandbyrd

Thanks. Unfortunately the binlog didn't capture the stack trace of the error here. Generally for crashes it will but with stack overflow it may or may not.

Do you happen to have a crash dump of the compiler when this happens? If not you should be able to get one pretty simply by enabling dump creation (see below) and re-running the scenario.

Registry Key HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDump
DumpType DWORD 2
DumpCount DWORD 2
DumpFolder String "c:\some_folder"

That will tell us definitively what is happening here.

My suspicion looking through the log though is that it's the Microsoft.Azure.WebJobs.Extensions.DurableTask.Analyzers.dll. The rest of the analyzers are ones the .NET team owns. There in high enough use that I would expect this type of bug would've been found already by our own dogfooding.

Either way though, want to track down what changed here that is causing this to happen

[jonathandbyrd]

The csc.exe output was 600Mb and it won't allow me to upload. The binlog is still roughly the same size. Is there an alternate way I can get you the two files?

[jaredpar]

@jonathandbyrd one way is to open a VS feedback bug and then attach the logs to that. It will support files of that size.

https://developercommunity.visualstudio.com/content/problem/post.html?space=8

Once you open the issue post the link here and I can grab the files.

Or if you have a dropbox, live share, etc ... you could upload them temporarily there.

[jonathandbyrd]

Link to dev community issue

[jaredpar]

@jonathandbyrd thanks for doing that. Got a stack trace from that which shows that the issue is indeed in the DurableTask analyzer

000000c48ddfcf40 00007ffdb66c3a09 Microsoft.Azure.WebJobs.Extensions.DurableTask.Analyzers.MethodAnalyzer.RegisterAnalyzers(System.Collections.Generic.List`1, Microsoft.CodeAnalysis.Diagnostics.CompilationAnalysisContext, Microsoft.CodeAnalysis.SemanticModel)
000000c48ddfd0e0 00007ffdb66c3a09 Microsoft.Azure.WebJobs.Extensions.DurableTask.Analyzers.MethodAnalyzer.RegisterAnalyzers(System.Collections.Generic.List`1, Microsoft.CodeAnalysis.Diagnostics.CompilationAnalysisContext, Microsoft.CodeAnalysis.SemanticModel)
000000c48ddfd280 00007ffdb66c3a09 Microsoft.Azure.WebJobs.Extensions.DurableTask.Analyzers.MethodAnalyzer.RegisterAnalyzers(System.Collections.Generic.List`1, Microsoft.CodeAnalysis.Diagnostics.CompilationAnalysisContext, Microsoft.CodeAnalysis.SemanticModel)
000000c48ddfd420 00007ffdb66c3a09 Microsoft.Azure.WebJobs.Extensions.DurableTask.Analyzers.MethodAnalyzer.RegisterAnalyzers(System.Collections.Generic.List`1, Microsoft.CodeAnalysis.Diagnostics.CompilationAnalysisContext, Microsoft.CodeAnalysis.SemanticModel)
000000c48ddfd5c0 00007ffdb66c3a09 Microsoft.Azure.WebJobs.Extensions.DurableTask.Analyzers.MethodAnalyzer.RegisterAnalyzers(System.Collections.Generic.List`1, Microsoft.CodeAnalysis.Diagnostics.CompilationAnalysisContext, Microsoft.CodeAnalysis.SemanticModel)
000000c48ddfd760 00007ffdb66c03b2 Microsoft.Azure.WebJobs.Extensions.DurableTask.Analyzers.OrchestratorAnalyzer.RegisterAnalyzers(Microsoft.CodeAnalysis.Diagnostics.CompilationAnalysisContext)
000000c48ddfd900 00007ffdb66bebb9 Microsoft.CodeAnalysis.Diagnostics.AnalyzerExecutor+c.b__48_1(System.ValueTuple`2,Microsoft.CodeAnalysis.Diagnostics.CompilationAnalysisContext>)
000000c48ddfd970 00007ffdb66be8a9 Microsoft.CodeAnalysis.Diagnostics.AnalyzerExecutor.ExecuteAndCatchIfThrows_NoLock[[System.ValueTuple`2[[System.__Canon, mscorlib],[Microsoft.CodeAnalysis.Diagnostics.CompilationAnalysisContext, Microsoft.CodeAnalysis]], mscorlib]](Microsoft.CodeAnalysis.Diagnostics.DiagnosticAnalyzer, System.Action`1>, System.ValueTuple`2, System.Nullable`1)
000000c48ddfda00 00007ffdb66be681 Microsoft.CodeAnalysis.Diagnostics.AnalyzerExecutor.ExecuteAndCatchIfThrows[[System.ValueTuple`2[[System.__Canon, mscorlib],[Microsoft.CodeAnalysis.Diagnostics.CompilationAnalysisContext, Microsoft.CodeAnalysis]], mscorlib]](Microsoft.CodeAnalysis.Diagnostics.DiagnosticAnalyzer, System.Action`1>, System.ValueTuple`2, System.Nullable`1)
000000c48ddfdb20 00007ffdb66be068 Microsoft.CodeAnalysis.Diagnostics.AnalyzerExecutor.ExecuteCompilationActionsCore(System.Collections.Immutable.ImmutableArray`1, Microsoft.CodeAnalysis.Diagnostics.DiagnosticAnalyzer, AnalyzerStateData)
000000c48ddfdd50 00007ffdb66bd67e Microsoft.CodeAnalysis.Diagnostics.AnalyzerExecutor.TryExecuteCompilationActions(System.Collections.Immutable.ImmutableArray`1, Microsoft.CodeAnalysis.Diagnostics.DiagnosticAnalyzer, Microsoft.CodeAnalysis.Diagnostics.CompilationEvent, Microsoft.CodeAnalysis.Diagnostics.AnalysisScope, Microsoft.CodeAnalysis.Diagnostics.AnalysisState)
000000c48ddfddd0 00007ffdb63d21ad Microsoft.CodeAnalysis.Diagnostics.AnalyzerDriver.TryExecuteCompilationActions(System.Collections.Immutable.ImmutableArray`1>>, Microsoft.CodeAnalysis.Diagnostics.CompilationEvent, Microsoft.CodeAnalysis.Diagnostics.AnalysisScope, Microsoft.CodeAnalysis.Diagnostics.AnalysisState)
000000c48ddfdee0 00007ffda55e257b Microsoft.CodeAnalysis.Diagnostics.AnalyzerDriver+d__131.MoveNext()
000000c48ddfdff0 00007ffdb63d1e9b System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Can

I'm going to track down the owners of that analyzer and see if we can work out with them what is going on here.

@mavasani, @jmarolf FYI in case u see other reports of this.

[mavasani]

@jaredpar Seems to be coming from https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.DurableTask.Analyzers/ https://github.com/Azure/azure-functions-durable-extension/

[jaredpar]

@mavasani indeed that's the repo. Took a look at the code, found the issue and managed to create a minimal repro. Will file a bug shortly.

[jaredpar]

@jonathandbyrd thanks again for reporting this. Now that we have confirmed it's in the analyzer and have a tracking issue to look at it I'm going to close out this issue. We're already in contact with that team on how to work around this problem and will keep up the chat with them in that bug.