Sdk: OutOfMemoryException in XAML designer due to failed assembly reference resolution based on PackageReference tag

[lmagyar]

XAML Workflow Activity designer 100% throws OutOfMemoryException and crashes when some specific NuGet packages are referred through PackageReference (NOT with packages.config) using the "old" project file format (<Project ToolsVersion="15.0" ...> and NOT <Project Sdk="Microsoft.NET.Sdk">).

I think this is caused by some assembly reference resolution error in the new PackageReference related code and related to #1244, #1499

Steps to reproduce:

• VS 2017 (any VS 15.x version I've tested fails)
• fresh new empty solution with Workflow Activity library added from original VS template (net461, though any version I've tested fails)
• add any ONE of these NuGet packages with PackageReference (NOT with packages.config)
  
  
  
  • Microsoft.Orleans.Core (v1.5.3, though any version I've tested fails)
  
  
  • Microsoft.Orleans.Core.Abstractions (v2.0.0-beta3, though any version I've tested fails)
  
  
  • Abp (v3.4.0) + add reference to System.Drawing
  
  
  • Dapper (v1.50.4)
  
  
  • Orleans.Activities (v0.3.0)
  
  
• open the empty Activity1.xaml in the designer

Instead of opening the designer, VS says:

System.OutOfMemoryException: Insufficient memory to continue the execution of the program.
   at VSLangProj.Reference.get_PublicKeyToken()
   at Microsoft.VisualStudio.Activities.Helper.CreateAssemblyNameFromReference(Reference r)
   at Microsoft.VisualStudio.Activities.Helper.GetReferenceAssemblies(VSProject project)
   at Microsoft.VisualStudio.Activities.EditorPane.get_ReferencedAssemblies()
   at Microsoft.VisualStudio.Activities.EditorPane.ResetShadowCopyPath()
   at Microsoft.VisualStudio.Activities.EditorPane.CreateWorkflowDesignerInIsolatedMode()
   at Microsoft.VisualStudio.Activities.EditorPane.Microsoft.VisualStudio.Shell.Interop.IPersistFileFormat.Load(String fileName, UInt32 formatMode, Int32 readOnly)

Some of the above packages (Microsoft.Orleans.Core, Orleans.Activities, Abp) also causes the "The referenced component 'System.Net.Http' could not be found." warning, so after adding the below code to the end of the project files, the warning disappears, but the XAML designer still crashes:

  <Target Name="UpdateOriginalItemSpecs" AfterTargets="ResolveAssemblyReferences">
    <ItemGroup>
      <ReferencePath>
        <OriginalItemSpec>%(ReferencePath.FileName)</OriginalItemSpec>
      </ReferencePath>
    </ItemGroup>
  </Target>

But 2 warnings remain even after adding the above target to the project file (I think the XAML designer uses some different target for assembly resolution):

• Could not run workflow validation because file 'System.Net.Http, PublicKeyToken=b03f5f7f11d50a3a' has an incorrect format. This will not prevent workflows from running; but any workflow that has a validation error will fail at runtime. If the file is a platform-specific library or executable, consider building the project using MSBuild.exe from a command prompt of the targeted platform.
• Could not compile workflow expressions because file 'file:///C:\Users\lmagyar.nuget\packages\system.net.http\4.3.0\ref\net46\System.Net.Http.dll' has an incorrect format. Workflows in this project may still run, if they do not require expression compilation. If the file is a platform-specific library or executable, consider building the project using MSBuild.exe from a command prompt of the targeted platform.

Note: referring Microsoft.Orleans.Core.Abstractions or Dapper doesn't cause any warning, the designer simply crashes. Adding the above target workaround, doesn't change anything.

I've tried to investigate:

• I've added all the ~50 nuget packages that is referenced through Microsoft.Orleans.Core (based on the packages.config in another project), with PackageReference, no problem, but finally adding Microsoft.Orleans.Core results in the exception
• I've tried several assembly binding redirects, nothing changes, exception
• devenv.exe /safemode, exception
• I've also tried to debug devenv, but I think due to CreateWorkflowDesignerInIsolatedMode(), I wasn't able to catch the exception
• MSBuild.exe builds without any warning

This error also reported multiple times at several places:

• https://developercommunity.visualstudio.com/content/problem/61637/vs-2017-outofmemoryexception-in-xaml-activity-desi.html
• https://developercommunity.visualstudio.com/content/problem/113977/out-of-memory-exception-when-loading-workflow-xaml.html
• https://developercommunity.visualstudio.com/content/problem/162045/windows-workflow-foundation-1.html
• aspnetboilerplate/aspnetboilerplate#2805

Using the new project file format (<Project Sdk="Microsoft.NET.Sdk">) is not an option, because it doesn't contain the XamlAppDef BuildAction and though the XAML designer starts without crash, it doesn't resolve any assembly, so useless. Using packages.config as workaround is awfully slow and painful.

Maybe @dmetzgar can add some insights about the internals of the XAML Workflow Activity designer.

[sharwell]

On it...

[sharwell]

Find another report: https://developercommunity.visualstudio.com/content/problem/191537/systemoutofmemoryexception-in-workflow-designer.html

[sharwell]

@lmagyar I tracked down the source of this bug in our internal code and assigned it a team that should be able to help out. Your bug report is outstanding, and I'm sorry to see that it took so long to get eyes on it. I'm going to start a separate internal discussion related to avoiding the type of delay that occurred in reviewing your feedback items.

/cc @Pilchie

[Pilchie]

Note that @tmeschter is already looking at this.

[livarcocc]

Closing the issue here as it is already being tracked somewhere else and worked on.