Arcade: New versions of GenAPI consistently crashes with InvalidCastException

Created on 18 Apr 2019 · 18Comments · Source: dotnet/arcade

GenAPI started breaking for aspnet in what appears to be the result of https://github.com/dotnet/arcade/pull/2517. This is blocking aspnetcore's ability to consume an update to Arcade.

  Unhandled Exception: System.InvalidCastException: Unable to cast object of type 'WhereEnumerableIterator`1[Microsoft.Cci.ITypeDefinitionMember]' to type 'System.Linq.IOrderedEnumerable`1[Microsoft.Cci.ITypeDefinitionMember]'.
     at Microsoft.Cci.Traversers.SimpleTypeMemberTraverser.Visit(IEnumerable`1 members) in /_/src/Microsoft.Cci.Extensions/Traversers/SimpleTypeMemberTraverser.cs:line 91
     at Microsoft.Cci.Writers.CSharpWriter.Visit(IEnumerable`1 members) in /_/src/Microsoft.Cci.Extensions/Writers/CSharp/CSharpWriter.cs:line 141
     at Microsoft.Cci.Writers.CSharpWriter.Visit(ITypeDefinition parentType, IEnumerable`1 fields) in /_/src/Microsoft.Cci.Extensions/Writers/CSharp/CSharpWriter.cs:line 205
     at Microsoft.Cci.Traversers.SimpleTypeMemberTraverser.Visit(ITypeDefinition type) in /_/src/Microsoft.Cci.Extensions/Traversers/SimpleTypeMemberTraverser.cs:line 74
     at Microsoft.Cci.Writers.CSharpWriter.Visit(ITypeDefinition type) in /_/src/Microsoft.Cci.Extensions/Writers/CSharp/CSharpWriter.cs:line 131
     at Microsoft.Cci.Traversers.SimpleTypeMemberTraverser.Visit(IEnumerable`1 types) in /_/src/Microsoft.Cci.Extensions/Traversers/SimpleTypeMemberTraverser.cs:line 62
     at Microsoft.Cci.Writers.CSharpWriter.Visit(IEnumerable`1 types) in /_/src/Microsoft.Cci.Extensions/Writers/CSharp/CSharpWriter.cs:line 110
     at Microsoft.Cci.Writers.CSharpWriter.Visit(INamespaceDefinition ns) in /_/src/Microsoft.Cci.Extensions/Writers/CSharp/CSharpWriter.cs:line 99
     at Microsoft.Cci.Traversers.SimpleTypeMemberTraverser.Visit(IEnumerable`1 namespaces) in /_/src/Microsoft.Cci.Extensions/Traversers/SimpleTypeMemberTraverser.cs:line 43
     at Microsoft.Cci.Writers.CSharpWriter.WriteAssemblies(IEnumerable`1 assemblies) in /_/src/Microsoft.Cci.Extensions/Writers/CSharp/CSharpWriter.cs:line 73
     at Microsoft.DotNet.GenAPI.Program.<>c__DisplayClass0_0.<Main>b__0() in /_/src/Microsoft.DotNet.GenAPI/Program.cs:line 95
     at Microsoft.DotNet.GenAPI.Program.Main(String[] args) in /_/src/Microsoft.DotNet.GenAPI/Program.cs:line 52
D:\a\1\s\eng\targets\ReferenceAssembly.targets(67,5): error MSB3073: The command "dotnet --roll-forward-on-no-candidate-fx 2 "D:\a\1\s\.packages\microsoft.dotnet.genapi\1.0.0-beta.19218.1\build\\..\tools\netcoreapp2.1\Microsoft.DotNet.GenAPI.dll" "D:\a\1\s\artifacts\bin\Microsoft.Extensions.Caching.Abstractions\Debug\netstandard2.0\Microsoft.Extensions.Caching.Abstractions.dll" --lib-path "D:\a\1\s\artifacts\bin\Microsoft.Extensions.Primitives\Debug\netstandard2.0;D:\a\1\s\.packages\netstandard.library\2.0.3\build\netstandard2.0\ref;D:\a\1\s\.packages\system.buffers\4.4.0\ref\netstandard2.0;D:\a\1\s\.packages\system.memory\4.5.2\lib\netstandard2.0;D:\a\1\s\.packages\system.numerics.vectors\4.4.0\ref\netstandard2.0;D:\a\1\s\.packages\system.runtime.compilerservices.unsafe\4.6.0-preview5.19217.11\ref\netstandard2.0" --out "D:\a\1\s\src\Caching\Abstractions/ref/Microsoft.Extensions.Caching.Abstractions.netstandard2.0.cs" --header-file "D:\a\1\s\/eng/LicenseHeader.txt"" exited with code -532462766. [D:\a\1\s\src\Caching\Abstractions\src\Microsoft.Extensions.Caching.Abstractions.csproj]

See https://dev.azure.com/dnceng/public/_build/results?buildId=160140&view=logs&jobId=2fe95fc5-ae3d-58fd-c2b8-46bb81541875

cc @ViktorHofer @safern

Source

natemcmaster

Most helpful comment

+1 to tests. Just having something set up so you can excercise the code goes a long way for making folks more productive. It wouldn't need to be exhaustive, just a start. Something like run GenAPI against some of the dependencies and then make sure they can be recompiled.

ericstj on 19 Apr 2019

👍2

All 18 comments

@natemcmaster the roll-forward policy 2 [major/minor/patch] only rolls forward to the next existing major if a compatible minor/patch shared framework can't be found. Can you provide more infos with which version you run GenAPI before and why picking a higher version would cause issues for you now?

ViktorHofer on 18 Apr 2019

@natemcmaster I see that you set roll-forward here: https://github.com/aspnet/Extensions/blob/master/eng/targets/ReferenceAssembly.targets#L58. Can you try remove it as it isn't necessary anymore now?

ViktorHofer on 18 Apr 2019

Working: Microsoft.DotNet.GenAPI/1.0.0-beta.19217.1
Fails with Microsoft.DotNet.GenAPI/1.0.0-beta.19218.1

The changes: https://github.com/dotnet/arcade/compare/4e21d52dabbb9f5705a90f097acb1465a0354c0d...46718d98c0fd03690a6a8c83da692a4a85a17902

Now that I'm looking closer at the actual changes that went into the new version of GenAPI, I see #2517 and the rollforward flag isn't actually part of this change yet. Sorry for the red herring.

natemcmaster on 18 Apr 2019

https://github.com/dotnet/arcade/blob/46718d98c0fd03690a6a8c83da692a4a85a17902/src/Microsoft.Cci.Extensions/Traversers/SimpleTypeMemberTraverser.cs#L91-L93

@tannergooding this throws InvalidCastException on .NET Core 3.0.0-preview5-27618-01

natemcmaster on 18 Apr 2019

I'm not sure why, considering the check was already made that members is a IOrderedEnumerable<ITypeDefinitionMember>: https://github.com/dotnet/arcade/blob/46718d98c0fd03690a6a8c83da692a4a85a17902/src/Microsoft.Cci.Extensions/Traversers/SimpleTypeMemberTraverser.cs#L89-L93

tannergooding on 18 Apr 2019

The .Where on line 90 is changing members to a different type... 'WhereEnumerableIterator`1[Microsoft.Cci.ITypeDefinitionMember]'

natemcmaster on 18 Apr 2019

Ugh. I missed that.

I'll have a fix up that reorders this shortly.

tannergooding on 19 Apr 2019

I've got a PR up here: https://github.com/dotnet/arcade/pull/2560

tannergooding on 19 Apr 2019

Thanks for the quick response time!

natemcmaster on 19 Apr 2019

Sure thing. Are you able to easily validate that the new PR resolves the problems ASP.NET was seeing? Or provide instructions on how I can do that?

tannergooding on 19 Apr 2019

@tannergooding - thoughts on adding a test or two to plug this hole after getting this fix in?

markwilkie on 19 Apr 2019

@markwilkie, It doesn't look like we have any test infrastructure in the repo today; and certainly nothing for CCI/GenAPI

Edit: Nevermind, just not in the tests folder, they are SxS in the src folder. Still, nothing for GenApi, CCI though...

tannergooding on 19 Apr 2019

It seems this would be good to add....

markwilkie on 19 Apr 2019

You can validate by updating the version of GenAPI referenced in aspnet/Extensions and run ./eng/scripts/CodeCheck.ps1. Here is the PR that caught this problem https://github.com/aspnet/Extensions/pull/1458 and the steps that failed: https://github.com/aspnet/Extensions/blob/master/azure-pipelines.yml#L26-L34

natemcmaster on 19 Apr 2019

👍1