Sdk: Cli tool throws exception when dispatching and calling into "PlatformAbstractions" APIs

[mrahhal]

Steps to reproduce

Repro repo here.

I have a tool that needs to load the target project, which targets the full .Net framework. As a result, I should dispatch the execution to the context of the target tfm (so net46 for example), because cli tools are always executed in the context of "netcoreapp". Dispatching like this used to work perfectly in the preview2 tooling.

After the dispatch, if I try to use any of the "PlatformAbstractions" APIs the following exception occurs:

Unhandled Exception: System.IO.FileLoadException:
Could not load file or assembly 'Microsoft.DotNet.PlatformAbstractions, Version=1.0.1.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

This line works when the context is "netcoreapp", but it throws the exception after the dispatch to "net46".

This is a major problem because "Command.CreateDotNet" fails as a result.
My tool has a reference to "Microsoft.DotNet.Cli.Utils" v1.0.1 which I suspect is the root of the problem. Or maybe this is not how dispatching should be done in 1.0? (which I doubt, because when PlatformAbstractions is not involved, everything works).

I can't find info or documentation at all about this. What's going on?

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0)

Product Information:
 Version:            1.0.0
 Commit SHA-1 hash:  e53429feb4

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\1.0.0

For reference this is the tool I maintain.

[livarcocc]

We have tests that cover this scenario running as part of every single one of our builds.

Also, Cli.Utils depends on version 1.0.3 of PlatformAbstractions, as can be seen at nuget.org. Can you check your project.assets.json and see who is depending on version 1.0.1 of platform abstractions?

[livarcocc]

Also, how can I repro this? Is there a particular command that I should use? just running the application fails, true, but with a completely different error. It fails because MSBuildExtensionsPath is not set and it fails to evaluate the project.

[mrahhal]

I'll list the exact steps I just did from scratch to repro this. But before that, some explanation:

The "dotnet-some" (this is just the assembly name of the CliToolRepro repo) tool simply prints RuntimeEnvironment.OperatingSystem before and after dispatching. This property comes from "PlatformAbstractions" so the problem occurs when this package is being loaded in the context of "net46" after dispatching. A lot of users of my real tool I'm maintaining consistently reported the exact same problem.

Exacts steps to repro:

• git clone https://github.com/mrahhal/CliToolRepro.git && cd CliToolRepro
• dotnet restore && dotnet pack
• Add the resulting .nupkg to a local feed.
• cd .. && md Test && cd Test && dotnet new mvc
• Edit csproj:
  
  
  
  • Change "TargetFramework" to "net46".
  
  
  • Add <PackageReference Include="Some.Tools" Version="1.0.0" PrivateAssets="All" />
  
  
  • Add <DotNetCliToolReference Include="Some.Tools" Version="1.0.0" />
  
  
• Finally:
  
  
  
  • dotnet restore && dotnet build // dotnet build is important here, usually the tool should have issued a build command if it does dispatching
  
  
  • dotnet some output:
  
  

> dotnet some
Windows // This is printed before dispatching
Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'Microsoft.DotNet.PlatformAbstractions, Version=1.0.1.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at CA.Program.Main(String[] args)

This problem does not occur when the app doesn't have a dependency on MVC (which pulls in "PlatformAbstractions").

There is a slight possibility that I'm doing "dispatching" wrong, it used to be easier in preview2. And I can't find real world examples of this (dispatching, and then loading and inspecting the target project's assembly).

Currently, what I'm doing is recommending that the tool be used in a separate pure class library that doesn't depend on MVC to avoid the problem.

[mrahhal]

Also,

Can you check your project.assets.json and see who is depending on version 1.0.1 of platform abstractions?

Nothing depends on 1.0.1 of PlatformAbstractions. The lowest version I'm seeing is definitely 1.0.3.

[nphmuller]

According to NuGet and project.assets.json the lowest version is indeed 1.0.3, but after building the project there's definitely a reference to 1.0.1 (InformationVersion 1.0.1-beta).
Same for Microsoft.Extensions.DependencyModel.

Here's a minimum .csproj to reproduce:
Just run dotnet restore && dotnet build

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net462</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.DotNet.Cli.Utils" Version="1.0.1" />
  </ItemGroup>
</Project>

It also reproduces with netcoreapp1.0, but it's easier to see the output with net462.

Update:
Did some more testing and <PackageReference Include="Microsoft.DotNet.PlatformAbstractions" Version="1.0.3" /> downloads the 1.0.1.0 (1.0.1-beta) package. Same for Microsoft.Extensions.DependencyModel

[livarcocc]

@mrahhal are your repro steps above still relevant? Trying it out but I see a ConsoleApp1 there instead of CliToolRepro. I also see that you are not packaging the dotnet-some.runtimeconfig.json, which is absolutely required for tools.

[mrahhal]

Not really sure. This was on SDK 1.0 too, if that makes a difference. I'll have to try again and update this repro.
But just curious, how do you package the runtimeconfig.json file? So it's not an automatic process when you build and pack?

[livarcocc]

It depends. We had a bug where it was not being packaged for an app targeting netcoreapp. This bug has since been fixed by NuGet, but I am not sure in which release this was fixed. I tried running dotnet pack on a 2.0.0 SDK and the runtimeconfig.json was packaged correctly. But my point is to make sure that that file is present in the generated nupkg.

Still interested in knowing if this still reproes, specially for an app packaged with the 2.0 SDK.

[mrahhal]

I'll check the nupkg of my latest release of Migrator.EF6 as it targets SDK 2.0 (using latest nuget at the time), and will get back to you.

[mrahhal]

It seems runtimeconfig.json is not packaged in net46, not sure what this means. Maybe I still have to try again with latest nuget and update this repro. Thoughts?

[livarcocc]

It is not needed for net46. runtimeconfig.json is only needed for .net core applications.

[livarcocc]

With this application, are you install running into PlatformAbstractions problems?

[mrahhal]

I see. Since a long time ago I've been avoiding using the tool in a project that has a reference to MVC (or PlatformAbstractions in general). And I've been recommending that. Other solutions included having assembly bindings in the csproj, as mentioned by @nphmuller above.

Maybe @nphmuller can confirm if this problem doesn't occur with latest version (2.0). I'll also give it a try later.

[nphmuller]

I always thought we ran in an edge case were we tried to run a DotNetCliToolReference on .NET full. The runetimeconfig.json file makes sure the binding redirects are right, but it's only included for a netcoreapp target. This means that if the target of the project were the CliTool is referenced in is .NET full (like net462), there's is no binding redirect file for the tool, hence the FileNotFoundException.

I think that is the underlying issue.

My workaround basically makes sure the binding redirect file of the main app is copied for the tool. See: https://github.com/mrahhal/Migrator.EF6/issues/37#issuecomment-288032738

Anyway, yeah I've had FileNotFound exceptions because of missing binding redirects in ASP.NET Core 2.0 and dotnet cli 2.0 with a net462 target. So not with a netcoreapp target. Not sure if it was PlatformAbstractions or another .NET base library. If you're interested in if it's specifically PlatformAbstractions, tell me and I'll retest it. But for now, I don't think it matters.

[livarcocc]

We don't use the runtimeconfig.json to determine binding redirects for full framework projects.

Also, if your app needs binding redirects, you should provide them.

Is the expectation that the tool invocation would do that for you somehow?

[nphmuller]

Been having a pretty hard time trying to create repro, and it turns out what I thought was plain wrong and I've been wasting your time. :wink: Every repro I tried to make took either care of the redirects or showed NU1605 warnings/errors during build.

Turns out the tool I made the workaround for uses Microsoft.Extensions.Internal.DotnetToolDispatcher, to basically re-launch the executable. While dotnet cli makes sure the binding redirects are valid for all launched tools, this doesn't work if the tool launches the executable again manually.

To see dotnet cli taking care of the binding redirects simply launch the tool executable by hand. If binding redirects were necessary, the tool crashes with a FileNotFoundException. If the tool is launched through dotnet cli, it works fine.

So I guess this can be closed as its most likely an application bug. Not a tooling bug.

@mrahhal I'll try to see if I can fix this in the coming days. It probably involves stripping out DotnetToolDispatcher.CreateDispatchCommand in Program.cs and simply calling Worker.Execute at that point. Except if there's another reason you used DotnetToolDispatcher, but I guess I'll simply find out.

[livarcocc]

Thanks for looking at this @nphmuller. It makes sense to me. I am going to close this for now. If there is still something wrong, either comment and I will re-activate or file a new issue.

[mrahhal]

@nphmuller I'll try to dig out the original conv where this whole thing started, way back when I was starting out with Migrator.EF6. But anyway, at that time, docs were very scarce on how to build a tool and I only had a couple of aspnet tools to investigate. But this was problematic because most of them didn't need to work under the full .net framework.

In my case, you know how we use EntityFramework6, which targets the full .net framework. So in order to use it, the tool must be running under full .net. Unfortunately, and as I've learned, dotnet tools are always launched in the context of netcoreapp.

So at the time, I asked a cli tooling dev and he guided me to the process known as dispatching. And that's really why I do another dispatch after the original execution. (1st execution is under netcoreapp, and here I do dispatching to re-execute the tool under full .net to be able to call EF6)

Right now, I'm not entirely sure if any tool that the aspnetcore team builds needs dispatching. I think they all target netcoreapp. That's why I wasn't able to confirm if what I'm doing is 100% compatible with how tooling works, but at least it worked, kinda 😅

But as a conclusion, it's important to stress that this problem occurs precisely because of the need to dispatch. Sorry if I didn't make this clear earlier. (As you can see from my old repro that I mentioned in this issue's description, it has dispatching.)

An excerpt from this issue's description:

After the dispatch, if I try to use any of the "PlatformAbstractions" APIs the following exception occurs.
This line works when the context is "netcoreapp", but it throws the exception after the dispatch to "net46".

(And thanks @nphmuller for the investigation!)

[nphmuller]

@mrahhal Ah, that also explains why the tool targets both netcoreapp and netfull. Makes me appreciate the effort that went into it all the more.

[nphmuller]

Just wrote a prototype and it indeed does not work yet. The project were I include a tool that only targets net46 throws a NU1202 error during package restore: Package tool-x 1.0.0 is not compatible with netcoreapp2.0.

I just found a newer version of DotnetToolDispatcher here: https://github.com/aspnet/Scaffolding/blob/8675aa97cd967d92bfa98e4008feb98853ca25e6/src/Shared/DotNetDispatcher/DotnetToolDispatcher.cs
Note the method EnsureBindingRedirects. I'm slightly optimistic they fixed the binding redirect issue.

@livarcocc : Is there an official way to get to work what @mrahhal described (a cli tool that targets netfull)? Or at least an official NuGet package with DotnetToolDispatcher? Because currently the source is copied over manually an part of the tool project, which seems error prone.

[Jupakabra]

I have similar problem, everything worked at some point in project.json era and now dispatching it or even finding examples on doing that for full .NET is hard, especially if you try to assembly load consuming project's dll. It starts throwing reference not found errors for dependency versions which are not used in any that project or the tool.

And netcoreapp2.0 just works without any problems even with additional dispatch.

[mrahhal]

I just found a newer version of DotnetToolDispatcher. Note the method EnsureBindingRedirects...

Awesome. Looks promising.

Is there an official way to get to work what @mrahhal described..

I did ask before but didn't get a direct answer. I'd really like to know if this is a use-case that was considered or not.