Sdk: Cannot invoke project dependency command on a library

[natemcmaster]

Update: the error is only on libraries, not standalone.

It looks like https://github.com/dotnet/cli/pull/2617 broke PackagedCommandSpecFactory in tools on standalone apps.

Steps to reproduce

Create a tool that invokes tries to execute a dependency command where projectContext is a netstandard library.

``` c#
new ProjectDependenciesCommandFactory(
projectContext.TargetFramework,
Constants.DefaultConfiguration,
null,
null,
projectContext.ProjectDirectory
)
.Create("dotnet-my-tool",
new []{"--dispatch", "--verbose"},
projectContext.TargetFramework,
Constants.DefaultConfiguration)

Repro project: https://github.com/natemcmaster/dotnet-sample-tool/tree/issue2645. 
## Expected behavior

ProjectDependenciesCommandFactory should create and execute a command on the netstandard library.
## Actual behavior

On a library project

Running C:\Users\namc\dev\dotnet-sample-tool.dotnet\shared\Microsoft.NETCore.App\1.0.0-rc2-3002476\corehost.exe --runtimeconfig C:\Users\namc\dev\dotnet-sample-tool\SampleLibrary\bin\Debug\netstandard1.5\SampleLibrary.runtimeconfig.json --depsfile C:\Users\namc\dev\dotnet-sample-tool\SampleLibrary\bin\Debug\netstandard1.5\SampleLibrary.deps.json --additionalprobingpath C:\Users\namc.nuget\packages C:\Users\namc.nuget\packages\dotnet-my-tool\1.0.0-tmp1461321718.66929\lib\netstandard1.5\dotnet-my-tool.dll --dispatch --verbose
Process ID: 55528
Could not resolve coreclr path

## Environment data

Microsoft.DotNet.Cli.Utils/1.0.0-rc2-002476

`dotnet --info` output:

.NET Command Line Tools (1.0.0-rc2-002476)

Product Information:
Version: 1.0.0-rc2-002476
Commit Sha: 1a8d393b9b

```

Fails on windows and linux.

Trace: corehost_trace.log.txt

cc @schellap @piotrpMSFT

[natemcmaster]

Updated the original issue description. It looks like I had two bugs conflated, so I redacted the description. The first bug was resolved by deleting muxer. The second bug, however, "Could not resolve coreclr path" error persists.

Incidentally, why don't the install scripts delete the muxer for in-place upgrades?

[schellap]

I took a first stab and this file doesn't exist:

C:\Users\namc\dev\dotnet-sample-tool\SampleLibrary\bin\Debug\netstandard1.5\SampleLibrary.runtimeconfig.json

probably because there is no target framework (Microsoft.NETCore.App) in the project.json.

[natemcmaster]

@schellap and I took a look at this offline. This looks like a design issue, not a bug. "Could not find coreclr" is accurate because there is no coreclr referenced from a library project.

This is a common scenario we want to enable for Entity Framework. User's often use EF on a library that is shared between web, desktop, etc.

{
  "version": "1.0.0-*",
  "name": "Contoso.DataAccess",
  "dependencies": {
    "NETStandard.Library": "1.5.0-*",
    "Microsoft.EntityFrameworkCore.Sqlite": "1.0.0-*",
    "dotnet-ef": {
      "version": "1.0.0-*",
      "type": "build"
    }
  },
  "frameworks": {
    "netstandard1.5": {
      "imports": "portable-net452+win81"
    }
  },
  "tools": {
    "dotnet-ef": "1.0.0-*"
  }
}

How can a user run a dependency tool on this kind of project?

cc @DamianEdwards @rowanmiller @piotrpMSFT

[natemcmaster]

User's can workaround this by invoking the tool a different project, such as a "netcoreapp1.0" web project that references the library. This is unfortunately misleading because means "tools" can still install in a library.

Example: to add migrations to "Contoso.DataAccess", you need to invoke dotnet-ef on "Contoso.Web" which references "Contoso.DataAccess".

I spoke offline with @rowanmiller and we are okay with this limitation for RC2, so I'm removing the blocking label and milestone.

Regardless, it would still be useful to invoke "dependency"-type tools on library projects. This is a general limitation, not just for EF.

[TheRealPiotrP]

@weshaggard @ericstj @davidfowl @yishaigalatzer for their thoughts on this one. Maybe we can have the library target both netstandard1.x and netcoreapp1.0. The tool would run against the netcoreapp target while we would only ship the netstandard target. Fairly easy to do via command line, but do we want to make this a cleaner experience? What are other options?

[natemcmaster]

Maybe we can have the library target both netstandard1.x and netcoreapp1.0.

Could work, but I've found this buggy. For example, this project.json has the same error as the original issue above. No runtimeconfig.json is produced for netcoreapp1.0.

{
  "version": "1.0.0-*",
  "dependencies": {
    "Microsoft.EntityFrameworkCore.Sqlite": "1.0.0-*"
  },
  "frameworks": {
    "netstandard1.5": {
      "imports": [
        "portable-net452+win81"
      ],
      "dependencies": {
        "NETStandard.Library": "1.5.0-*"
      }
    },
    "netcoreapp1.0": { 
      "compilationOptions": {
        "emitEntryPoint": true
      },
      "imports": [
        "portable-net452+win81"
      ],
      "dependencies": {
        "Microsoft.NetCore.App": {
          "type": "platform",
          "version": "1.0.0-*"
        }
      }
    }
  }
}

[ericstj]

Which is the run-able entity here, the TOOL? In that case should the restore be happening for the TOOL's TFM and merging the dependencies of the project as if it were a dependency of the tool?

[TheRealPiotrP]

I chatted with @ericstj today. He'll put some thought into the model and may have feedback/suggestions. For the time being, however, we need to understand why the project.json above is not producing a deps file/runnable output.

[eerhardt]

I pulled https://github.com/natemcmaster/dotnet-sample-tool and was able to get Pass. Called inside the project. :) printed out by build.ps1 for SampleLibrary. The thing I needed to fix is to put netcoreapp1.0 as the first TFM in the project.json. The CLI is using the first TFM in the file to run.

I chatted with @brthor today, and he mentioned a decent idea: If the library isn't already generating the necessary runtimeconfig.json file, maybe the CLI should generate the runtime files for the library as if the project.json was targetting netcoreapp1.0. That way, it wouldn't be the responsibility of the library developer. Instead the tools would have the responsibility of making this scenario work.

Thoughts?

[eerhardt]

I've figured out why I needed to put netcoreapp1.0 as the first TFM in the project.json - it was because of this line in the dotnet-my-tool code:

https://github.com/natemcmaster/dotnet-sample-tool/blob/master/dotnet-my-tool/Program.cs#L21

C# var framework = project.GetTargetFrameworks().Select(f=>f.FrameworkName).First();

The tool was only running against the first TFM, which in this case was netstandard1.5, which isn't a "runnable" TFM.

I've pushed some changes to the dotnet-my-tool in my fork to try invoking for all TFMs until it finds one that works: https://github.com/eerhardt/dotnet-sample-tool/commit/1f97aab0e21891ce0df625604fc550c1aaced038

The CLI behavior is currently by design. In order for dependency tools to be invoked, they need to be invoked on a runnable TFM - namely netcoreapp. To run them against a library you would either need to:

1. Add netcoreapp1.0 as a TFM to the project.json.
   
   
   
   • Note the tool will need to find this runnable TFM and use it.
   
   
2. Add a new project that references the library, and invoke the tool on it. (like @natemcmaster mentions above)

There isn't anything the CLI can do in this area in the RC2 timeframe. Pushing to RTM in the hopes we can get support for this scenario from CoreFX. Note: the general scenario is pretty common: you have a tool that wants to load up a library and start invoking code in it. Test runners have this exact same scenario as well. The problem is that you need a runtime to run the app, but there isn't a runtime for the library. And you can't use the tool's runtime because it may not be the correct version that the library needs.

[natemcmaster]

Has there been any progress on this? There has been a lot of chatter about this limitation, especially because the workarounds are not easy.

[Villason]

Potentially related to https://github.com/aspnet/EntityFramework/issues/5460

[kjbetz]

@natemcmaster I put a different solution for https://github.com/aspnet/EntityFramework/issues/5460 here: https://github.com/aspnet/EntityFramework/issues/5460#issuecomment-221167712

cc/ @Villason

[ahmedanis03]

Ok
there is really a big headache from this issue and i am stucked now.
first i was have this issue #https://github.com/aspnet/EntityFramework/issues/5460
then applied this work around https://docs.efproject.net/en/latest/cli/dotnet.html#dotnet-cli-issues
another issue raised cannot remember the link but it is related that buildoptions should be on root in project.json fixed that and Yes! dotnet ef working fine with the commands and project running.
Now i added a new entity in the "Converted Class library" updated in the database and start using it in the Web App project and it is building fine and then trying to run on iis it gave me this error

System.TypeLoadException Could not load type 'ApplicationDataModel.Models.CarsAd' from assembly 'ApplicationDataModel, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.

What ! build again and again using visual studio and nothing
then i build the "Converted Class Library" using dotnet build in the package manager console and it seems that the whole project cannot be discovered as it cannot read the dbcontext class
System.TypeLoadException Could not load type 'ApplicationDataModel.Models.ApplicationDBContext' from assembly 'ApplicationDataModel, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.TypeLoadException: Could not load type 'ApplicationDataModel.Models.ApplicationDBContext' from assembly 'ApplicationDataModel, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.
   at DubaiAlNasClassifiedAds.Startup.ConfigureServices(IServiceCollection services)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at Microsoft.AspNetCore.Hosting.Startup.ConfigureServicesBuilder.Invoke(Object instance, IServiceCollection exportServices)
   at Microsoft.AspNetCore.Hosting.Startup.ConfigureServicesBuilder.<>c__DisplayClass4_0.b__0(IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureApplicationServices()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()

and this is weird as i check the output and visual studio is using dotnet.exe !!! so why the assembly only updated when i used dotnet build via package manager.

build output:

1>------ Rebuild All started: Project: ApplicationDataModel, Configuration: Debug Any CPU ------
1>  C:\Program Files\dotnet\dotnet.exe build "D:\FreeLanceWork\ClassifiedAds\Project\ApplicationDataModel" --configuratio
1>  n Debug --no-dependencies --no-incremental
1>  Project ApplicationDataModel (.NETFramework,Version=v4.6) will be compiled because project is not safe for incrementa
1>  l compilation. Use --build-profile flag for more information.
1>  Compiling ApplicationDataModel for .NETFramework,Version=v4.6
1>  Compilation succeeded.
1>      0 Warning(s)
1>      0 Error(s)
1>  Time elapsed 00:00:02.0573535
1>   (The compilation time can be improved. Run "dotnet build --build-profile" for more information)
2>------ Rebuild All started: Project: DubaiAlNasClassifiedAds, Configuration: Debug Any CPU ------
3>------ Rebuild All started: Project: CarDatabasePopulateWebJob, Configuration: Debug Any CPU ------
4>------ Rebuild All started: Project: ImageManagementWebJob, Configuration: Debug Any CPU ------
2>  C:\Program Files\dotnet\dotnet.exe build "D:\FreeLanceWork\ClassifiedAds\Project\DubaiAlNasClassifiedAds" --configura
2>  tion Debug --no-dependencies --no-incremental
2>  Project DubaiAlNasClassifiedAds (.NETFramework,Version=v4.6) will be compiled because project is not safe for increme
2>  ntal compilation. Use --build-profile flag for more information.
2>  Compiling DubaiAlNasClassifiedAds for .NETFramework,Version=v4.6
2>  Compilation succeeded.
2>      1 Warning(s)
2>      0 Error(s)
2>  Time elapsed 00:00:03.2102388
2>D:\FreeLanceWork\ClassifiedAds\Project\DubaiAlNasClassifiedAds\Controllers\RelatedAdsViewComponent.cs(28,49): warning C
2>S1998: This async method lacks 'await' operators and will run synchronously. Consider using the 'await' operator to awa
2>it non-blocking API calls, or 'await Task.Run(...)' to do CPU-bound work on a background thread.
2>   (The compilation time can be improved. Run "dotnet build --build-profile" for more information)
3>  C:\Program Files\dotnet\dotnet.exe build "D:\FreeLanceWork\ClassifiedAds\Project\CarDatabasePopulateWebJob" --configu
3>  ration Debug --no-dependencies --no-incremental
4>  C:\Program Files\dotnet\dotnet.exe build "D:\FreeLanceWork\ClassifiedAds\Project\ImageManagementWebJob" --configurati
4>  on Debug --no-dependencies --no-incremental
3>  Project CarDatabasePopulateWebJob (.NETFramework,Version=v4.6) will be compiled because project is not safe for incre
4>  Project ImageManagementWebJob (DNX,Version=v4.6) will be compiled because project is not safe for incremental compila
4>  tion. Use --build-profile flag for more information.
3>  mental compilation. Use --build-profile flag for more information.
3>  Compiling CarDatabasePopulateWebJob for .NETFramework,Version=v4.6
4>  Compiling ImageManagementWebJob for DNX,Version=v4.6
4>  Compilation succeeded.
4>      0 Warning(s)
4>      0 Error(s)
4>  Time elapsed 00:00:02.2059212
3>D:\FreeLanceWork\ClassifiedAds\Project\CarDatabasePopulateWebJob\Functions.cs(30,24): warning CS0168: The variable 'y'
3>is declared but never used
3>  Compilation succeeded.
3>      1 Warning(s)
3>      0 Error(s)
3>  Time elapsed 00:00:02.2175889
4>   (The compilation time can be improved. Run "dotnet build --build-profile" for more information)
3>   (The compilation time can be improved. Run "dotnet build --build-profile" for more information)
========== Rebuild All: 4 succeeded, 0 failed, 0 skipped ==========

my ApplicationDataModel Project.json:

{
  "version": "1.0.0-*",
  "description": "ApplicationDataModel Class Library",
  "authors": [ "Ahmed" ],
  "frameworks": {
    "net46": {
      "buildOptions": {
        "emitEntryPoint": true
      }
    }
  },
  "buildOptions": {
    "emitEntryPoint": true
  },
  "dependencies": {
    "Microsoft.EntityFrameworkCore": "1.0.0-rc2-final",
    "Microsoft.EntityFrameworkCore.SqlServer": "1.0.0-rc2-final",
    "Microsoft.AspNetCore.Identity": "1.0.0-rc2-final",
    "Microsoft.AspNetCore.Identity.EntityFrameworkCore": "1.0.0-rc2-final",
    "Microsoft.EntityFrameworkCore.Tools": "1.0.0-preview1-final",
    "ImageProcessor": "2.3.3"
  },

  "tools": {

    "Microsoft.EntityFrameworkCore.Tools": {
      "version": "1.0.0-preview1-final",
      "imports": [
        "portable-net45+win8+dnxcore50",
        "portable-net45+win8"
      ]
    }
  }
}

dotnet -- info

.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.10586
 OS Platform: Windows
 RID:         win10-x64

BTW, the same "Converted Class Library Project" working fine in another two class library projects (web jobs) in the same solution.

Sorry for English mistakes

[natemcmaster]

@ahmedanis03 can you report this issue to https://github.com/aspnet/EntityFramework instead? Your error is different from the issue this bug was opened to track.

[natemcmaster]

We've had a few conversations internally about this. The most recent thinking is that the ASP.NET tooling that needs to load and execute user code can use an assembly load context to load the user's library into the tool process, rather than launching a new process.

Using a load context hasn't been proven to work yet, and it will likely require some tweaks to the build system to make sure a load context on a class library can get the right dependencies.

[kierenj]

Glad I found this issue here, very useful. Sorry for the annoying question, but is there a ballpark ETA on this one - just so I can make some plans this end? (An order of magnitude is good enough for me! - 1d/1w/1m/1q/1y)

[natemcmaster]

Closing. In aspnet, we refactored our tools a long time ago to not use ProjectDependenciesCommandFactory. Also, this doesn't seem like something we could ever fix in the CLI: netstandard libraries projects are not executable.