Sdk: dotnet build puts wrong native executable/libraries into runtime specific output

Created on 7 May 2016 · 15Comments · Source: dotnet/sdk

Steps to reproduce

On OS X 10.11.4 (not tested on other OSes):

Create a standalone app targeting net451 and netcoreapp1.0 for win7-x64 and osx.10.11-x64 runtimes.
Add a native dependency for reference (e.g. Libuv).
Build and Publish for all runtimes and frameworks (Release or Debug).

Repository with a repro is available here: https://github.com/dasMulli/cli-build-artifact-issue-repro

The master branch contains the complete build output as well as the used CLI build.
To reproduce:

git clone https://github.com/dasMulli/cli-build-artifact-issue-repro.git
cd cli-build-artifact-issue-repro
git checkout initial-state
./repro.sh > repro.log

Expected behavior

Each runtime-specific directory should contain only native components and executables for the corresponding runtime.
OR
Each runtime-specific directory should only contain native components and executables for the host runtime (osx.10.11-x64 in this case). (this may make sense to run mono TestApp.exe and having *.dylib files)

Actual behavior

net451

The output for net451 is fine. When dotnet publish is run, build and publish output matches.

bin/Release/net451/osx.10.11-x64/libuv.dylib
bin/Release/net451/osx.10.11-x64/TestApp.exe
bin/Release/net451/osx.10.11-x64/TestApp.pdb

bin/Release/net451/osx.10.11-x64/publish/libuv.dylib
bin/Release/net451/osx.10.11-x64/publish/TestApp.exe
bin/Release/net451/osx.10.11-x64/publish/TestApp.pdb

bin/Release/net451/win7-x64/libuv.dll
bin/Release/net451/win7-x64/TestApp.exe
bin/Release/net451/win7-x64/TestApp.pdb

bin/Release/net451/win7-x64/publish/libuv.dll
bin/Release/net451/win7-x64/publish/TestApp.exe
bin/Release/net451/win7-x64/publish/TestApp.pdb

netcoreapp1.0

The output for netcoreapp1.0 is different: For the win7-x64 build output, an OS X executable and OS X libraries are dropped. The publish output is correct, containing Windows executable / runtime.

bin/Release/netcoreapp1.0/osx.10.11-x64/libhostfxr.dylib
bin/Release/netcoreapp1.0/osx.10.11-x64/libhostpolicy.dylib
bin/Release/netcoreapp1.0/osx.10.11-x64/TestApp <-- executable
bin/Release/netcoreapp1.0/osx.10.11-x64/TestApp.dll
bin/Release/netcoreapp1.0/osx.10.11-x64/TestApp.pdb
… (json-files)

bin/Release/netcoreapp1.0/osx.10.11-x64/publish/TestApp <-- executable
bin/Release/netcoreapp1.0/osx.10.11-x64/publish/TestApp.deps.json
bin/Release/netcoreapp1.0/osx.10.11-x64/publish/TestApp.dll
bin/Release/netcoreapp1.0/osx.10.11-x64/publish/TestApp.pdb
bin/Release/netcoreapp1.0/osx.10.11-x64/publish/libuv.dylib
… (more dylib-files, runtime, dependencies)

bin/Release/netcoreapp1.0/win7-x64/libhostfxr.dylib <-- bug?: OS X dylibs for Windows runtime
bin/Release/netcoreapp1.0/win7-x64/libhostpolicy.dylib
bin/Release/netcoreapp1.0/win7-x64/TestApp <-- bug?: OS X executable for Windows runtime
bin/Release/netcoreapp1.0/win7-x64/TestApp.dll
bin/Release/netcoreapp1.0/win7-x64/TestApp.pdb
… (json-files)

bin/Release/netcoreapp1.0/win7-x64/publish/TestApp.dll
bin/Release/netcoreapp1.0/win7-x64/publish/TestApp.exe <-- fine: Windows executable
bin/Release/netcoreapp1.0/win7-x64/publish/hostfxr.dll <-- fine: Windows libraries
bin/Release/netcoreapp1.0/win7-x64/publish/hostpolicy.dll
bin/Release/netcoreapp1.0/win7-x64/publish/libuv.dll
… (more dll-files, runtime, dependencies)

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0-preview1-preview1-002697)

Product Information:
 Version:     1.0.0-preview1-preview1-002697
 Commit Sha:  08b3ad9e88

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.11
 OS Platform: Darwin
 RID:         osx.10.11-x64

Source

dasMulli

All 15 comments

I think its a duplicate of https://github.com/dotnet/cli/issues/2343. We do not have support for cross-plat build yet.

Sridhar-MS on 8 May 2016

Possibly. Didn't find that before but looking at it i see dotnet/sdk#5631 more as the "for all runtimes in one invocation" part. dotnet build --help shows that there is a runtime switch and if i invoke dotnet build -f netcoreapp1.0 -r win7-x64 then a (runnable) OS X output is produced into bin/Debug/netcoreapp1.0/win7-x64.
However, if dotnet/sdk#5631 works correctly, that should fix this problem too.

dasMulli on 8 May 2016

@dasMulli ... that sounds very similar to this: https://github.com/dotnet/cli/issues/317#issuecomment-210562700 ... where we were told to either have the latest Microsoft.NETCore.App package or have the Microsoft.NETCore.DotNetHostPolicy package to get the host you need.

If that turns out to be the same thing you're seeing, you can either specify Microsoft.NETCore.App with a version of 1.0.0-* and feeds that limit to rc2 packages (e.g., CI Release) or specify Microsoft.NETCore.App with a version of 1.0.0-rc2-* and feeds that don't limit to rc2. I don't use that package; I just use the Microsoft.NETCore.DotNetHostPolicy directly with version 1.0.1-*.

guardrex on 8 May 2016

@GuardRex thanks for the input, but cross-publishing works fine. dotnet publish -f netcoreapp1.0 -r win7-x64 produces runnable outputs for windows, however dotnet build -f netcoreapp1.0 -r win7-x64 creates OS X output.
My point is it should either put executables / libs into the build for the targeted RID or nothing at all (if cross-building is not supported).
Browsing around the code, i think this TODO explains what is missing.

dasMulli on 8 May 2016

mmm ... I see. I can't repro this behavior on Windows. dotnet build provided good outputs for both win7-x64 and osx.10.11-x64 using the command dotnet build -f netcoreapp1.0 -r <RID>.

guardrex on 8 May 2016

@GuardRex Interesting. I just tried it on my win10 vm and it repros giving me a
C:\…\cli-build-artifact-issue-repro\TestApp\bin\Debug\netcoreapp1.0\osx.10.11-x64\TestApp.exe together with hostfxr.dll and hostpolicy.dll.
Which version of the CLI did you use? I used this project.json, also varied using the Microsoft.NETCore.DotNetHostPolicy package and putting the dependencies on the root level removing net451

dasMulli on 8 May 2016

@dasMulli I'll try your project.json. When I did my test, I was using my test self-contained project.json ...

    "buildOptions": {
        "preserveCompilationContext": true,
        "emitEntryPoint": true,
        "warningsAsErrors": true,
        "debugType": "portable",
        "xmlDoc": true,
        "compile": {
            "exclude": [
                "wwwroot"
            ]
        },
        "copyToOutput": {
            "include": [ "Views", "wwwroot", "Logs" ]
        }
    },
    "dependencies": {
        "Microsoft.AspNetCore.Diagnostics": "1.0.0-*",
        "Microsoft.AspNetCore.Server.IISIntegration": "1.0.0-*",
        "Microsoft.Extensions.Logging.Console": "1.0.0-*",
        "Microsoft.AspNetCore.StaticFiles": "1.0.0-*",
        "Microsoft.AspNetCore.Mvc": "1.0.0-*",
        "Microsoft.AspNetCore.Server.Kestrel": "1.0.0-*",
        "NETStandard.Library": "1.5.0-*",
        "Microsoft.NETCore.DotNetHostPolicy": "1.0.1-*"
    },
    "frameworks": {
        "netcoreapp1.0": {
            "imports": [ "portable-dnxcore50+net45+win8+wp8+wpa81" ]
        }
    },
    "publishOptions": {
        "include": [ "Views", "wwwroot", "Logs" ]
    },
    "runtimes": {
        "win81-x64": {}
    },

I'll try the one you're testing now.

guardrex on 8 May 2016

I'm still unable to repro it. All I did to your project.json was add the deps that my test self-contained app needs ...

{
  "version": "1.0.0-*",
  "buildOptions": {
    "emitEntryPoint": true
  },
  "dependencies": {
    "Microsoft.NETCore.Platforms": "1.0.1-rc2-*",
    "Libuv": "1.9.0-rc2-*",
    "Microsoft.AspNetCore.Diagnostics": "1.0.0-*",
    "Microsoft.AspNetCore.Server.IISIntegration": "1.0.0-*",
    "Microsoft.Extensions.Logging.Console": "1.0.0-*",
    "Microsoft.AspNetCore.StaticFiles": "1.0.0-*",
    "Microsoft.AspNetCore.Mvc": "1.0.0-*",
    "Microsoft.AspNetCore.Server.Kestrel": "1.0.0-*"
  },
  "frameworks": {
    "net451": {},
    "netcoreapp1.0": {
      "imports": "dnxcore50",
      "dependencies": {
        "Microsoft.NETCore.App": {
          "version": "1.0.0-rc2-*",
          "type": "default"
        }
      }
    }
  },
  "runtimes": {
    "win7-x64": {},
    "osx.10.11-x64": {}
  }
}

I'm getting all good output in both directions ...

dotnet build 
    --framework netcoreapp1.0 
    --configuration <Debug|Release> 
    --runtime <win7-x64|osx.10.11-x64>

capture

.NET Command Line Tools (1.0.0-preview1-preview1-002697)

Product Information:
 Version:     1.0.0-preview1-preview1-002697
 Commit Sha:  08b3ad9e88

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.10586
 OS Platform: Windows
 RID:         win10-x64

guardrex on 8 May 2016

Can you check your registry for old dotnet keys and your dotnet folder in Programs and see if you have old SDK's in there?

[EDIT] You might be hitting this: https://github.com/dotnet/cli/issues/2948

[DOUBLE-EDIT :smile: ] ... It can wreak HAVOC on build/publish with all kinds of strange results. :alien:

guardrex on 8 May 2016

@GuardRex will do, currently afk. I'd like to know what's in the osx.10.11-x64 folders in your screenshots.. if it is working, there shouldn't be an .exe file.

dasMulli on 8 May 2016

oooops!

[EDIT] So much for "good outputs" :smile: I'm going back to the actual test self-contained proj now to repro this there, since the project.json in that one is a bit different.

capture

guardrex on 8 May 2016

👍1

Good ... at least its a stable bug ...

capture

guardrex on 8 May 2016

👍1

This issue is essentially a dupe of dotnet/sdk#5631, though I appreciate the thorough investigation and want to be sure @krwq confirms this full scenario is covered before he closes this as a duplicate.

We haven't fixed this yet because the issue is a bit tightly tied to tooling experiences which we don't want to destabilize before getting the next named release locked down. Before cross-publish and cross-build were options, CLI would copy its own host binaries into the build output to facilitate running without publishing. We already enabled cross-publish, which was a new feature and therefore moved to the correct long-term behavior of getting the host out of its resolved nuget package. However, for Build we did not replace the behavior to facilitate short-term stability. This will come soon after we branch.

TheRealPiotrP on 8 May 2016

👎1

@krwq please confirm that your fix for dotnet/sdk#5631 covers this issue fully and then close this as a duplicate. If there are gaps, let's track them in this issue.

TheRealPiotrP on 8 May 2016

Duplicate of dotnet/sdk#5631