Runtime: Question: how does targeting netstandard versus netcoreapp work?

[polarapfel]

Hi,

I'm not sure I'm asking in the right place.

I have a question regarding targeting. I created new console app using dotnet new console.

Now, the project file looks like this:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp1.1</TargetFramework>
  </PropertyGroup>
</Project>

Can I target netstandard1.6 instead? Will it still run?

Changing the project file to:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netstandard1.6</TargetFramework>
  </PropertyGroup>
</Project>

This builds using dotnet build. It doesn't run however.

Unable to run your project.
Please ensure you have a runnable project type and ensure 'dotnet run' supports this project.
The current OutputType is 'Exe'.

Is targeting .NET Standard only for libraries, not for executable binaries? I was hoping I could target a standard instead of any runtime and then choose a runtime of my liking to run that binary, as long as that runtime supports that standard version. Like choosing to either point dotnet or mono against a binary that targets netstandard. Is this even possible?

And speaking of OutputType Exe, even in the original console project template, the build drops a dll, not an exe. Why is that?

thanks,

Tobias W.

[davidsh]

This might be helpful as it talks about NETStandard:
https://github.com/dotnet/standard/blob/master/docs/faq.md

cc; @ericstj

[polarapfel]

Hi @davidsh,

thanks for the link. I've been through that already, it doesn't help me understand as to my specific question.

I understand how this affects when building a _library_ and targeting netstandard there. So far so good. A library targeting netstandard can be referenced from its assembly across .NET solutions targeting different runtimes as long as all of them support the same API surface described by the standard.

But I don't understand if targeting netstandard for an _executable_ project type is supported or not. It doesn't seem to be. But what good is a standard across runtimes, when I still have to create various project templates against each runtime supporting the same standard and API surface to begin with? Why can't I just have one executable project definition, targeting a standard and any runtime with a standard compliant framework behind it will then run it?

By the way, the particular issue with how dotnet treats a project targeting netstandard seems to be related to how it's packaged and how the CLR handles the binary. Pointing dotnet to the dll renders:

A fatal error was encountered. The library 'libhostpolicy.so' required to execute the application was not found in '/mnt/c/Users/tobiaw/Source/Repos/AzTools/AzTools/azcd/bin/Debug/netstandard1.6'.

The same executes just fine when pointing the latest mono runtime at the same binary.

--

In essence, I was hoping .NET Standard would deliver the promise that Java made its developers a long time ago: Write once, run anywhere. Any Java bytecode compiled by any compliant runtime runs on any other Java runtime as long as they support the same Java version.

So really, only non-executable libraries benefit from .NET Standard, executables don't. Am I wrong here?

[davidsh]

@ericstj and @weshaggard are the experts here...so it would be best for them to help answer your questions.

[ViktorHofer]

When you compile a .NET Standard library it will build against netstandard.dll which is only a reference assembly and does nothing more than TypeForwarding to the actual implementation inside netfx, netcoreapp or mono. Therefore a .NET Standard Library can't be an executable file because the actual implementation is missing.

@terrajobst: Please correct me if I'm wrong.

[danmosemsft]

Is the exe part the confusion? See https://github.com/dotnet/cli/issues/6237

[polarapfel]

@danmosemsft Thanks for linking that, that was part of the confusion, though not the critical part. Seems like the "--self-contained" option is in 2.0, not 1.1 yet. Thanks!

[polarapfel]

@ViktorHofer If that's the case, mono shouldn't be able to execute the project after compiling it against netstandard1.6 instead of netcoreapp1.1, right? That works however.

My underlying misconception evolves around:

What's the best way to build an executable binary that is runtime agnostic? My thought was: targeting netstandard with the assumption that any runtime/framework configuration compliant with that netstandard version will be able to run that binary. Doesn't seem to be the case.

Specifically, what's currently the best way to work with the dotnet toolchain when I want to remain within the netstandard 1.6.1 API surface but distribute and run resulting binaries on both netcore and mono?

[ViktorHofer]

Mono can execute a library which targets and builds against netstandard1.6? That's seems weird to me. A .NET Standard Library doesn't have an application entry point (Program.Main) so what would it execute? I'm really curious about that.

What's the best way to build an executable binary that is runtime agnostic?

Framework-dependent deployment (FDD). _You do not have to define the target operating systems that your .NET Core app will run on in advance. Because .NET Core uses a common PE file format for executables and libraries regardless of operating system, .NET Core can execute your app regardless of the underlying operating system. For more information on the PE file format, see .NET Assembly File Format._

From here: https://docs.microsoft.com/en-us/dotnet/articles/core/deploying/

Then you can execute your application with "dotnet [path to app dll]". Please don't get confused that it does not contain an ".exe" file, the "[appname].dll" is still executable.

[polarapfel]

Hi @ViktorHofer,

Create a new console app: dotnet new console

This template has a single C# file with a main that prints Hello World to the console.

Change the TargetFramework from netcoreapp1.1 to netstandard1.6. Restore packages and build.

The build works without issues.

This binary won't run with dotnet however.

Point mono to the dll and you'll see main executing, outputting Hello World to the console.

I ran a hex diff against the dll when built against netstandard1.6 and netcoreapp1.1. There is almost no difference in the binaries, byte by byte. Apart from around ~100 bytes, the files are identical and have the same size. Main certainly is in both. Just when running dotnet against the netstandard binary, it complains there's no hostpolicy assembly.

[ericstj]

Portable (netstandard) EXEs is something we've talked about in the past but never really scoped in. /cc @blackdwarf @terrajobst @davidfowl @Petermarcu

Currently the .NETCore host has some requirements that should probably have better defaults to enable hacking this scenario. There are also some roadblocks in the SDK like you mentioned.

Here's what I tried:

dotnet new console
<change to netstandard2.0>
dotnet build
dotnet bin\Debug\netstandard2.0\myapp.dll

That gave me:

A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application was not found in 'F:\nsapp\bin\Debug\netstandard2.0\'.

I suspect there are some arguments you can specify to make this work. @gkhanna79 would know better. If you want to find out for yourself you can also look at https://github.com/dotnet/core-setup/blob/3089d7a55442551e53d13a422156ae0a458e9c64/src/corehost/cli/fxr/fx_muxer.cpp#L868. Also enabling the host tracing is interesting to see what its doing (set COREHOST_TRACE=true).

I suspect it has trouble if its missing a runtimeconfig and/or deps file. If its important to enable, file an issue in https://github.com/dotnet/core-setup. That could put it on par with Mono and desktop. In both of those frameworks you can activate a exe (potentially through a host) and it'll try to run the IL using assemblies next to the EXE and some central location (GAC/Framework directory). Once everyone supports ns2.0 this will provide a good amount of surface area for a portable exe that just works without dependencies.

The problem comes if you have any dependencies. Suppose one of those dependencies needs to be different per framework, or per OS. There isn't any agreed upon convention to represent application binaries in that way across frameworks. If we really wanted to make this work we'd have to get all .NET runtimes to agree upon how to locate framework and runtime specific assets at runtime. .NET Core does this already for runtime specific assemblies: that's what its using the deps file for, but not for framework specific assemblies. One could conceivably implement and assembly load scheme at the application layer that would make this work, but those sorts of things always sound good and simple in principal and get hairy when you try to actually build them. To do this right and complete we'd probably want to do it at the framework level.

[polarapfel]

@ericstj

There isn't any agreed upon convention to represent application binaries in that way across frameworks.

That's the key point indeed. In my humble opinion, the usefulness of .NET Standard being applicable not just to a single, non-executable library assembly but also to the ability to package executable applications against a standard would really help though, so extending the standard by such a convention might not be a bad idea. It would also put .NET on par with how Java solved this problem (well, they've never been as fragmented as the .NET platform to begin with...).

[gkhanna79]

Hostpolicy.dll is expected to be in the app folder when it is determined that the application is being built is standalone. Is that not the case (for standalone apps, runtimeconfig.json also does not carry FX details).

CC @ramarag who owns the host.

[andycmaj]

interesting case i ran into RE netstandard vs netcoreapp.

here's a code snippet:

        public override HttpMessageHandler CreateMessageHandler()
        {
            if (config.MarketingApiDisableSslCertValidation)
            {
                return new HttpClientHandler
                {
                    ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
                };
            }

            return base.CreateMessageHandler();
        }

when the csproj has <TargetFramework>netstandard2.0</TargetFramework>, this fails.

Common/MarketingApiFlurlClient.cs(70,83): error CS0117: 'HttpClientHandler' does not contain a definition for 'DangerousAcceptAnyServerCertificateValidator' [/Workspace/ims/TheChunnel/activities/TheChunnel.Activities.Marketing/TheChunnel.Activities.Marketing.csproj]

i noticed the same snippet was working in a netcoreapp2.0 project so i changed this project to netcoreapp2.0 and viola, dotnet can find the correct API i was referencing.

when i look at the assembly HttpClientHandler is coming from in the 2 cases (Left: netcoreapp2.0 project, Right: netstandard2.0 project), here's what i see:

in the netcoreapp2.0 case, it's using Assembly System.Net.Http, Version=4.2.0.0 and in the netstandard2.0 case, Assembly netstandard, Version=2.0.0.0.

i know i am probably doing something wrong here, but i don't get why netstandard has a smaller set of reference APIs than netcoreapp2.0. i thought the difference was in "library" project vs "executable application".

Can someone explain this difference? I think it's relevant to this topic.

[davidsh]

cc: @karelz

[karelz]

There are plenty of articles and explanations on internet - in nutshell standard is a standard (like HTML is a standard) - it is the common API surface for 4 platforms (think browsers as implementations of HTML standard): .NET Core, .NET Framework, UWP and Mono. Each platform can have more APIs than just the standard - like in your case HttpClientHandler.DangerousAcceptAnyServerCertificateValidator.

To learn more, search internet for ".NET Standard vs. .NET Core" - you can start with .NET blog post on the topic.

[andycmaj]

FWIW, @karelz, most of the articles i've tried to read on netstandard are very theoretical with lots of historical context about WHY netstandard (and often a bit dated with respect to the actual latest dotnet versions, and not much practical advice/concrete examples like the one i posted above.

https://dotnetcore.gaprogman.com/2016/11/24/net-standard-what-it-is-and-how-it-applies-to-net-core/

https://dotnetcoretutorials.com/2017/01/13/net-standard-vs-net-core-whats-difference/

https://gist.github.com/davidfowl/8939f305567e1755412d6dc0b8baf1b7

https://stackoverflow.com/questions/36962930/whats-the-difference-between-the-new-netstandardapp-and-netcoreapp-tfms

After i noticed this difference, it made more sense to me on an intuitive level what the differences are here...

I also remember articles back in v1.x that recommended that executables target netcoreapp and libraries target netstandard.

This may still be true, but it isn't the full story. in my example, i have a bunch of "library" projects that are really just ways to structure my application for the purposes of organization, dependency management, service boundaries, etc. In my case, i still NEED to make my "libraries" target netcoreapp unless i want to refactor my application. the old "you should re-examine your design" argument. Which may be true, but it wasn't really clear WHY until i found an API that isn't part of the Standard, but is part of a platform.

All i'm really saying here is that the documentation available is not super helpful. official articles like https://docs.microsoft.com/en-us/dotnet/core/packages (which is helpful, but again a bit out of date) aren't actually even in top google search results for "netstandard vs netcoreapp", and the fact that there are so many SO questions/blog posts trying to explain this might hint at a lack of official, practical documentation/examples here. Might just be a case of code outpacing docs, but that's not great.

[Petermarcu]

@andycmaj thanks for the feedback.

@terrajobst , whats the best place to track this and the creation of a clear document about making this choice? In its simplest form, .NET Standard is what you should use if you want your library to be shareable across the maximum set of .NET Platforms. If you are intending to publish a general purpose reusable library to NuGet, for example, ideally you can target .NET Standard so people can use it in the most places.

If you are only targeting a single platform and just using libraries as architectural or implementation details of your application or just sharing between multiple applications that target the same .NET platform, there isn't much value in targeting .NET Standard other than "future proofing" in case you ever do want to move to another .NET Platform.

[mika76]

_(I posted an SO question here and a comment directed me to this issue)_

I think the distinction that .net standard is only for class libraries and not executables (at least not yet re comment from @ericstj) should really be more distinct in the documentation and FAQ.

Looking at the .net standard docs and the faq there really seems to be no mention of it, and it's not nearly as logical as people seem to think.

If we look at the FAQ https://github.com/dotnet/standard/blob/master/docs/faq.md#what-is-net-standard there is no mention this is only for class libraries.

The only mention I see now is in the blog post at https://blogs.msdn.microsoft.com/dotnet/2016/09/26/introducing-net-standard/ in the "Why do we need a standard?" section:

Applications. In the context of applications you don’t use .NET Standard directly. However, you still benefit indirectly. First of all, .NET Standard makes sure that all .NET platforms share the same API shape for the base class library. Once you learn how to use it in your desktop application you know how to use it in your mobile application or your cloud service. Secondly, with .NET Standard most class libraries will become available everywhere, which means the consistency at the base layer will also apply to the larger .NET library ecosystem.

But reading though it seems pretty ambiguous to me. It does not mention that executables can not be run as netstandard, but applications which could mean a whole lot more conceptually than executables. I only now understand what it means having some context.

[jnm2]

Executables and tests, at least until we finally get .NET Standard tests: https://github.com/dotnet/sdk/issues/1561

[michkot]

I digged for ~4hrs via google into several SO issues and various dotnet/* github issues and MS.com docs, to find out more about this, what I found out:

• .csproj with OutputType Exe and TFM netstandard produces a binary (with .dll suffix, but that's nothing new in the portable .NET world)
• Mono really seems to run these somehow (how well?) , here at https://github.com/dotnet/runtime/issues/21042#issuecomment-293394403 and also there https://github.com/dotnet/standard/issues/1689#issuecomment-579550494
• dotnet (at least on Windows) is also happy tu run them, if you supply *.runtimeconfig.json with reasonable .NETCore.App framework specified (for example, target netstandard2.0 and run with netcoreapp2.0 or netcoreapp3.0)

I personally have a .netstandard2.0 app, with .net472 lib and .netstandard2.0 lib, that works fine run via .netcoreapp3.0. For .netcoreapps2.1<= it fails on IsReadOnlyAttritbute not found in mscorlib. If I build the .net472 lib against .netstandard2.0 too, the app works even on .netcoreapp2.0.

I assume this works, because of the proclaimed binary compatibility of .NET Standard 2+, ?
Is this a "fluke" a working "not supported scenario" or something that is going to be (officially) documented and supported?
I also stumbled upon a notion of .NET FW binaries beeing supported via some kind of shim as .NET Standard (or Core, don't remember) app. dependencies.
Is there a (cross-platform) tool, that will look into the .dll and tell me what was the target FW / TFM used for compilation?

Note: overall, the biggest problem for me to get my head wrapped around .NET Standard + .NET Core 2+ as a .NET FW dev, is clustered info and a lot of Q/A for older version of .NET Standard and .NET Core in search results everywhere. I hope that ".NET 5.0" will rectify those problems in a way.

[reflectronic]

Some of this is related to https://github.com/dotnet/runtime/issues/34852, which asks for a portable executable binary format

[cfbao]

I assume this works, because of the proclaimed binary compatibility of .NET Standard 2+, ?
Is this a "fluke" a working "not supported scenario" or something that is going to be (officially) documented and supported?

I too found out about .netstandard app, and am wondering a similar thing - why _isn't_ this officially documented and supported? The only thing I can think of is, because of the launching problem, officially documenting/supporting building .NET Standard apps will generate too many questions, confusion, and frustration from users/developers, that they decided it's not worth it.

In my (admitted not very deep) understanding of CLI and .NET Standard, there's no reason why a .NET Standard app wouldn't work on any platform.
All platforms (.NET framework, Core, Mono etc.) must conform to the CLI spec and produce the same assembly format. The only platform specific details are

1. how dependent assemblies are referenced/discovered/loaded, and
2. how apps are launched

.NET Standard takes care of the first point. A pure .NET Standard app will only reference netstandard.dll and packaged third party libraries/assemblies (which also only reference netstandard.dll). This assembly netstandard.dll is present on all conforming platforms that implement the compatibility shim.

For the second point, users/developers have to provide the details.

• to run on .NET Core, an appropriate *.runtimeconfig.json file must be provided, and app should be launched by running dotnet <appname>.dll or using a custom app host.
• to run on .NET Framework (Windows only obviously), rename the assembly from <appname>.dll to <appname>.exe and Windows should take care of the rest
• to run on mono, run mono <appname>.dll.

I can see how these different launching options can cause confusion, and difficult to support in a seamless manner. There's also question like how should VS studio launch it when debugging? So it probably makes sense for .NET team to not officially support it, even though it may be technically sound.

I'd love to know if I'm missing any technical details that could make a .NET Standard app not work properly on any conforming platform.

[ericstj]

1. how dependent assemblies are referenced/discovered/loaded, and
2. how apps are launched

I believe you hit on two important missing parts.

NET Standard takes care of the first point.

Not entirely. .NETStandard only defines common surface area for netstandard.dll that is supported by all frameworks. API in this assembly is covered but nothing else. The NuGet ecosystem consists of many packages which build on top of this and require framework specific implementations. There is nothing that encodes these TFM-specific assemblies at runtime today and running a library on a specific framework while not honoring that library's best asset for that framework as defined by the package is a incorrect and breaks the contract defined by that package. Sure in many cases it "can work" but in general it does not. What's missing here is some runtime version of NuGet to make this work today.

how apps are launched

What you describe is something that appears to work but it omits application state. Consider that both .NETFramework and mono are ignoring the deps file which contains runtime-specific implementation details for some packages (those that cross-target by runtime). We're missing a config file for .NETFramework which means that most of the assemblies will fail to load due to .NETFramework's stricter binding rules. This also also omits other state like runtimeconfig details. There are other platforms which support .NETStandard but aren't even covered by the set of launching scenarios (consider UWP, Xamarin). So again this is something that "can work" but in general, does not.

For ".NETStandard" applications to be supported I think the standard would need to address these things: a common format for defining binding rules (or a more tolerant binder), a common format for defining assembly selection and defining the pivots, a common host/activation model, possibly more.

cc @terrajobst @jkotas