Runtime: Need a command line tool to check if .NET Core runtime dependency is met

[MSLukeWest]

Porting internal email thread with @vitek-karas @SvetBonev @swaroop-sridhar @AaronRobinsonMSFT @MSLukeWest @NikolaMilosavljevic @jeffschwMSFT to this issue. Latest replies first:

---

From @svetbonev

The goal is to make it really simple for customers to determine if the runtime version of .NET Core that they need is available on the machine as part of their installers.

Copy/paste from Luke’s original email:

Example Syntax: “DotNetChk.exe 3.1.0 3.1.4 Microsoft.WindowsDesktop.App X86”

Parameters:
1-2: A range of acceptable runtime versions to look for
3: The framework to look for
4: The architecture

Return values
1 if valid runtime is found
0 otherwise

The installers clarification above is important. There is a variety of installer technologies out there and they have different capabilities. Parsing text may not be possible for all of them. On the other hand, exit codes are standard.

The other consideration is that some installers run elevated. We want to make sure we don’t create conditions for elevation of privilege by calling dotnet.exe that we don’t know anything about.

A simple executable with no satellite files will be best.

Would it be possible to make the functionality available in dotnet.exe in such a way that it works without having to add extra configuration files?

Like, can we add the discovery code to dotnet.exe?

If this is not a good idea, can dotnet.exe be changed to try to load hostfxr.dll from the same folder instead of looking under the host\fxr sub-folder? That way we can ship only dotnet.exe+hostfxr.dll.

---

From @vitek-karas

Currently both approaches (dotnet.exe or programmatically) would need temporary files – we don’t have the APIs necessary to do this fully “in memory”.

There are potentially two ways to do this programmatically:

• Initialize everything as if it’s running an application (without actually running the app) – this has the advantage that everything will work just like if you try to run an app. The downside is that this requires not only the app.runtimeconfig.json and at least a fake app.dll, but also at least a barebone app.deps.json (I don’t know from the top of my head if this would work correctly without the .deps.json). This would handle self-contained apps as well. Basically if you have the actual app to run, you can point it to the app directly and it will tell you if it can be executed.
• Initialize the hosting as if load a component – this will do very similar things, but there are some small differences in behavior versus the app approach. One notable limitation is that this will not work for self-contained apps – you would have to detect that yourself and skip the check (which is appropriate, there’s no need to check for preinstalled dependencies if the app you’re trying to run is self-contained). This might work without any .deps.json (validation required I haven’t tried this specifically).

The high-level points of the programmatic solution would be:

• Use nethost library to search for hostfxr.dll – you can take a look at this guide (and the related sample) how that is done: https://docs.microsoft.com/en-us/dotnet/core/tutorials/netcore-hosting#create-a-host-using-nethosth-and-hostfxrh
• Either hostfxr_initialize_for_dotnet_command_line (the app-like approach from above) or hostfxr_initialize_for_runtime_config (the component approach from above).
• The result of that call will be the result of your application – if it succeeds, the given dependencies should be available on the machine.
• The above will load the appropriate hostpolicy.dll into the process if the framework resolution succeeds.
• Close the context via hostfxr_close – this should unload the hostpolicy.dll loaded above.
• Important note: with this approach only hostfxr.dll and hostpolicy.dll is loaded into the process and only hostfxr.dll will remain loaded after the attempt – and it should be possible to unload it manually. This will NOT load coreclr in any way.

Let us know if you we can help with anything.

---

From @MSLukeWest

Thanks Vitek. We do want to do this programmatically, and we only need .NET Core 3.1 and above. Can you elaborate on what that would look like? Would we still need to create a temporary .runtimeconfig.json file along with an app.dll?

Good note regarding needing a separate binary for x86 and x64. Given how small dotnet.exe is I don’t think it’s a problem that we’d need two of them.

---

From @vitek-karas

Actually you should not need hostfxr either. If it’s not on the machine then no .NET Core is available and as such you will get a nice error from dotnet.exe.

If all you want is to check if an app requiring a specific version would run, then that should do it. In that case you don’t even need to parse the output of dotnet.exe too much. Basically just create a temporary .runtimeconfig.json file with the right content another temporary app.dll and try to “run” it via dotnet.exe.

You could do this programmatically as well if you need only .NET Core 3.0 and above – in that case you would not need to spawn a separate process.

Either way doesn’t require hostfxr to come with you app – it would only use it if it’s available on the machine.

The downside of the programmatic solution is that it would have to be a separate binary for X86 and X64 (as the APIs are designed to run everything in-proc, and so can’t load the other architecture).

To answer Svet’s question: Framework/runtime discovery and resolution is done in hostfxr.dll.

---

From @SvetBonev

Is runtime discovery done in hostfxr.dll?

---

From @MSLukeWest

Thanks Swaroop and Aaron for the replies. Swaroop’s suggestion about just including dotnet.exe and hostfx.dll and parsing the output of dotnet --list-runtimes Is certainly interesting. I see that these two binaries combined are < 1mb. Swaroop – Can you confirm that if I just drop these binaries on any Windows desktop where .NET Core is supported (Win7 SP1+) they will run without any kind of framework dependencies?

---

From @swaroop-sridhar

I second @AaronRobinsonMSFT 's suggestion about opening a thread on github about this tool.

I’m curious why you want to build a separate tool for discovering the frameworks, rather than post-process the output of dotnet --list-runtimes.
The framework discovery is all native code, and you’d only need dotnet.exe and hostfxr.dll to get a list of all frameworks installed on the system.

We can also consider implementing the tool as a custom host that uses hostfxr but doesn’t initialize the runtime.
But I’m sure it has to live in the runtime repo.

---

From @AaronRobinsonMSFT

This goal sounds reasonable. I don’t know if we need a new tool or perhaps enrich an existing one. If this does go into the repo it sounds like it would be placed under https://github.com/dotnet/runtime/tree/master/src/installer/corehost. This code area is non-trivial and has incredibly high compat restrictions so heading into this just know there are going to be a lot of questions and scrutiny to add something like this to the repo.

The recommended way to get this new tool added will be through an official issue on the https://github.com/dotnet/runtime repo with the design goals, constraints, and intended test matrix. Vitek and Swaroop are the owners of this area and can review any design you have, but it needs to start on github. We are an open source repo and fully transparent about all asks and features. Getting some early feedback here is fine, but officially it needs to be on github so the community can also participate.

My experience in this area indicates this is a reasonable ask with a goal we should be able to achieve. I will say this does sounds like something ASP.NET needed at some point.

---

From @MSLukeWest

Swaroop / Aaron-

I’m planning on building a very simple command line tool in the dotnet runtime repo that checks whether or not a specific .NET Core runtime dependency is satisfied on a machine. You can read the attached mail for more details, but the main purpose of this is to run inside the bootstrapper of a ClickOnce/Installer project to determine if we need to install the runtime before installing an app with a .NET Core dependency. It would just perform a series of registry and folder checks, similar to what happens today when you run “dotnet --list-runtimes". A few more details:

• This would only need to be built for Windows x86 (for now at least)
• This will have to be written in native code since we can’t make any assumptions about what frameworks are installed on the machine
• I want to put it in the runtime repro since it may share some code with the dotnet cli tool, though it won’t actually ship with the runtime
• We don't need to consider the self contained app scenario here, we can assume by the time that this runs we know it's framework dependent

Example Syntax: “DotNetChk.exe 3.1.0 3.1.4 Microsoft.WindowsDesktop.App X86”
Parameters:
1-2: A range of acceptable runtime versions to look for
3: The framework to look for
4: The architecture

Return values
1 if valid runtime is found
0 otherwise

Nikola identified the two of you as potential contacts for this, I’m mainly just looking for help identifying where in the repo you’d recommend I place such a tool, as well as maybe an example project I can model mine after. Can one of you guys help point me in the right direction?

[msftbot[bot]]

Tagging subscribers to this area: @vitek-karas, @swaroop-sridhar
Notify danmosemsft if you want to be subscribed.

[MSLukeWest]

Forgot one person, @dleeapho

[swaroop-sridhar]

The installers clarification above is important. There is a variety of installer technologies out there and they have different capabilities. Parsing text may not be possible for all of them. On the other hand, exit codes are standard.

I think the proposal (for this approach) is to add wrapper tool around the dotnet --list-runtimes that does the text processing and provides a return value, so that individual installers don't have to worry about it.

Would it be possible to make the functionality available in dotnet.exe in such a way that it works without having to add extra configuration files? Like, can we add the discovery code to dotnet.exe?

No this is not a good idea. dotnet is designed to be a minimal-functionality executable that doesn't change much. Much of the host functionality is implemented in hostfxr and hostpolicy components.

A simple executable with no satellite files will be best.
Initialize everything as if it’s running an application (without actually running the app) – this has the advantage that everything will work just like if you try to run an app.

We could try running a dummy single-file app, but this doesn't resolve the issue about not exec-ing another process you've mentioned below.

The other consideration is that some installers run elevated. We want to make sure we don’t create conditions for elevation of privilege by calling dotnet.exe that we don’t know anything about.

If hostfxr exposes an entry-point to obtain a list of available frameworks, the tool can invoke it in-process.

[SvetBonev]

If hostfxr exposes an entry-point to obtain a list of available frameworks, the tool can invoke it in-process.

I like this idea as the tool will be able to control where hostfxr is loaded from.

Stretching it a bit - what about compiling that code directly into the tool rather than consuming it through a DLL?

[vitek-karas]

I think the proposal (for this approach) is to add wrapper tool around the dotnet --list-runtimes that does the text processing and provides a return value, so that individual installers don't have to worry about it.

My understanding of the ask here is to answer a question like "Will application requiring framework XYZ run on this machine?". Answering that question from just a list of installed frameworks is actually non-trivial. The entire machinery of framework resolution, version conflict resolution and roll-forward is on top of the flat list. That's a lot of logic. So I don't think that parsing the dotnet.exe output is the right approach here.

We could try running a dummy single-file app, but this doesn't resolve the issue about not exec-ing another process you've mentioned below.

I don't see how running dotnet.exe is different from running any other process - I also don't understand why spawning a new process is a problem. On the other hand I do agree that this should not run potentially "random" code on the machine as it's likely to run with elevated privileges.

Shipping with hostfx as oppose to using the one from the machine

On paper we don't guarantee hostfxr forward compatibility - that is older hostfxr resolving and potentially running newer framework. So for example on paper we don't guarantee that 3.0 hostfxr will work with 5.0 framework. So shipping the hostfxr with the app could potentially cause problems for apps which are for example 3.0, but have RollForward=Major and thus should easily run on a machine with just 5.0 on it. (In reality this will work, we try not to break this compatibility, but if there's a good reason we could).

Using hostfxr, dotnet.exe or any other part of .NET Core from the machine

I understand that we don't want to introduce a vector through which the likely elevated installer could run potentially untrusted code from the local machine. That would be bad. Running already globally installed .NET Core should be fine. Globally installed .NET Core is "trusted" (by trusted I mean a location which can only be written to by Admin-only or where the location can be changed by Admin-only) - that is it either lives in a trusted location (the default C:\Program Files\dotnet) or the location is specified in an registry key HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\<arch>\InstallLocation which is read-only for normal users. As such running code from the above locations from an elevated process should be OK as those locations are read-only for non-elevated processes.

With that in mind I think the best solution would be to:

• use nethost to locate hostfxr already installed on the machine. This is the algorithm which looks at the well known locations described above (spec here). As described above this should only look into trusted locations.
• generate a temporary .runtimeconfig.json or if possible use the one from the app being installed
• load that hostfxr and use the hostfxr_initialize_for_runtime_config (as that's the easier one) with that .runtimeconfig.json
• this will load hostpolicy.dll from the resolved framework (on success basically), but that also comes only from trusted location (next to hostfxr)
• if this call fails, it should report an error explaining why (and error code as well) - but basically it should fail if the required framework is missing.
• close it via hostfxr_close - this would unload potentially loaded hostpolicy.dll
• unload hostfxr.dll

We do ship nethost as a static lib, so there's no need for any additional files.

Note that nethost (and later on hostfxr and hostpolicy) behavior can be influenced via environment variables which could potentially make them look into other location on the local machine. But that should be OK from security standpoint. If the installer process is running as elevated it follows all the rules of not getting env. variables from untrusted sources....

[rseanhall]

close it via hostfxr_close - this would unload potentially loaded hostpolicy.dll

The hostfxr code today does not unload hostpolicy.dll under any circumstances, there's even a comment saying that it is intentionally not unloading it. If it weren't for the comment, I would have created an issue. For this exact scenario (using initialize_* apis from hostfxr as a way to tell whether the application's framework is available), the fact that hostpolicy.dll stays loaded (and initialized - there's no way to reinitialize) makes it problematic.

[vitek-karas]

You're right - unloading was the original intent, but implementation happened (and I forgot).
I think that for cases where we don't load the CLR we still might be able to unload - the problem will be the locking/initialization sequence, but it doesn't sound impossible.

Curious - why do you think it's problematic for this scenario?

• I would not use this 100% in-proc approach with an installer which at some point in time actually loads CoreCLR - it might work, but for example nethost would have problems in that scenario (it detects already loaded hostfxr in the process and uses it if available).
• Would you need to actually run two such checks in a row? Currently without unload this is obviously not workable, but I think the unload can be achieved - in the original implementation we just didn't have a need for it really

[rseanhall]

I don't have a scenario where I need two checks in a row. But if this is the recommended way to solve this problem, I wouldn't be surprised if someone wants this someday. The current approach of this area of the code seems to be to only code for scenarios you can come up with today, and that's very frustrating when you run into things like this where the implementation takes the easy way out and it's near impossible to undo in the future.

[MSLukeWest]

@vitek-karas Thank you for outlining a possible solution, I'm giving it a try right now, using NetHost.lib to call get_hostfxr_path in order to locate hostfxr.dll. The first thing I noticed is that this will fail unless I also copy nethost.dll next to my app, with the error "The code execution cannot proceed because nethost.dll was not found. Reinstalling the program may fix this problem. ". Did you expect that this would be a requirement?

[AaronRobinsonMSFT]

@MSLukeWest It would appear you are linking against nethost.lib instead of libnethost.lib. The latter represents a complete static library where as the former is merely an import lib.

[MSLukeWest]

@AaronRobinsonMSFT I'm linking against both and I get that same error. I also get it when I link with just nethost.lib. If I do just libnethost.lib I get unresolved external symbol errors.

Right now I'm using the versions from these directories:

• Program Filesdotnet\packs\Microsoft.NETCore.App.Host.win-x86\5.0.0-preview.3.20214.6runtimes\win-x86\native
• Program Filesdotnet\packs\Microsoft.NETCore.App.Host.win-x64\5.0.0-preview.3.20214.6runtimes\win-x64\native

Should I be looking somewhere else?

[AaronRobinsonMSFT]

@MSLukeWest Sigh... No, you are doing the right thing. You should be linking against libnethost.lib only and the symbol errors are a bug I recently discovered and fixed - https://github.com/dotnet/runtime/pull/35431. If you update to the nightly version of .NET 5.0 this should no longer be an issue.

[MSLukeWest]

I was able to get this working by upgrading to the nightly 5.0 build as Aaron suggested along with defining NETHOST_USE_AS_STATIC. I no longer require the dll.

I have a prototype that follows Vitek's suggested steps above and it seems to work well so far. I'll continue to test and refine this code and plan on sharing something soon.

[MSLukeWest]

Now that I have a prototype working I'd like to start the process of getting this checked in. To that end, please review the following design and reply back with any concerns.

Design Goal

Create a simple command line tool that can be used to determine if the necessary runtime to run a .NET Core app that requires a specific framework name/version combination is present on a Windows desktop machine. This will be used by various deployment technologies to determine if a machine meets the requirements for installing an app. The first consumer will be the Bootstrapper Packages used by ClickOnce and Visual Studio Installer Projects.

Requirements

1. We can't make any assumptions about what frameworks are installed on the machine, so the tool must be native
2. There will need to be two versions of the tool, one for Win-x86 and one for Win-x64
3. Scenarios like self contained apps are out of scope, we can assume that if the tool is running we already know what framework is needed.
4. Size is a concern here since this tool will be included in customer's installers. We should aim for < 1MB (current prototype is ~300kb)
5. Support .NET Core 3.1+

Usage

Arguments:

1. Required framework name (Examples: "Microsoft.WindowsDesktop.App", "Microsoft.NETCore.App")
2. Required major/minor/build framework version (Examples: "3.1.4", "5.0.0")

Return Values:

• Success: 0, Means the required runtime is installed on the machine
• Failure:
  1 - Failed to load HostFxr, meaning no runtime is installed
  2 - Failed to initialize HostFxr for the config, meaning the required runtime isn't installed
  3- Invalid Args
  Other - Unexpected failures

Examples:

1. NetCoreChk.exe Microsoft.WindowsDesktop.App 3.1.4
2. NetCoreChk.exe Microsoft.NETCore.App 5.0.0

Dependencies

• hostfxr.h
• libnethost.lib, which is included in the runtime

Implementation Details

The app will do the following:

1. Argument Validation
2. Attempt to locate hostfxr.dll using get_hostfxr_path, failure here results in return code 1
3. Generate temporary runtimeconfig.json file based on the passed in arguments, example: "{ "runtimeOptions": { "framework": { "name": "Microsoft.NETCore.App", "version": "3.1.4" } } }"
4. Load library hostfxr.dll and call hostfxr_initialize_for_runtime_config using the generated runtimeconfig.json file, success here means the necessary runtime is installed, failure means it isn't and leads to return code 2
5. Call hostfxr_close and unload hostfxr library

Testing

Along with unit testing we need to do pairwise testing for the following variations:

• All Windows OS versions that support .NET Core 3.1
• x86/x64 machines
• Required runtime version is installed, but for a different framework
• Required runtime framework is installed, but for a different version
• Verify failure in case where only the x86 version of a required runtime is installed but we're running the x64 version, and vice versa
• Correct handling of invalid versions and framework names

Open Issues:

1. Where should this code live? - @AaronRobinsonMSFT suggested above that src/installer/corehost seems like a logical spot, but I wonder if that still makes sense since this tool is Windows only and most of the code there is cross plat. If someone could point me to an example proj/cmake file to follow that would be great.

[AaronRobinsonMSFT]

There will need to be two versions of the tool, one for Win-x86 and one for Win-x64

Dependencies
hostfxr.h
libnethost.lib, which is included in the runtime

Given that the only dependencies are official public APIs and this tool would be exclusively targeted toward Windows (on a subset of architectures), it might not make sense to add it to the official repo. If we can fold the feature into an existing tool, I think that would be appropriate, but the current tool seems very targeted to your teams specific requirements and not a general purpose cross-platform application that is typical of the dotnet/runtime repo.

Basically I am suggesting this tool be added to the repo that needs the feature since it doesn't need any private hosting support so it may not be appropriate to be added here.

@jkotas, @vitek-karas, and @jeffschwMSFT for other thoughts.

[jkotas]

I agree with @AaronRobinsonMSFT conclusion that this does not seem like a good fit for dotnet/runtime. If the installer tech you are building needs one-off tool like this, the tool can build as part of the installer tech.

[jkotas]

Size is a concern here since this tool will be included in customer's installers. We should aim for < 1MB (current prototype is ~300kb)

Why was the small dummy app option discussed above rejected? The overhead of the small dummy managed app (without host) would be <10kB.

[vitek-karas]

Why was the small dummy app option discussed above rejected? The overhead of the small dummy managed app (without host) would be <10kB.

Just pure managed app has the problem that it needs to be executed via dotnet.exe - but how is that located? Technically it's incorrect to rely on %PATH% to find dotnet.exe as that can be changed and ultimately is not the technique used by host anyway.

What would work is a dummy application with host - but if the requirements are specific around command line and so on, there would still have to be a small "test app" which implements the command line parsing, creates the dummy app and tries to run it. So the above ends up being probably smaller and simpler.

[MSLukeWest]

We've created the tool, see sources here: https://github.com/dotnet/deployment-tools/tree/master/src/clickonce/native/projects/NetCoreCheck

[AaronRobinsonMSFT]

Thanks @MSLukeWest. Looks great. We are going to close this issue then.