Home: [RFC] Runtime Identifiers are not scalable

[ericstj]

I'm opening this issue here rather than https://github.com/dotnet/core-setup (host, platform abstractions), https://github.com/dotnet/corefx (RID-graph), or https://github.com/dotnet/sdk (runtime deps generation). The issue impacts all of these, including NuGet which was the source of the RID behavior as an opaque string.

The problems are as follows:

1. Existing supported OS releases a new version resulting in a new calculated RID and this RID cannot be used for restore/runtime-selection due to missing compatibility mappings until we update the RID-graph.
2. New linux distro comes along and should be treated as Linux-arch but is not.

This stems from the definition of RIDs being opaque strings and the only source of relationships being runtime.json / nuget.

Definitions:

• OS-name: specific name of an operating system or distro excluding version, eg: osx, ubuntu, win
• OS-version: specific version of an operating system or distro, eg: 10, 10.13, 7.4
• OS: a combination of OS-name and OS-version to represent a specific release of an operating system or distro.
• OS-family: a group of OS-name for the purpose of representing compatible behavior. eg: linux, unix

See also: https://github.com/dotnet/corefx/blob/master/pkg/Microsoft.NETCore.Platforms/readme.md

The behavior I'd like to see is as follows:

1. For an OS that's already defined an OS-name relationship in the RID-graph, that relationship should hold for future OS-versions that aren't present in the RID graph. EG: osx.10.20-x64 should be considered compatible with osx-x64 (and all of its mappings in turn). If a RID is present in the graph explicitly this inference would not need to happen.
2. For an OS that's already defined an OS-version relationship in the RID-graph, that relationship should hold for future versions that aren't present in the RID graph. EG: osx.10.20-x64 should be considered compatible with osx.10.11-x64 (and all of its mappings in turn). If a RID is present in the graph explicitly this inference would not need to happen.
3. For an OS-name that does not appear in the RID-graph that OS-name could be treated as part of some default OS-family (potentially defined by the project).

I can imagine solutions to this that have changes in just the host and tooling, as well as changes that may require a NuGet update as well. I wanted to get a discussion going to understand if its something we can fix.

/cc @petermarcu @davidfowl @anurse @emgarten @eerhardt @steveharter

[eerhardt]

What's the plan with this issue? Is this something that can be addressed in the .NET Core 2.1 time frame?

[tmds]

I think source-build is responsible for updating the runtime.json so it includes the target rid (and possible parents).
If the rid isn't in the corefx runtime.json, other dotnet sdks won't be able to target it, but it should be fully functional on its own.

I know source-build has had a task for updating the runtime.json file, but I'm not sure what the current behavior is. When trying to build 2.0 for fedora.27, the build failed because the rid was unknown.
This relates to: https://github.com/dotnet/source-build/issues/297.

[eerhardt]

We hit this issue again with alpine 3.8, since it isn't in the RID graph, but it is the distro that is used in our 2.2 docker images. See https://github.com/dotnet/corefx/issues/34316.

[rrelyea]

@eerhardt - talking with @dsplaisted, @nguerrera and @livarcocc about RIDs recently.
The goal is to not need runtime.json in netcore 3 scenarios.

I'm unsure how that recent discussion intersects with the inadequacy mentioned in the RID system this issue by @ericstj

[bording]

@rrelyea Is there any place I can find more information about the plans for .NET Core 3 not needing runtime.json?

I'm curious about this because it seems like even if .NET Core 3 itself isn't relying on it, there still could be a problem for packages that need to include RID-specific assets if NuGet still needs the RID graph to determine which assets to use.

[ericstj]

@rrelyea that's not entirely true.

We don't need runtime.json to define RID-specific package associations for the framework package, since that is going to be lifted into the toolchain with the runtime-packs feature: https://github.com/dotnet/designs-microsoft/pull/38.

We still need runtime.json to define the RID graph for RID-specific asset selection from a single package. In a shared framework app this would be important for defining the RID fallback chain that gets used by a host (granted the host could change how it determines this fallback chain). For a self-contained app NuGet still uses this for selecting the "best" runtime-specific assets from a package.

[nguerrera]

Public link for .net core 3 targeting packs and runtime packs: dotnet/designs#50.

Yes, we still need a rid graph. We have been exploring two things:

1. Can the graph be supplied to NuGet without coming from a package: https://github.com/NuGet/Home/issues/7351
2. Can we defer the selection of rid specific assets from packages to after restore: https://github.com/dotnet/cli/issues/10528

[dsplaisted]

We will still need a runtime graph (currently defined via runtime.json) for .NET Core 3. However, instead of coming in via a a package reference, we plan to bundle it in the SDK. We are also considering handling RID selection in the SDK instead of NuGet.

These changes for .NET Core 3 won't directly help address the RID graph complexity, but they may help a bit with supporting new RIDs (as they can be delivered in an updated SDK).

[tmds]

Question: the runtime itself is also aware of the rid graph, right? If not, how does a framework-dependent application pick managed&native libraries which are rid specific?

[dsplaisted]

@tmds I believe the runtime has a "RID linked list", ie the subset of the RID graph that is compatible with that runtime, which allows it to pick the best compatible RID from available assets for framework-dependent apps.

[tmds]

Do you know where the list is stored? And what code is using that list to pick the libraries?
Does mono support this too?

[eerhardt]

Do you know where the list is stored?

It is stored in the shared framework's .deps.json file. dotnet\shared\Microsoft.NETCore.App\3.0.0-preview-27122-01\Microsoft.NETCore.App.deps.json. See the "runtimes" section at the bottom.

And what code is using that list to pick the libraries?

It is contained in the hostpolicy.dll|so|dylib. https://github.com/dotnet/core-setup/search?utf8=%E2%9C%93&q=rid_fallback_graph_t&type=

NOTE: It is also exposed through the DependencyModel library. You just need to ensure you are parsing the shared framework's .deps.json file. Here's an example of the CLI reading this info in managed code

Does mono support this too?

I don't believe so. But I could be wrong.

[tmds]

Thanks for the explanation @eerhardt

[am11]

1. For an OS that's already defined an OS-version relationship in the RID-graph, that relationship should hold for future versions that aren't present in the RID graph. EG: osx.10.20-x64 should be considered compatible with osx.10.11-x64 (and all of its mappings in turn). If a RID is present in the graph explicitly this inference would not need to happen.

Would this have side effects? Say in 10.20, Apple has a breaking change in AppleCrypto/SecureTransport API behavior which impacts downstream API in System.Security.Cryptography. In this case, today the user gets package restore error and in future will receive a runtime exception. It might as well be an acceptable trade-off.

[nguerrera]

@vitek-karas

[eerhardt]

I wonder if we could somehow add a "hybrid" approach here to make progress where it really counts - our linux RIDs. (Windows and macOS RIDs don't really have this problem in the current world.)

Looking at the proposal for the next manylinux spec in Python- https://discuss.python.org/t/the-next-manylinux-specification/1043, the Option 2: A ‘perennial’ manylinux section seems to have a pretty decent structure that we could follow.

manylinux_${libc provider}_${libc version}_${arch}

Examples:

• manylinux_glibc_2_12_x86_64
• manylinux_musl_1_1_x86_64

We could have a hybrid RID approach where we use the current RID fallback mechanisms in place today. And if no assets are found, then fall back into the manylinux_ RID checks where we check for assets for the current libc_provider, libc_version, arch. The important part here is that the manylinux_ set of RIDs is explicitly NOT an opaque string. People can format and parse this RID using the above structure. And even more important is the libc_version is a "greater than or equal" clause, meaning "this asset will work on all versions greater than or equal to this version of this libc provider."

If, in the future, we needed to support other many{OS Family}_ RIDs, we could add them. But today we could solve this with linux, which would solve it for the majority of the places we currently have issues.

[tmds]

We have hit this issue here: https://github.com/dotnet/source-build/issues/1083
Fedora 30 isn't in the graph, so it doesn't derive from fedora-x64, but falls back to the generic linux-x64.

This is similar to the issue here: https://github.com/dotnet/corefx/issues/34316. Alpine 3.8 not in the graph, so it doesn't derive from linux-musl-x64, but falls back to the generic linux-x64.

It would be nice if we can improve the situation for known distributions that have a newer version.

CC @omajid

[rrelyea]

@nguerrera - the recent work for netcore 3 to provide the ability to have it provide the runtime.json -- does that help things?

[nguerrera]

@rrelyea Not really, we still have to update the graph too much. I think where the json file comes from is orthogonal to not having to spell out all the versions, etc.

[tmds]

I had a look at what happens to runtime.json for Linux platforms if you leave out the implicit relationship that comes from the rid name: e.g. ubuntu.18.04[-x64], imports ubuntu.18[-x64], ubuntu[-x64].

Derive from linux/linux-musl:

debian: linux
fedora: linux
gentoo: linux
opensuse: linux
rhel: linux
sles: linux
alpine: linux-musl

Mapping between distros:

ol: rhel
ol.7: rhel.7
ol.8: rhel.8
linuxmint.17: ubuntu.14.04
linuxmint.18: ubuntu.16.04
linuxmint.19: ubuntu.18.04
ubuntu: debian

Express compatibility between versions:

alpine.3.10: alpine.3.9
alpine.3.11: alpine.3.10
alpine.3.7: alpine.3.6
alpine.3.8: alpine.3.7
alpine.3.9: alpine.3.8

rhel.7.1: rhel.7.0
rhel.7.2: rhel.7.1
rhel.7.3: rhel.7.2
rhel.7.4: rhel.7.3
rhel.7.5: rhel.7.4
rhel.7.6: rhel.7.5
rhel.8.1: rhel.8.0

ol.7.1: ol.7.0, rhel.7.1
ol.7.2: ol.7.1, rhel.7.2
ol.7.3: ol.7.2, rhel.7.3
ol.7.4: ol.7.3, rhel.7.4
ol.7.5: ol.7.4, rhel.7.5
ol.7.6: ol.7.5, rhel.7.6

sles.12.2: sles.12.1
sles.12.3: sles.12.2
sles.12.4: sles.12.3
sles.15: sles.12.4

linuxmint.17.2: linuxmint.17.1
linuxmint.17.3: linuxmint.17.2

linuxmint.18.2: linuxmint.18.1
linuxmint.18.3: linuxmint.18.2

linuxmint.19.2: linuxmint.19.1

This has rids which shouldn't be used. If the distro has binary compatibility between major versions because it's meant for in-place upgrades, the base rid should be used (rhel.7 instead of rhel.7.1).
I think this is the case for all these distros except alpine.

For Alpine, you need the minors (e.g. alpine.3.11) to target a version where binary compatibility changed, and this mapping is needed to assume forwards compatibility.

[ericstj]

@tmds if I understand you correctly, you're suggesting that the minor version RIDs for most distros aren't all that useful (assuming folks don't need to target those versions specifically).

Even if minor version were omitted from the RID, we'd still need to define all major versions and their relationships. Today the system requires literal entries in the graph for every possible option. We'd really like more flexibility here to avoid this.

[tmds]

I guess all code using the rid graph ends up doing these steps:

1. determine current platform rid, or get a value from user (e.g. -r argument).
2. use rid graph to turn it in to an ordered list (e.g. fedora.30-x64 is preferred over linux-x64).
3. use from available assets the ones that are highest in the ordered list.

The suggestion is that we can reduce the size of the rid graph if we take some information from the rid value from step 1: e.g. ubuntu.18.04[-x64], imports ubuntu.18[-x64], imports ubuntu[-x64].

What then remains in the rid graph is:

• Derive from linux/linux-musl
• Mapping between distros
• Express forward compatibility between minors

The graph is much smaller, as we don't have to list out every distro version.

Next, it would also make sense to generally assume forward compatibility. This requires that step 3 is capable of extracting version info from the rids to fall back. For example, an asset is available for fedora.31, and the current system is fedora.32.
Only alpine has such forward compatibility declared in the rid graph. For other distros, this scenario fails.

[tmds]

@ericstj , all, do you have some thoughts on the suggestions from my previous comment?

[ericstj]

@tmds I don't disagree. We try to represent the basic concepts of the RID graph with the code-generation steps we did in CoreFx around it. I don't think this issue is yet at the "suggest a workable implementation phase", it's at the "acknowledge the problem and agree to fix it phase". We need @rrelyea to agree to invest in the NuGet side and @jeffschwMSFT to agree to do the same on the host side. Once we have that we can move on to designing a solution together.

[tmds]

at the "acknowledge the problem

This is how it manifests for us:

• New version of Fedora don't work because they don't map to the appropriate parent rids. To avoid this, we try to patch the runtime.json up-front for versions that don't exist yet (but are expected in the near future).
• Since for Fedora rids don't map to their previous version, it's not possible to express 'fedora.30' and higher.
• Patching runtime.json is a manual, ad-hoc operation.

[tmds]

@ericstj @eerhardt are there some plans in .NET 5 to improve this?

[tmds]

@ericstj @eerhardt are there some plans in .NET 5 to improve this?

I hope this issue gets some love in .NET 6.
There shouldn't be a need to continue adding new releases of a distro to keep it working with .NET Core.