Runtime: Packages with platform specific implementation in net4* TFM

[marek-safar]

We've identified following netstandard/corefx nuget packages as breaking when used with VS for Mac and targeting .NET Framework.

• System.Runtime.InteropServices.RuntimeInformation
• System.Globalization.Extensions
• System.Net.Http
• System.Text.Encoding.CodePages
• System.IO.Compression

The underlying issue is when running on non-Windows platform using .NET TFM e.g. net462 the nuget lib implementation is used and assembly copied to application folder for execution but that will fail because for packages above the implementation has hard Windows dependencies.

This is becoming serious issue as more developers adopt netstandard packages and even if Mono provides platform independent implementation the assembly from local folder is preferred over Mono implementation in GAC.

[akoeplinger]

/cc @ericstj @weshaggard

Related issue for RuntimeInformation: https://github.com/dotnet/corefx/issues/9012
We're contemplating a "fix" in the Mono runtime on our side to essentially blacklist those assemblies from loading from the app folder, but a more long-term solution would be great.

[karelz]

@akoeplinger changing the right Eric alias above (missing 'j' at the end)

[ericstj]

The reason these packages do this is to make the packages work with Desktop packages.config.

FWIW you have the same exact problem for any package that works with desktop and needs to pinvoke.

We could try and make these assemblies do "the right thing" by detecting mono and doing something different, but I would imagine that in any case you'd prefer to use the mono implementation.

Rather than blacklist solely on name why not qualify it with something that lets you uniquely identify them and we can agree on that as a public contract?

[marek-safar]

FWIW you have the same exact problem for any package that works with desktop and needs to pinvoke.

Correct any package could end up in this situation but for netstandard packages the assumption that they will just work is much higher.

Fixing the assemblies to do the right thing and work on all platform might not be that easy. In some cases we have another dependencies which would have to be abstracted and moved to corefx, sometime we use internal API via IVT etc.

Blacklist-ing is another option which is basically what we do today but it's quite a hack because we blacklist based on MVID as there is nothing else we can use to identify such assemblies.

Another option could be to have a new TFM for desktop mono which fallback to .net desktop TFM and avoid scenario all together because we could have _._ for such packages.

[ericstj]

TFM for desktop mono which fallback to .net desktop TFM

We discussed this with NuGet folks, @migueldeicaza, and @mhutch a few months ago and folks didn't want to do that. Folks were saying that the benefit of desktop mono is that it is identical to desktop. That means no build-time solution is acceptable.

The downsides of a black-list / unification table is it makes it harder for you to allow for app-local copies of these assemblies. Is that a scenario you care about in general: replacing the in-framework assembly with an applocal version?

[marek-safar]

I haven't though about copy deployment but agree it's important aspect.

Yes, we care about replacing framework assemblies with local version. As Mono is open source project anyone can build and copy fixed/improved/optimized version of system assembly to local folder and expect it to work.

[akoeplinger]

An idea we had for a more long-term solution would be adding a new (internal) attribute to an assembly, e.g. PreferMonoInboxVersion. It would be similar to the existing ReferenceAssembly attribute where the runtime will skip loading the assembly.

Adding this to each CoreFX package which only supports Windows/.NET Framework should be trivial and it could simply be removed down the line if a package switches to a portable implementation that works on Mono.

Thoughts?

[akoeplinger]

ping, any updates on this?

[karelz]

BTW: @ericstj is OOF if you're asking for his opinion.

[marek-safar]

Another ping, we'd love to get this sorted out in NS2

[karelz]

@ericstj @mellinoe can you please help here?

[ericstj]

Adding an attribute is indeed easy and something I'd be willing to push through. Rather than a new type, how about AssemblyMetadataAttribute? That's a generic name-value pair where we stuff data like this (eg: the serviceable attribute).

Some other naming options

• "WindowsSpecific" : "true"
• "Platform": "PlatformId.Win32NT" (where the value can be parsed as a PlatformID enum)
• "OSPlatform": "WINDOWS" (where the value can be passed to System.Runtime.InteropServices.OSPlatform.Create)

/cc @weshaggard @terrajobst for thoughts on naming and using such a scheme.

[weshaggard]

I'd be OK with using AssemblyMetadataAttribute and marking the assembly as "Windows" if it has Windows PInvokes or is specific to .NET Framework on Windows in some way.

My only question is what would the mono runtime do if it detects this? Will it block it and fail like the loader does for reference assemblies? Or will it just silently ignore it? If it it ignores it then how will assemblies that actually depend on it work?

[marek-safar]

I think Mono (or possibly netcore) should behave same as with ReferenceAssemblyAttribute. The assembly will be ignored in execution context but you can load it in reflection only context.

[ericstj]

For ReferenceAssemblyAttribute we throw when it is loaded for execution. Are you saying that Mono will just ignore it, then continue probing for a different assembly?

[marek-safar]

IIRC yes, Mono loader will continue looking for next candidate assembly which if I remember correctly is .net desktop behaviour too.

[weshaggard]

I don't think the loader does anything interesting with the ReferenceAssemblyAttribute. However for reference assemblies we also set the AssemblyFlag 0x70 (https://github.com/dotnet/buildtools/blob/master/src/Microsoft.DotNet.Build.Tasks/PackageFiles/ReferenceAssemblies.targets#L37) which the loader does look at and will block loading for execution. By block it will throw an exception for things marked with https://github.com/dotnet/coreclr/blob/32f0f9721afb584b4a14d69135bea7ddc129f755/src/inc/corhdr.h#L746.

So it the behavior here would need to something different if you wanted this marker to essentially ignore the assembly.

[ericstj]

@weshaggard what we do here isn't as interesting since we're not likely to change destkop or coreclr. I confirmed that desktop throws for a reference assembly and does so at a place after fusion is already done so it doesn't continue looking. If mono is different here then that's great and I think it'll serve our purpose.

Here's where we throw:
https://github.com/dotnet/coreclr/blob/a9b25d4aa22a1f4ad5f323f6c826e318f5a720fe/src/vm/pefile.inl#L127

[ericstj]

Given the difference, @marek-safar, it'd be good to test this on mono. The way I did it was I built a desktop app referencing System.Net.Http package. Then I put the implementation assembly in a bin subfolder and added that as a probingPath in the app.config. I then took the reference assembly out of the package and put that next to the application.

[ericstj]

@marek-safar / @akoeplinger can you please chime in on this? We need to close on it pretty soon if we want to get a change into the corefx packages for .NET Core 2.0.

We need to close on the mechanism. I think AssemblyMetadataAttribute is better than a new type for us, so long as that works for you.

Suggestions of the content are as follows:

• "PreferInboxVersion": "true"
• "PreferMonoInboxVersion": "true"
• "WindowsSpecific" : "true"
• "Platform": "PlatformId.Win32NT" (where the value can be parsed as a PlatformID enum)
• "OSPlatform": "WINDOWS" (where the value can be passed to System.Runtime.InteropServices.OSPlatform.Create)

[karelz]

FYI: Adding back area-Infrastructure as it covers also packaging and we need 1 area label per issue.

[ericstj]

This has nothing to do with packaging. It's about adding an assembly attribute to our assemblies. I suppose that's "infrastructure" but perhaps we need some other classification.

[karelz]

OK, that sounds like Meta - cross-cutting code work which is not infra :). (I hope @tarekgh doesn't mind)

[ericstj]

I'm actually leaning more towards "PreferInbox" beahvior. I suspect this problem isn't unique to just net4-specific stuff. Consider the cases where we produce an out-of-band library (like System.ValueTuple) then eventually promote it to inbox.

This is in-line with the desktop "GAC wins" behavior. If the framework has one at the same version (using whatever unification rules you want) then use that. Otherwise use the one from the app.

[marek-safar]

I think there are few angles to the issue but at the bottom it's still down to the problem if the nuget package works with any runtime or is runtime specific (including version). I don't think we necessary have to cover case where the package works with specific runtime on specific OS only.

PreferInbox won't solve the issue fully, it could work for some assemblies like System.ValueTuple but for the ones which work on e.g. .NET runtime and not on others this won't help.

We could have PreferMonoInbox either as a attribute or as ctor value to AssemblyMetadataAttribute but keep in mind that the attribute needs to be parsed by runtime loader so it'd better to have simple ctor syntax for it.

[ericstj]

PreferInbox won't solve the issue fully, it could work for some assemblies like System.ValueTuple but for the ones which work on e.g. .NET runtime and not on others this won't help.

Can you describe how this is different than PreferMonoInbox? In both cases I'd expect mono to load the one in the app if it doesn't have one inbox. At that point it doesn't matter if the one in the app is runtime specific.

In other words, I'm imagining the behavior of PreferInbox and PreferMonoInbox as the same, just the name being different.

[marek-safar]

Maybe I misunderstood the PreferInbox intention but we'd like to e.g. block loading System.Runtime.InteropServices.RuntimeInformation (use inbox version) with next minor nuget update.
If .NET FX doesn't recognize this attribute then we are good but I though you wanted to use it to control this behaviour on any .NET runtime.

[ericstj]

For other .NET runtimes we already have loader policy and build infrastructure that determines which assemblies to load, and all of them already follow the "prefer inbox" semantics given equal versions.

[surban]

Is this supposed to be fixed? I think I might be experiencing this issue, but because I was not sure, I first created a new issue at mono/mono#7011.