Runtime: Compute a list of OOB packages that would benefit from getting a .NET Framework targetted asset and make require changes so that they cross-compile

[joperezr]

cc: @ericstj @safern @ahsonkhan

Description

If you have an application that targets .NET Framework versions 4.6.1, 4.6.2, 4.7, or 4.7.1 and are trying to reference a .NET Standard library then it is likely your app will require runtime facades which get injected into your bin folder as part of the build, along with some binding redirects that get added to your app.config file. Sometimes, this can be painful when all you want is to add one single package reference to one of our OOB packages, and this in turn causes your app to have many facades copied to your bin folder and binding redirects added to your app config. We have gotten many asks(for example: https://github.com/dotnet/corefx/issues/30638, https://github.com/dotnet/runtime/issues/745, https://github.com/dotnet/corefx/issues/37929) to fix some of these OOB packages by adding a cross-compilation target to them for .NET Framework but we have definitely not fixed all of them yet and we have been treating them as case-by-case but this is an attempt to find out all of the ones that would need this and see if it is feasible to fix them all.

Customer Impact

Customers that have hit problems caused by this issue have been trying to add manual workarounds on their projects and custom logic for deploying. Once this is fixed, customers will have to remove those workarounds from their projects and reference the new packages in order to get the benefits of not needing the extra facades or binding redirects for their application.

Regression?

No.

Packaging reviewed?

It will involve packaging changes, so it will require a review by packaging experts.

Risk

Low. We plan on not increase the AssemblyVersion for this in order to not require additional binding redirects, and customers will only ingest the new packages if they want to get a better experience while targetting .NET Framework.

[joperezr]

I have finished analysing both release/2.1 and release/3.1 branches for packages in there that may benefit from a cross-targeting configuration for netfx in order to not pull in the facades. The full list is:

For release/2.1 branch:

System.Buffers - ref cross-compiles but implementation doesn't
System.Collections.Immutable
System.Composition.AttributedModel
System.Composition.Convention
System.Composition.Hosting
System.Composition.Runtime
System.Composition.TypedParts
System.IO.Pipelines
System.IO.Ports - ref cross-compiles but implementation doesn't
System.Json
System.Memory
System.Net.WebSockets.WebSocketProtocol
System.Reflection.DispatchProxy
System.Reflection.Metadata
System.Runtime.CompilerServices.Unsafe
System.Security.Cryptography.openssl
System.Text.Encoding.CodePages
System.Text.Encodings.web
System.Threading.Channels
System.Threading.Tasks.Dataflow
System.Threading.Tasks.Extensions
System.ValueTuple - Only applicable for versions < .NET 4.6.1 since it will resolve the netstandard1.0 asset from the package. We most likely won't care about this one since the facades aren't even injected when targeting < 4.6.1.

For release/3.1 branch:

System.Collections.Immutable
System.Composition.AttributedModel
System.Composition.Convention
System.Composition.Hosting
System.Composition.Runtime
System.Composition.TypedParts
System.IO.Pipelines
System.Json
System.Net.WebSockets.WebSocketProtocol
System.Numerics.Tensors
System.Reflection.DispatchProxy
System.Reflection.Metadata
System.Reflection.MetadataLoadContext
System.Resources.Extensions
System.Runtime.CompilerServices.Unsafe
System.Security.Cryptography.OpenSsl
System.Text.Encoding.CodePages
System.Text.Encodings.Web
System.Threading.Channels
System.Threading.Tasks.Dataflow

It is worth pointing out that a) there are duplicates in both branches and that is expected, since the same package might get built on both places and it is likely that it doesn't cross-compile on either one, and b) these lists include all packages that will be able to be restored on a .NET Framework app, and will provide a .net standard targetted asset for it which will most likely cause the facades to get injected into the bin folder.

[joperezr]

Just to add more info into this, I just locally made the required changes in order to get System.Memory to cross-compile and produce a package with netfx assets on it, and it took about 30-40 mins of my time. A big chunk of that time was just restoring release/2.1 branch for all configurations in order to be in a position to be able to make the changes, so I expect that once you do one, subsequent ones will be faster. I estimate roughly that the time per package that is fixed should be around 20-30 mins accounting for problems you could encounter with each one.

This basically means that it would take a dev about 2 days to modify all 25 packages across the branches with plus an extra day of buffering in case we run into any issues with one specific package, plus about two more days in order to calculate all packages upstack that would need to get recompiled and ship a new version in order to get a reference to the new packages that would get fixed by this which gives us around 5 days worth of work to get all changes ready.

After that, we would need to add an extra day or two in order to get reviews and PR iterations, plus an extra day or two for testing and making sure all issues are fixed and the actual packages get serviced. That would add another week to make it 2 works in total to complete this workitem.

TL;DR The rough costing of this is 1 week for the actual code changes, plus one more week for PR loop feedback, testing and release/shiproom process to give a total of 2 weeks for a single dev.

[ahsonkhan]

Do we need to re-build and ship updates for packages like Microsoft.Bcl.AsyncInterfaces as well to depend on the newer versions of things (probably Microsoft.Bcl.HashCode is fine as is since it doesn't have a dependency)?

[joperezr]

Yes that will be done on the Release/3.1 wave of changes because that is where we build that package from. AsyncInterfaces depends on System.Threading.Tasks.Extensions when running in .NET Framework which is one of the packages we are fixing on the Release/2.1 wave, so you are correct that this package will have to get fixed.

[ahsonkhan]

Cool. I brought it up since it wasn't listed in the For release/3.1 branch: category above. Might be worth adding to it :)

[joperezr]

Yeah it is not because it itself doesn't need to cross compile. The lists above only list out packages that are causing the problem, but for every branch once I'm done fixing those packages, I will also rebuild (and re-ship) all packages from that branch that reference these packages, so that is when AsyncInterfaces will be detected and re-shipped. I haven't made the full calculation for 3.1 of all of those which will have to be fixed.

[joperezr]

Closing as we have now finished the work for both 2.1 and 3.1