Runtime: Make parts of the build not require coreclr to be built

[steveisok]

As of right now there are parts of the build that require a dotnet in order to be built. Since mono has been added to dotnet/runtime, this places a dependency on building coreclr beforehand. Because mono can also be used as a 'dotnet', the goal of this issue is to track and remove this dependency where applicable.

Here are the spots we know of. Please feel free to correct / add:

• [ ] Libraries build
• [ ] Coreclr (runtime) tests (probably should include moving them to a shared spot)
• [ ] Libraries tests

@jaredpar

[jkotas]

parts of the build that require a dotnet in order to be built.

Could you please share more details on this?

We do support cross-targeting for Linux build. For example, you can build for ARM64 on x64 machine today. dotnet used by the build is the x64 version that comes as a downloaded package, and the dotnet produced by this build is the arm64 version. I do not think we are building x64 dotnet in this mode at all.

Can building for iOS work the same way?

[akoeplinger]

Yeah the goal is to not need coreclr for the iOS build.

I found one test project that relies on the testhost to generate some artifacts:

https://github.com/dotnet/runtime/blob/58cf1468e1d9519c3752dfd40402eb4fda6fb9e9/src/libraries/Microsoft.XmlSerializer.Generator/tests/Microsoft.XmlSerializer.Generator.Tests.csproj#L11

[steveisok]

By dotnet in this instance, I meant System.Private.CoreLib, but wasn't sure if that was everything.

It was easier to be generic ;-)

[jaredpar]

@steveisok

By dotnet in this instance, I meant System.Private.CoreLib, but wasn't sure if that was everything.

There are two versions of System.Private.Corelib that are produced by the build. Do you mean that we need to be able to build mono without buliding the coreclr version of System.Private.Corelib?

[steveisok]

@jaredpar Yeah, ideally. Libraries seems to have a hard dependency on the coreclr build because of SPC. If libraries could use mono's SPC instead for the mono build, that would be ideal.

https://github.com/dotnet/runtime/blob/9981b1f128283c9fcff32689f34bf51289209b03/eng/liveBuilds.targets#L90-L136

[jaredpar]

Yeah that looks like one of the targets I saw in my earlier PR that needed to be properly conditioned on whether we were building Mono / CoreCLR.

[steveisok]

One other point... If libraries just needs SPC to exist in order to build, it may be faster to default to mono's. No data to support that other than observation.

[jaredpar]

It's unclear. When I was unwinding this a few weeks ago it needed a specific one. All are likely a case by case issue.

[jkotas]

I found one test project that relies on the testhost to generate some artifacts:

This test has a long list of exclusions in SkipTestsOnPlatform, so you can just add to that, and file issue to get it fixed properly.

[jaredpar]

@steveisok

If libraries just needs SPC to exist in order to build, it may be faster to default to mono's. No data to support that other than observation.

I have data and it supports that position :smile:

Grabbing an arbitrary run of the rolling builds and averaging out the product build times I see the following:

• Mono: 9 minutes
• CoreCLR: 27 minutes

That's a rough measure though. Just averaging out all of the product timeline builds. But still a decent approximation.

Still though I think it's likely most of these targets are looking for a specific runtime, not a general one. At least that's true in the current state.

Note: this is just a rough approximation. It's possible our CoreCLR product build is including other items that drive the time up. Haven't actually dug very deep.

[jaredpar]

Made some progress here and already found bugs. We have different nullable annotations in the Mono specific parts SPC. 😄

[jaredpar]

Anyone interested in the progress can follow this branch: https://github.com/jaredpar/runtime/tree/mono-only

I've fixed all of the build targets which effectively hard coded references to CoreCLR. Using the following two commands I can now essentially compile the libraries using the System.Private.Corelib generated by Mono

> .\build.cmd -subsetCategory Mono
> .\build.cmd -subsetCategory libraries /p:RuntimeFlavor=Mono

The use of /p:RuntimeFlavor=Mono is necessary because we don't have a top level switch to control which runtime the libraries should compile against. After discussion with @jkoritzinsky will likely add a -runtimeFlavor switch to the root build.

There are a few API compat issues that need to be worked through with Mono. It's fairly straight forward to fix most of them, the rest I can conditionally suppress when building against Mono.

Where I'm stuck though is how to deal with COM specific types like EventRegistrationToken. These aren't included in Mono, presumably because it doesn't support COM or $(FeatureCominterop), hence the Windows runtime and interop DLLs can't build:

System\Runtime\InteropServices\WindowsRuntime\MarshalingHelpers.cs(67,39): error CS0246: The type or namespace name 'EventRegistrationToken' could not be found (are you missing a using directive or an assembly reference?) [P:\runtimesrc\libraries\System.Runtime.WindowsRuntimesrc\System.Runtime.WindowsRuntime.csproj]

Unsure what the best path forward is from here:

1. Define stub types in Mono for these types that just throw PlatformNotSupportedException for the necessary members
2. Work to exclude the windows runtime specific projects from the build. That may be a bit difficult though.
3. Another idea I'm not considering

[jkotas]

Define stub types in Mono for these types that just throw PlatformNotSupportedException for the necessary members

This is the best way to solve it. It should be pretty straightforward to do.

[akoeplinger]

The use of /p:RuntimeFlavor=Mono is necessary because we don't have a top level switch to control which runtime the libraries should compile against. After discussion with @jkoritzinsky will likely add a -runtimeFlavor switch to the root build.

I'm not against such a switch but maybe we can default that e.g. if the subsetCategory includes coreclr or not.

E.g. doing .\build.cmd -subsetCategory mono-libraries would build mono and libraries with RuntimeFlavor=mono.

In my iOS branch I currently default subset categories to mono-libraries so you can do a single ./build.sh -os iOS in the top level and it builds the correct things.

[steveisok]

@jaredpar I started stubbing out the com interop items and it's going to extend to other features that are enabled like $(FeatureUtf8String).

[steveisok]

@jaredpar Since what's not working is isolated to windows, do you think we can land your changes and make it apply to non-windows?

[akoeplinger]

@steveisok I think this issue can be closed, we can build without the coreclr subset now.