Runtime: Changes to CoreCLR workflow

[jkoritzinsky]

PR #31701 is changing the developer workflow for CoreCLR to help increase reliability, reduce local build times, and move us closer to a place where we can better parallelize CI to improve build times.

The largest part of the change is that the build.cmd and build.sh scripts under src/coreclr will be deprecated and later (probably in a few weeks) removed. The src/coreclr/src/build.proj file will also become unused and will be deleted. To build the coreclr product, the supported path will be to build from the root build script.

For those who strongly dislike building with MSBuild, the native runtime build will still be buildable with new build-runtime.cmd and build-runtime.sh scripts in src/coreclr that only build the native runtime build and the cross-arch native build. There will also be separate scripts available for running crossgen on System.Private.CoreLib. All managed builds will be driven exclusively through the Arcade scripting.

The way to control what parts of the CoreCLR build you run will be via the "subsets" infrastructure. See #31701 for specific subset names.

Below are some examples of commands for common workflows:

• Build CoreCLR runtime, System.Private.CoreLib, crossgen System.Private.CoreLib all in Debug. Build all of Libraries in Release. (Common workflow when syncing)

./build -subsetcategory coreclr-libraries -subset runtime-corelib-nativecorelib-all -c Release -runtimeconfiguration Debug

• Build CoreCLR runtime

./build -subsetcategory coreclr -subset runtime

• Build CoreCLR runtime (build-runtime version)

src/coreclr/build-runtime

• Build CoreCLR runtime and Crossgen2

./build -subsetcategory coreclr -subset runtime-tools

The subsets should give as much fine-grained control as was previously provided within the managed components. The native build scripts continue to have some finer-grained control and the build-runtime scripts are available for those who need it.

[stephentoub]

@jkoritzinsky, I'm struggling to follow the examples a little. If today I do this:
C# src\coreclr\build -release -skiptests
what does that translate to now? Thanks.

[jkoritzinsky]

@jkoritzinsky, I'm struggling to follow the examples a little. If today I do this:

src\coreclr\build -release -skiptests

what does that translate to now? Thanks.

That translates to build -subsetcategory coreclr -c Release.

CoreCLR tests are now only built through the src/coreclr/build-test scripts, not the product build scripts, so there’s no need for a skiptests argument.

[jkotas]

What is going to be equivalent of src\coreclr\build.cmd linuxmscorlib ?

[jkoritzinsky]

What is going to be equivalent of src\coreclr\build.cmd linuxmscorlib ?

The equivalent is ./build -subset corelib -os Linux.

[jkotas]

build -subsetcategory coreclr -c Release

This fails very early for me with:

Restore completed in 78.15 ms for D:\runtime\src\coreclr\src\tools\crossgen2\crossgen2\crossgen2.csproj. 'py' is not recognized as an internal or external command, operable program or batch file. The filename, directory name, or volume label syntax is incorrect. D:\runtime\src\coreclr\src\System.Private.CoreLib\System.Private.CoreLib.csproj(467,5): error MSB3073: The command ""'py' is not recognized as an internal or external command,;operable program or batch file.;C:\Users\jkotas\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\python.exe" -B -Wall "D:\runtime\src\coreclr\/src/scripts/genRuntimeEventSources.py" --man "D:\runtime\src\coreclr\/src/vm/ClrEtwAll.man" --intermediate "D:\runtime\artifacts\obj\coreclr\Eventing\x64\Release"" exited with code 123.

What is the recommended way to deal with the python dependency?

[jkoritzinsky]

That is a bug in eng/python.targets that didn't show up in testing somehow. I'll go fix that today. The workaround is to set the PYTHON environment variable to point to your python installation.

[jkotas]

Thanks! FWIW, I have the python setup according to https://github.com/dotnet/runtime/blob/master/docs/workflow/requirements/windows-requirements.md#python

[BruceForstall]

@jkoritzinsky You suggest using:

Build CoreCLR runtime, System.Private.CoreLib, crossgen System.Private.CoreLib all in Debug. Build all of Libraries in Release. (Common workflow when syncing)
./build -subsetcategory coreclr-libraries -subset runtime-corelib-nativecorelib-all -c Release -runtimeconfiguration Debug

So, subsetCategory can be concatenated with -? Same with -subset?

What does the "all" subset refer to? It looks like possibly one or all of:

  - Libraries|All
      The .NET libraries comprising the shared framework.
  - Mono|All
      The Mono .NET runtime.
  - Installer|All
      The .NET Core hosts, hosting libraries, bundles, and installers. Includes these projects' tests.

but maybe because only libraries is specified in -subsetCategory, it only applies to libraries and not Mono or Installer?

Is there a reason why there is no all for CoreClr?

[jkoritzinsky]

Yes, subsetCategory can be concatenated with - just like -subset.

When you specify -subset all, you specify the all subset within all selected subset categories.

There's no all for coreclr since the all subset there was only because we didn't have the separate subsets. The all subset for installers doesn't actually work. When we fix that, we'll add back an all subset for CoreCLR.

[BruceForstall]

I use the src\coreclr\build.cmd "all" functionality to be able to build multiple architectures in one command. If I want to continue doing that, how should I build (from scratch)? Maybe:

call build.cmd -subsetcategory libraries -c Release
call src\coreclr\build.cmd all x64 x86 arm arm64 debug checked release

followed by a series of:

src\coreclr\build-test.cmd x64 debug   GenerateLayoutOnly

for each arch/flavor to generate all the Core_Root directories? (without GenerateLayoutOnly to also build the tests)

[jkoritzinsky]

I believe that's the current best solution for building coreclr and S.P.CL for multiple architectures. If you're just building coreclr, then you can use src/coreclr/build-runtime all x64 x86 arm arm64 debug checked release (the build-runtime script still supports any relevant command line options from the old system).

If you need that for the full coreclr build, can you open another issue so that we make sure to enable that before removing the old scripts?

[sandreenko]

@jkoritzinsky have you seen errors like:

F:\git\runtime\src\libraries\Directory.Build.targets(9,3): error MSB4024: The imported project file "F:\git\runtime\Directory.Build.targets" could not be loaded. The process cannot access the file 'F:\git\runtime\Directory.Build.targets' because it is being used by another process. [F:\git\runtime\src\libraries\shims\manual\mscorlib.csproj]

Build FAILED.

I was trying to do build -subsetcategory coreclr-libraries -subset runtime-corelib-nativecorelib-all -c Release -runtimeconfiguration Checked -arch x86 and saw it twice in different repo copies.

Log files in runtime\artifacts\log\Debug\CoreCLR_Windows_NT__x86__Checked.* did not show anything useful.

[sdmaclea]

@jkoritzinsky The deprecation warning message in CoreCLR makes little sense. It says to use build-native.sh which only exists in libraries. I suspect it should say build-runtime.sh? Also building docs seem to be out of date.

[jkoritzinsky]

Yep that’s a typo from when I was renaming. I was going to name the files build-native but then changed my mind. Can you point me to the specific docs that are out of date? I thought I updated them.

[briansull]

Can you tell me what the very first command to use is to build everything after a clean sync?

For Checked?

[jkoritzinsky]

@sandreenko I have not seen any errors like that locally.

@briansull If you want to only build coreclr and libraries: ./build -subsetcategory coreclr-libraries -runtimeconfiguration checked -c release
If you want to build the whole repo (including Mono and Installer): ./build -runtimeconfiguration checked -c release

[briansull]

Do I ever need to build Mono and the Installer, to do development on coreclr?

[briansull]

I will try using:
./build -subsetcategory coreclr-libraries -runtimeconfiguration checked -c release
on my branch new clean up to date enlistment and report back.

My build was successful!!

[jkoritzinsky]

It's very unlikely you would need to build Mono and the installer, but I wanted to give that command as well just to cover my bases.

[sdmaclea]

Can you point me to the specific docs that are out of date?

I was looking at the cross compilation and Linux coreclr instructions. They seemed to still be referring to the build.sh script in src/coreclr.

Also a lot of your noted in the top post seem to have not made it to docs. Seems that is why there is traffic on this issue.... It is not clear where the build recipes are documented for developer scenarios...

[jkoritzinsky]

I think #33325 updates most of the docs. I'll make sure to add the commands above to a new "commonly used CoreCLR build commands" doc so they're easy to find.

[sdmaclea]

I was going to name the files build-native but then changed my mind

I found the build-runtime name counter intuitive. I was looking to build the DAC which is a native component, but is not the runtime. So I would have preferred, buuild-native for the non-runtime components...

[jkoritzinsky]

The reason I chose build-runtime was so the script matched the subset name. Basically so the command build -subsetcategory coreclr -subset runtime was equivalent to src/coreclr/build-runtime. I realize now that that may not have been the thing to aim for (or that the subset should also be named “native”)

[ViktorHofer]

Unpinning (3 weeks pinned should be sufficient) to make room for new announcements.