Runtime: R2RDump questions for 5.0 source-build

[dagood]

I ran into the R2RDump project while investigating prebuilts in the 5.0 source-build effort. I have some questions:

1. Does R2RDump end up in the product (SDK or Runtime)?
   
   
   
   • This means we need to be able to build it from source without prebuilts. If not, maybe we can just skip it.
   
   
2. Why does it have a dependency on Microsoft.NETCore.CoreDisTools/1.0.1-prerelease-00005?
   
   
   
   • https://github.com/dotnet/runtime/blob/c5a3f49c88d3d907a56ec8d18f783426de5144e9/src/coreclr/src/tools/r2rdump/R2RDump.csproj#L21-L23
   
   
   • Context: this package appears in the dotnet5-transport feed: https://dev.azure.com/dnceng/public/_packaging?_a=package&feed=dotnet5-transport&package=Microsoft.NETCore.CoreDisTools&protocolType=NuGet&version=1.0.1-prerelease-00005, but they were all pushed at once (pointing to some manual migration) and they don't appear to be regularly updated.
   
   
3. Where is the source for Microsoft.NETCore.CoreDisTools/1.0.1-prerelease-00005?
   
   
   
   • The nuspec links the coreclr repo as projectUrl, but I can't even find it there in 3.1 or 2.1.
   
   

This package brings in a few others via runtime.json, runtime.win-x64.[...] and runtime.win-x86.[...]. (We only build from source on non-Windows platforms, so it seems odd to me that we get Windows prebuilts but nothing like linux-x64. The direct cause is <RuntimeIdentifiers>win-x64;win-x86</RuntimeIdentifiers>, but I don't know the reason that line is in the csproj.)

The broader context is that I see this package in the annotated prebuilt report from 5.0-preview8, where R2RDump is the only project detected as using these packages:

<AnnotatedUsage Id="Microsoft.NETCore.CoreDisTools" Version="1.0.1-prerelease-00005" File="src/runtime.bf456654f9a4f9a86c15d9d50095ff29cde5f0a4/artifacts/obj/coreclr/R2RDump/project.assets.json" IsDirectDependency="true" Project="src/runtime.bf456654f9a4f9a86c15d9d50095ff29cde5f0a4/" />
...
<AnnotatedUsage Id="runtime.win-x64.Microsoft.NETCore.CoreDisTools" Version="1.0.1-prerelease-00005" File="src/runtime.bf456654f9a4f9a86c15d9d50095ff29cde5f0a4/artifacts/obj/coreclr/R2RDump/project.assets.json" Rid="win-x64" Project="src/runtime.bf456654f9a4f9a86c15d9d50095ff29cde5f0a4/" />
<AnnotatedUsage Id="runtime.win-x86.Microsoft.NETCore.CoreDisTools" Version="1.0.1-prerelease-00005" File="src/runtime.bf456654f9a4f9a86c15d9d50095ff29cde5f0a4/artifacts/obj/coreclr/R2RDump/project.assets.json" Rid="win-x86" Project="src/runtime.bf456654f9a4f9a86c15d9d50095ff29cde5f0a4/" />

@mangod9 @nattress (area owners) can you help us figure out what to do with these prebuilt dependencies?

/cc @mmitche, you're listed as the Microsoft.NETCore.CoreDisTools pusher, so cc in case you remember the reasoning.

[mangod9]

Tagging @cshung since he is tracking a related issue: https://github.com/dotnet/runtime/issues/41379 about packaging R2R.

[cshung]

@dagood, to answer your questions:

1. I don't think R2RDump ends up in the runtime or SDK,
2. Because it needs to disassemble native instructions stored in a ready to run image, and
3. I am uncertain, but here is a copy of the code (which may or may not match the binary)

I am only trying to package ILCompiler.Reflection.ReadyToRun, a managed only library used by R2RDump. The native dependency of R2RDump is irrelevant to my work there.

[janvorli]

@dagood, @echesakovMSFT has recently cleaned up and moved building of the coredistool packages and made them cover all the architectures we support. AFAIK, he's now working on enabling publishing of the new packages. I am waiting with an update to R2RDump for that. See https://github.com/dotnet/jitutils/pull/293

[echesakovMSFT]

@dagood, @echesakovMSFT has recently cleaned up and moved building of the coredistool packages and made them cover all the architectures we support. AFAIK, he's now working on enabling publishing of the new packages. I am waiting with an update to R2RDump for that. See dotnet/jitutils#293

Thank you Jan for replying! Yes, I am in the middle of testing/publishing the new coredistools packages. I have a plan to restructure the nupkg so it would contain all the binaries (for all hosted architectures) in one package (not multiple).

[dagood]

Hmm. https://github.com/dotnet/llilc doesn't build using Arcade infra and doesn't hook into dependency flow, which isn't good for traceability or easily bringing it into source-build. 😕 Exactly what form the packages take doesn't matter so much as being able to reproduce them in a source-build context and trace back to exact commit hashes by following dotnet/runtime's eng/Version.Details.xml--or at least some reliable method. (dotnet/llilc has no Git tags, either.)

But, the critical takeaway for source-build here is that R2RDump is not used by the Runtime or SDK, so we don't have to build it or its dependencies from source. This will let us skip R2RDump for 5.0.

[echesakovMSFT]

@dagood I see your point. The sources of updated yet to publish coredistools are in https://github.com/dotnet/jitutils not https://github.com/dotnet/llilc and we have a low priority goal of using Arcade for building jitutils (see https://github.com/dotnet/jitutils/pull/219). But this probably doesn't matter much, since coredistools is not a part of the product - it's purely used for testing (gcstress,superpmi) and debugging (r2rdump) purposes only.

[dagood]

I've opened https://github.com/dotnet/source-build/issues/1816 to track building R2RDump from source in the future and discuss how much the feature is important to source-build users. (I.e. will someone need to run R2RDump but can't because they can only use built-from-source tooling?) The plan is still to skip it for now.