Source-build: create linux-x64 compatible tarball for each release

[tmds]

Create a public tarball for each release that includes nuget packages fetched from various feeds to enable an offline build.
Cfr gitter chat.

@dotnet/source-build-contrib

[tmds]

@crummel @dagood feel free to close this.

[dagood]

I see this has happened once (3.0 Preview 4) but I don't see it on the release checklist. +cc @dseefeld

[omajid]

Create a public tarball for each release that includes nuget packages fetched from various feeds to enable an offline build.

Based on where we are with bootstrapping, I would also be satisfied if we produced a tarball containing just the sources (and no prebuilts). We could then plug that in in the distro build system (with separate reference packages and source-built assemblies) to produce a final SDK using just the sources from source-build.

Since the tarball would contain only sources, it could be used by any/all distributions, without having to produce, say, RHEL 7 vs RHEL 8 vs Fedora 32 tarballs.

In Fedora, we already do a bunch of processing to produce a tarball that contains just sources. It's about 200 MB in size compared to the tarball with prebuilts which is/was roughly 2 GB in size.

[tmds]

I would also be satisfied if we produced a tarball containing just the sources (and no prebuilts)

:+1:

nuget packages and SDK can be separate tarballs.

Ideally, prebuilts is an SDK and nothing more.
I don't know if that is feasible. In theory, every nuget package used could be built from source using the SDK.

The SDK could be a pre-built SDK from Microsoft for bootstrapping.
Or an SDK that is available from the distro from a previous build.

separate reference packages

For me, splitting of the reference packages makes things more complex than building them from source each time. I don't understand why we're doing that.

[dagood]

For me, splitting of the reference packages makes things more complex than building them from source each time. I don't understand why we're doing that.

This was part of the discussion at https://github.com/dotnet/source-build/issues/1456, and yeah, the 5.0 plan involves getting rid of the special case (https://github.com/dotnet/source-build/issues/1456#issuecomment-640722786). If we can manage it, hoping to do a similar change in 3.1.

[omajid]

With the recent software supply chain attacks, having an official release tarball would also help us make sure we are working with verified and official sources releases and not something accidentally or maliciously modified by a bad actor.

[dagood]

I'm curious which artifacts you're thinking of here:

something accidentally or maliciously modified by a bad actor.

My understanding is that each distro will build its own tarball based on our signed Git tags, which seems kind of ideal. (Instead of involving Microsoft infra + distro infra to get it done, only use distro infra.)

The pre-release tarballs mentioned in the release checklist would certainly be better with official infra producing them rather than dev manipulation. Removing the need to build on RHEL by producing portable tarballs is one step. (We could also get RHEL CI, but we don't need to be blocked by that.) We will also need to get the build to produce the right version from the start (https://github.com/dotnet/source-build/issues/747).

[omajid]

My understanding is that each distro will build its own tarball based on our signed Git tags, which seems kind of ideal. (Instead of involving Microsoft infra + distro infra to get it done, only use distro infra.)

I have a slightly different perspective on this.

The current process looks like this:

1. The current maintainers (Microsoft using github infra) produce signed (or otherwise verified) tags
2. Each distro takes the tags, re-run the build and produce source tarballs (which may or may not contain prebuilts). Importantly, for the case of Fedora/RHEL (and Debian...) this is done offline on some developer's machine, often not in the distro build system.
3. The tarballs are used to do official builds, in the distro's build system (at least in Fedora/RHEL, Debian may still do this on a developer's machine).

For step 1, I am not sure how I can immediately verify that the tags used in source-build are all valid (source-build and all the individual component repos), consistent and haven't been tampered.

Step 2 is where a single actor can be malicious (imagine if someone paid me to inject malicious code in the source archive) or be vulnerable (suppose my machine get exploited).

Step 2 is also not 100% verifiable. For example, I can't re-run step 2 with the tag v5.0.100-SDK today to confirm my original tarball was the same. It now fails due to the myget repo move. As another example, I can't quickly compare the tarballs for RHEL and Fedora using checksums to verify validity. This gets even more complicated when distributions like Debian and Ubuntu enter the picture.

If step 1 and 2 were combined, most of this would go away: a simple checksum (sha256sum or, better, gpg verify) could be used to verify that the source archive for this release was valid and hasn't been tampered.

[dagood]

Those are excellent points, thanks for the explanation.

[mid-kid]

Is any distro currently hosting their source tarballs anywhere? As far as I can tell debian doesn't package this yet, archlinux just allows network during build, everyone repackages the binaries, and only fedora seems to package this in any "proper" sense, but I'm not familiar enough with fedora infrastructure to know if I can download their source tarballs.

[dagood]

Is any distro currently hosting their source tarballs anywhere?

For Fedora, if you explore from here:
https://src.fedoraproject.org/rpms/dotnet3.1
https://src.fedoraproject.org/rpms/dotnet5.0
you can download the src RPMs for the builds in the table, like this one:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1747331

[omajid]

fedora seems to package this in any "proper" sense, but I'm not familiar enough with fedora infrastructure to know if I can download their source tarballs.

Fedora puts all "source-tarballs" in a "lookaside" cache that you can use directly. I am not aware of any reason this can't be used by non-Fedora folks.

For example:

• .NET 5: https://src.fedoraproject.org/lookaside/pkgs/dotnet5.0/
• .NET Core 3.1: https://src.fedoraproject.org/lookaside/pkgs/dotnet3.1/

Watch out though, these are source-only tarballs. They don't contain any prebuilts (and have also removed a number of files with unclear licenses and so on). To build this source tarball, you will need a working source-built SDK, prebuilt artifacts from a previously source-built SDK, and source-build-reference-packages.

This was the process for bootstrapping .NET in Fedora, hopefully you can adapt it to your build system.