Runtime: Upload build assets and generate partial build manifest in the installer legs

[ViktorHofer]

We could save 7-10 minutes in our build by uploading build assets and generating a partial build manifest directly in the installer legs instead of in the "Prepare Signed Artifacts" leg which only runs after the whole build stage is done. By doing that we could shrink the "Prepare Signed Artifacts" leg down to only merge the partial build manifests and push the BuildId to BAR.

@jkoritzinsky is looking into if we need tooling support for this.

cc @mmitche @NikolaMilosavljevic @safern

[hoyosjs]

Is the time saving from this mostly download + upload time for the artifacts?

[jkoritzinsky]

Yes. And if we can remove the job, it would save us time cloning the repo as well.

[hoyosjs]

@jkoritzinsky uploading to the bar already merges the manifests into one. So maybe this could even work transparently. The only thing to verify is PB and in-build signing work/end up in the manifest.

[jkoritzinsky]

I was looking through the aspnetcore build and noticed that. That should make this much easier. I'm just trying to figure out how their repo does it since they explicitly exclude generating binlogs in the official build.

[jkoritzinsky]

I think just passing the -publish flag will work, but I need to figure out how to ensure that all of the uploaded manifests have unique names.

[jkoritzinsky]

They just explicitly pass the AssetManifestFileName property with a preset name. We should be able to do the same thing with our jobs.

[hoyosjs]

I was looking through the aspnetcore build and noticed that. That should make this much easier. I'm just trying to figure out how their repo does it since they explicitly exclude generating binlogs in the official build.

Yeah, that always makes their's more... fun...

I think just passing the -publish flag will work, but I need to figure out how to ensure that all of the uploaded manifests have unique names.

We can use the job name as the manifest name. It's guaranteed to be unique.

[ViktorHofer]

Sounds like we could get rid of the "Prepare Signed Artifacts" leg entirely. That would be fantastic.

[safern]

As a note... we could also save some time if Installer didn't depend on All Configurations leg, and in the All Configurations leg we could produce the manifest for libraries OOB packages.

[jkoritzinsky]

Installer doesn't depend on AllConfigurations any more. I agree with Santi that we should have AllConfigurations produce the manifest for the libraries OOB packages.

[safern]

Installer doesn't depend on AllConfigurations any more.

That's great to know! That means we could prototype a collapsed official build that might be faster and more reliable 😄