Conda-forge.github.io: Only build on PRs

[CJ-Wright]

Rebuilding on master takes up extra CI time, since we already successfully built things for them to get merged.

To avoid this we could:

1. build the package in the PR as usuall
2. if the build succeeds push the package to a staging area for the package
3. when the PR is merged move the package from the staging into the main

The main issues that need to be tackled to implement this are:

• [ ] Finding the correct package to move, we may produce a lot of packages with very similar names so picking among them might be an issue (maybe pick the latest)
• [ ] Make a staging area, using a staging account so that attackers can't reveal our tokens.
• [ ] Make something to do the moving, maybe a single Circle job or a webservice listening for merged PRs.
• [ ] Require CI to pass for merging, since we don't build on master we can't merge failing PRs.

ping @scopatz

[CJ-Wright]

A potential variant of this would be to have a label per PR. Right before pushing new packages to the label, remove the contents of the label, then push up the new packages.

This would allow users to access the label to get the hot off the CI build for local testing. The label would be very clean as it only has one package set in it at any time and no mixing with other packages (see the rc label).

[bgruening]

A potential variant of this would be to have a label per PR. Right before pushing new packages to the label, remove the contents of the label, then push up the new packages.

While this seems to be very appealing I think this is a security nightmare since I can trick the tests in my PR to get everything passed - and so everything will be uploaded to anaconda.org. Even if it's under a hidden/secret label it's uploaded and in the wild. However, we could do this for the bots and trusted users only.

The staging area sounds cool!

[jakirkham]

This is a good idea. Though it's worth noting it duplicates issue ( https://github.com/conda-forge/conda-forge.github.io/issues/124 ). Should we consolidate these somehow?

cc @kmuehlbauer

[CJ-Wright]

@bgruening your point is well taken, and I think having "trusted sources" could be a good theme for this and a few other issues floating around.

On the other hand if people actively download packages from a channel that they needed to a) opt in to and b) we very actively say not to touch unless they know exactly what they are doing. I imagine this ethos is a bit like the linux super unstable repos (looking at you arch testing repo). I think we need to be active in telling people that the only guarantee that we offer for that channel is that the builds passed conda-build nothing more.

[jakirkham]

Should add (failed to note this in my previous comment) this proposal IMHO addresses the provenance concerns raised in the older issue. Namely by not relying on CI cache, but just uploading the package and tweaking the label. This makes it very similar to issue ( https://github.com/conda-forge/conda-forge.github.io/issues/204 ), which tries to do the same thing as this issue, but for different reasons.

cc @astrofrog

[djsutherland]

I really like this idea, but perhaps we can help with both https://github.com/conda-forge/conda-forge.github.io/issues/647#issuecomment-426818182 and some other potential security concerns by uploading to a different channel entirely, e.g. conda-forge-staging? That would make the "finalization" process a little more complicated since it's not just a relabel, but it seems that if PRs can upload to a non-main label then the PR could easily be maliciously changed to upload to main as well.

[CJ-Wright]

I like the idea of having a separate account for this. One issue is how we would manage the moving from one channel to the other. We need a way to specify which artifacts to copy over, otherwise we may run into an issue where an attacker uploads a more recent artifact and then we copy that one over than the correct one.

[djsutherland]

That's a good point. We might need the build process to output a hash of the produced package file(s) in some non-fakeable place, e.g. in the build log, and then check those when master goes to copy the package over.

[CJ-Wright]

Yea, the build log might be the only non-fakeable place, since all other places require keys which could be spoofed. Idk if we can access the build logs with a nice API though.

[jakirkham]

Yeah I wonder to what extent this overlaps with CFEP 13/cf-staging work.

cc @beckermr

[beckermr]

You should consider the cf-staging staging channel an implementation detail of CFEP-13 and not rely on it for anything.