Zebra: Consider adding a stable clippy job

[teor2345]

The goals of this ticket are:

• identifying errors that are caused by code changes, and
• avoiding or distinguishing errors that are caused by dependencies or infrastructure

In #1144, nightly clippy introduced a new lint, which caused errors in all our builds.

This has happened at least three times that I remember - so it seems like we might get a similar error every month or two.

Let's think about switching to stable clippy jobs. Ideally, we'd run one job on nightly, and use a setting like Travis CI's "allow failures", so builds still succeed if it's just nightly that fails.

Related to #924, where we're thinking about adding a nightly compile job.

TODO

• [x] allow failure on the nightly clippy job
• [ ] add a separate stable clippy job, and rename the nightly clippy job to say "nightly"
• [ ] can we make allow failure and required jobs more distinct?

[yaahc]

I'd prefer to leave it on nightly. Clippy passing shouldn't be a required check in our CI pipeline, so I think that the benefits of staying up to date with the newest version of clippy outweigh the costs of occasionally having errors show up in CI without any changes to the code, especially since those errors don't prevent us from merging code.

[teor2345]

I'd prefer to leave it on nightly. Clippy passing shouldn't be a required check in our CI pipeline, so I think that the benefits of staying up to date with the newest version of clippy outweigh the costs of occasionally having errors show up in CI without any changes to the code, especially since those errors don't prevent us from merging code.

We can do both stable and nightly - if GitHub actions has an allow failures or similar setting. I've clarified the title of the PR.

[teor2345]

In this ticket and #924, I'd like to isolate infrastructure and dependency errors from errors that are caused by code changes.

That's really important for new and occasional contributors, who might not understand why all their PRs are suddenly failing. And it's useful for the core team, because it's less distracting - we don't have to drop what we're doing to fix CI, or ignore warnings or errors in our builds.

We might not be able to have perfectly reproducible builds, but we can:

• eliminate spurious errors,
• change how we present them, or
• provide a stable job for manual comparison.

[yaahc]

We can do both stable and nightly - if GitHub actions has an allow failures or similar setting.

It does have an allow failures and its already allowed as far as I could tell when I just checked the settings.

As far as I can tell the CI configuration is already in the state described by this issue. This does however seem to be non obvious with the current github ui, so if there's a setting to more clearly separate jobs that are and aren't required we should definitely enable that.

[teor2345]

Yeah, I'm used to Travis CI, which has a yellow marker (?) for failed allow failures jobs, and a build summary marker which ignores allow failure jobs.

Does the Google Cloud trigger run a stable clippy job?
If not, we should add a stable clippy job for comparison.

I'll also write a TODO list in the ticket summary, so we can track what we want to do as part of the ticket.

[dconnolly]

Yeah, I'm used to Travis CI, which has a yellow marker (?) for failed allow failures jobs, and a build summary marker which ignores allow failure jobs.

Does the Google Cloud trigger run a stable clippy job?
If not, we should add a stable clippy job for comparison.

I'll also write a TODO list in the ticket summary, so we can track what we want to do as part of the ticket.

The gcloud trigger job is building the Docker container image which currently does :

RUN rustc -V; cargo -V; rustup -V; cargo test --all && cargo build --release

[teor2345]

Okay, so gcloud is stable, that's good.

Can we add a separate stable clippy job that's allow failure?