nixpkgs has (SRI?) hashes that the Nix version in use on travis-ci (2.0.4) can't handle

[abathur]

I had a travis-ci build break with error: unknown hash algorithm '' on a simple merge+squash+push of work that passed on May 27. I re-tried a job from a clean run and it also broke (good @ nixpkgs-20.09pre226586.571212eb839, bad @ nixpkgs-20.09pre227892.b27a19d5bf7) so I did a little digging.

I suspected this had something to do with the dated Nix 2.0.4 (May 21 2018) and clever pointed me to the hashing/SRI change. Grepping the diff for \bhash\b = identified pip as the likely culprit (change in https://github.com/NixOS/nixpkgs/commit/193cdc77703f1635b3fb55f80086eb0211499def) with hash = "sha256:0s9z72cpa15p2bp7zq1lid8k2ykrznfzjwpq5f41v3f30faraxg7";, but I was a little confused at the format since I thought the SRI format only used a dash/hyphen.

(I later found a note about the legacy <algo>:<hash> format and confirmed pip caused the break with a --show-trace build, but the main focus here is what I noticed while grepping. I don't have much context/history on this, so I'm just dumping what I found without much sense of which parts are/aren't problems.)

I'm not sure which (if any) combinations of the attribute name and hash format are supposed to be "in the wild" at this point (nor have I hand-validated each match) but git grep -E "(sha256|hash) = \"sha256[:-]" -- "*.nix" against master identified 52 instances in 51 files:

• 8x hash = "sha256-..."
• 10x sha256 = "sha256-..." (also spotted a cargoSha256 = "sha256-...", not included in these numbers)
• 15x hash = "sha256:..."
• 19x sha256 = "sha256:..."

(I can post a list of files and/or commits if needed?)

Notify maintainers
I'm not sure who all needs to know, so I'll start by apologizing to anyone unduly spammed, and list a reason for each:

• travis-ci @domenkozar @garbas @matthewbauer @grahamc
• https://github.com/NixOS/nix/commit/6024dc1d97212130c19d3ff5ce6b1d102837eee6 @zimbatm @edolstra
• https://github.com/NixOS/nix/pull/3433 @LnL7
• regulars I recognize who committed 2+ of the hashes that matched grep: @FRidh @jonringer @emilazy @colemickens @jtojnar

[FRidh]

Nix on Travis hasn't really been maintained as can be seen from the version. Note GitHub Actions uses a much more recent version.

[LnL7]

That doesn't really matter, minvers.nix states 2.0 compatibility and no decision was made to drop support.

[zimbatm]

One issue is that some versions of Nix output SRI hashes in the hash mismatch message and it's not configurable. For some fetchers it's possible to fall back on nix-prefetch-url but not for cargoSha256 and friends.

[LnL7]

Indeed, that's why I made a pull request to make that configurable a while back.

[Zimmi48]

For the record, a workaround for Travis users is to add an explicit nix: 2.3.5.

[Ma27]

Is Nix 2.0 even maintained atm? And if not, is there any reason not to bump the minimal required Nix version on unstable (and the upcomming 20.09)?

[LnL7]

That's an option, however these problems go back to 19.03 which strictly speaking means there's no transition point.

[Ma27]

Yeah, I'm aware of that. I mentioned this option to have a mid-term solution for the current problems. We should probably bump this minver more regularly in the future to minimize the amount of Nix-related eval-problems in nixpkgs.

[FRidh]

Possibly also evaluate with the minver in CI.

[Infinisil]

We can't bump minver too often because then people with older NixOS systems can't upgrade easily anymore, see https://github.com/NixOS/nixpkgs/issues/46387

[LnL7]

Possibly also evaluate with the minver in CI.

Definitively, however in it's current form it's not really possible to use a different version depending on the target branch so changing it there would apply to everything.

[edolstra]

We should probably add some asserts to fetchurl to disallow sha256 = "<sri-hash>" and hash = "<type>:<hash>" (since the latter isn't SRI).

[abathur]

For the record, a workaround for Travis users is to add an explicit nix: 2.3.5.

FWIW, this seems to only help with linux builds. I threw a bunch of these in my build matrix for grins and no version after 2.0.4 installed without error on macOS.

nix:
  - 2.3.5
  - 2.3.4
  - 2.3.3
  - 2.2.2
  - 2.1.3
  - 2.0.4

https://travis-ci.org/github/abathur/resholved/builds/693499850

This appears to be known, at least. It took me a bit to find it, but there's ongoing/stalled work in https://github.com/travis-ci/travis-build/pull/1904 and https://github.com/travis-ci/travis-build/pull/1833

[jtojnar]

In addition to this, there are base64-encoded non-SRI hashes. Are those supported by older Nix?

[LnL7]

@jtojnar Indeed, making a follow up pr to replace those.

[jtojnar]

Ugh, then we really need https://github.com/NixOS/nix/pull/3433.

[edolstra]

That doesn't really matter, minvers.nix states 2.0 compatibility and no decision was made to drop support.

Then we should increase minvers.nix.

[LnL7]

I'm not particularly against bumping the version requirement, but the feedback on the first pr gives me the impression that there's a consensus that it's a bit early to do this. https://github.com/NixOS/nixpkgs/pull/89308#issuecomment-636927054

[abathur]

I confirmed that after #89308 my initial CI run passes again on Nix 2.0.4 (at "20.09pre228333.1ae28ebfdb7").

Given some of the other comments about additional changes, I'm not sure if I should close it yet. Thoughts?

[jonringer]

I think the original intent of this issue has been solved. However, we could have a separate ticket for preventing future breakages. I guess it depends on how we want to align the issues.

[LnL7]

With #89476 merged I would consider this resolved. For the future I created https://github.com/NixOS/nix/issues/3663, feedback/ideas welcome there.

I have no idea that it would take to update nix on travis but that's a separate problem https://docs.travis-ci.com/user/languages/nix#community-supported-warning.

[abathur]

Sounds good.

I'll ping again in travis-ci/travis-build#1904 and/or travis-ci/travis-build#1833 to pursue that issue.

[abathur]

Just in case anyone who lands here or subscribed needs to know: Nix support is working on Travis again, and the default version is updated to 2.3.6, as of about an hour ago.

[Zimmi48]

FWIW, the update in question happened there https://github.com/travis-ci/travis-build/pull/1833 and this update broke Cachix substitution, as discussed in the last comments of this PR (there's a workaround too).