Nixpkgs: cargoSha256 mismatch when it shouldn't - mismatches on it's own suggested sha

[06kellyjac]

Describe the bug
I'm packaging up a rust package & its complaining about the cargoSha256 mismatching.

I've built it with cargoSha256 = "0000000000000000000000000000000000000000000000000000"; to get the updated sha.

hash mismatch in fixed-output derivation '/nix/store/6sih2rlvwrq78kznws964bw8qck10216-hors-0.6.2-vendor.tar.gz':
  wanted: sha256:0000000000000000000000000000000000000000000000000000
  got:    sha256:14i18vngfn27g006rakhfz6ajvmi5b8rxmy016i23kb42gya0xyi

Setting it to cargoSha256 = "14i18vngfn27g006rakhfz6ajvmi5b8rxmy016i23kb42gya0xyi"; still results in out of date errors

Validating consistency between /build/source/Cargo.lock and /build/hors-0.6.2-vendor.tar.gz/Cargo.lock
llvm-diff: /build/source/Cargo.lock:1:1: error: expected top-level entity
# This file is automatically @generated by Cargo.
^
llvm-diff: /build/hors-0.6.2-vendor.tar.gz/Cargo.lock:1:1: error: expected top-level entity
# This file is automatically @generated by Cargo.
^

ERROR: cargoSha256 is out of date

Cargo.lock is not the same in hors-0.6.2-vendor.tar.gz

To fix the issue:
1. Use "0000000000000000000000000000000000000000000000000000" as the cargoSha256 value
2. Build the derivation and wait it to fail with a hash mismatch
3. Copy the 'got: sha256:' value back into the cargoSha256 field

A diff & sha256sum of both the build source Cargo.lock & vendor Cargo.lock is the same

λ sha256sum /nix/store/iqqk3d3dmc3gby2iy41n9dqkk9k6sz4h-source/Cargo.lock
ff6fe0e5809ad367d44242b67d988e2ed8ef3d9b3ea78a4c601d78843b358e43  /nix/store/iqqk3d3dmc3gby2iy41n9dqkk9k6sz4h-source/Cargo.lock

λ sha256sum ./hors-0.6.2-vendor.tar.gz/Cargo.lock
ff6fe0e5809ad367d44242b67d988e2ed8ef3d9b3ea78a4c601d78843b358e43  ./hors-0.6.2-vendor.tar.gz/Cargo.lock

To Reproduce
Steps to reproduce the behavior:

1. Use the below nix snippets
2. run with cargoSha256 of 0s
3. run with the new cargoSha256 after being prompted
4. see errors
5. double check both Cargo.lock files for differences

{ stdenv, fetchFromGitHub, rustPlatform, llvmPackages, pkg-config, openssl }:

rustPlatform.buildRustPackage rec {
  pname = "hors";
  version = "0.6.2";

  src = fetchFromGitHub {
    owner = "WindSoilder";
    repo = pname;
    rev = "v${version}";
    sha256 = "1ssiflkaclcql41prrmyg3ndpnykhxnysj1zq3v5rv023bvsf50m";
  };

  nativeBuildInputs = [
    llvmPackages.libclang
    llvmPackages.clang
    llvmPackages.bintools
    pkg-config
  ];

  buildInputs = [
    openssl
  ];

  # cargoSha256 = "0000000000000000000000000000000000000000000000000000";
  cargoSha256 = "14i18vngfn27g006rakhfz6ajvmi5b8rxmy016i23kb42gya0xyi";

  meta = with stdenv.lib; {
    homepage = https://github.com/WindSoilder/hors/;
    description = "Instant coding answers via the command line (howdoi in rust) ";
    license = licenses.gpl3;
  };
}

{ config, pkgs, lib, ... }:

let
  hors = pkgs.callPackage ./pkgs/hors { };
  # ...
in {
  # ...
  home.packages = with pkgs; [
    # ...
    hors
  ];
  # ...
}

Expected behavior
When I put the updated cargoSha256 in "got:" it shouldn't then complain that same SHA is out of date

Screenshots
See snippets

Additional context

Notify maintainers

@bhipple @jonringer @zimbatm

Metadata

 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.35, NixOS, 20.09pre223023.fce7562cf46 (Nightingale)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.4`
 - channels(root): `"home-manager, nixos-20.09pre223023.fce7562cf46, nixos-hardware"`
 - channels(username): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Thank you for reading this wall of text :smiley:
If you need more details let me know
Kind regards

Maintainer information:

[jonringer]

I'll defer to @bhipple

[bhipple]

Thanks for the detailed bug report with the easy reproduce! This is indeed a bug in the Rust builder.

The issue is that the builder is just using diff at the cmdline, without specifying whose diff. Your build input is causing it to find llvmPackages.binutils's diff instead of the standard unix utility.

A workaround you can do for your package now is to add diffutils to the front of your nativeBuildInputs.

As an extra tip, when you want to build Rust with clang, you need to set LIBCLANG_PATH. A fully working expression for your package is thus:

{ stdenv, fetchFromGitHub, rustPlatform, llvmPackages, pkg-config, openssl, diffutils }:

rustPlatform.buildRustPackage rec {
  pname = "hors";
  version = "0.6.2";

  src = fetchFromGitHub {
    owner = "WindSoilder";
    repo = pname;
    rev = "v${version}";
    sha256 = "1ssiflkaclcql41prrmyg3ndpnykhxnysj1zq3v5rv023bvsf50m";
  };

  cargoSha256 = "14i18vngfn27g006rakhfz6ajvmi5b8rxmy016i23kb42gya0xyi";

  nativeBuildInputs = [
    # TODO: Workaround for llvmPackages.bintools shadowing `diff`. Remove once
    # buildRustPackage is fixed.
    diffutils
    llvmPackages.libclang
    llvmPackages.clang
    llvmPackages.bintools
    pkg-config
  ];

  buildInputs = [
    openssl
  ];

  LIBCLANG_PATH = llvmPackages.libclang + "/lib";

  meta = with stdenv.lib; {
    homepage = "https://github.com/WindSoilder/hors/";
    description = "Instant coding answers via the command line (howdoi in rust) ";
    license = licenses.gpl3;
  };
}

(Well, fully working except that the test cases fail!)

The systemic fix is going to have to work its way through staging, it's a change to the Rust package builder and will cause a lot of recompilation, but you should be unblocked with the above.

[06kellyjac]

Thanks for the help!

I pulled clang in because one of the packages wouldnt build without the lib. Thanks for the advice on how to do it properly

[06kellyjac]

The fix is working well. The hors tests don't seem to like the readonly file system used for build, and even in a normal dir the tests are pretty temperamental right now and fail like 50% of the time :sweat_smile:

Shall I keep this open until the fix hits my machine on the unstable channels or close it now since there's been a fix merged?

[jonringer]

It's only been merged into staging, it will be a while for it to appear in master