Nixpkgs: unreproducible python3.7 binaries refer to /build/pip-unpacked-wheel-​j79n_99h

[raboof]

Describe the bug
As can be seen on https://r13y.com, a number of python3.7 packages are unreproducible because the .pyc binaries refer to paths like '/build/pip-unpacked-wheel-​j79n_99h', where ​j79n_99h appears to be arbitrary.

To Reproduce
For example nix-build -A python37Packages.importlib-resources --check

Expected behavior
Success ;)

Additional context
Bisecting shows this problem appears to have been introduced with #78344 /cc @jonringer

[FRidh]

Thanks for reporting. I've opened https://github.com/pypa/pip/issues/7808.

[misuzu]

Looks like this would be a long time before it would be fixed upstream.
Maybe we should just add something like this for buildPythonPackage for the time being?

# putting this in postCheck doesn't work 
postFixup = ''
  for pycache in $(find $out -name __pycache__) ; do
    rm -fr ''${pycache}
  done
'';

[FRidh]

We would also have to generate the bytecode again but in a reproducible way.

[misuzu]

We would also have to generate the bytecode again but in a reproducible way.

Something like this? :D

postFixup = ''
  for pycache in $(find $out -name __pycache__) ; do
    rm -fr ''${pycache}
  done
  python -m compileall $out
'';

This seems to be enough:

postFixup = ''
  python -m compileall -f $out
'';

[FRidh]

@misuzu yes something like that https://github.com/NixOS/nixpkgs/pull/90208

[FRidh]

Fix is now in staging.

[simonchatts]

This seems to break python3Packages.invoke.

This package includes vendored YAML packages for both Python 2 and 3, and ignores the unwanted one at build time. But the find in pythonRecompileBytecodePhase tries to compile it all, and hits syntax errors for the Python2 code.

Edit: obviously a dontUsePythonRecompileBytecode = true in that package works. But I don't know whether this is the preferred workaround.

[FRidh]

Regressions can be reported in https://github.com/NixOS/nixpkgs/issues/90516

[FRidh]

Additional change in https://github.com/NixOS/nixpkgs/pull/102222 that will be reverted because pip 20.2 resolves the issue!

Issue is now fixed with 61a04f735af6a3cdbbb6d2d234fd2dfea934fd18.

[raboof]

This is great!