Nixpkgs: Infinite recursion in Haskell overlays for some packages

[Ptival]

Describe the bug
Under nixos-unstable, as of at most a couple days ago, overlays that used to work suddenly undergo infinite recursion.

To Reproduce
Steps to reproduce the behavior:

1. Try and build this shell

{ nixpkgs ? import <nixpkgs> {
  overlays =
    [
      (self: super: {
        haskellPackages = super.haskellPackages.extend(selfH: superH: rec {
          time-compat =
            super.haskellPackages.callCabal2nix
              "time-compat"
              (super.fetchFromGitHub
                {
                  owner = "phadej";
                  repo = "time-compat";
                  rev = "822e39074222ee01222c515a447c62a7df611ca3";
                  sha256 = "1di817ayj4xz43l3msahp8lxzf3slqmm8jdslcq6cik2m0ci962a";
                })
              {}
          ;
        });
      })
    ];
}
}:
nixpkgs.mkShell {
  buildInputs = [
    nixpkgs.haskellPackages.ghcide
  ];
}

Expected behavior
It used to work, and now it says infinite recursion. I'm not sure whether it was never supposed to work, but I don't think I'm doing anything wrong.

Works on nixos-20.03
Fails on nixos-unstable (as of recently)

I'm going to try and bisect the issue now.

[Ptival]

Ok, this happened with commit 2645e1a1eb41742905394db9d7679a1bb3343eeb so I guess @madjar might know?

[cdepillabout]

I believe this is happening because cabal2nix itself transitively depends on time-compat, so overriding time-compat to _use_ cabal2nix causes cyclic dependencies.

An easy fix is just to get cabal2nix from a separate package set:

diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 68f16bae741..822df0de4a9 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -1351,7 +1351,7 @@ in

   cue2pops = callPackage ../tools/cd-dvd/cue2pops { };

-  cabal2nix-unwrapped = haskell.lib.justStaticExecutables (haskell.lib.generateOptparseApplicativeCompletion "cabal2nix" haskellPackages.cabal2nix);
+  cabal2nix-unwrapped = haskell.lib.justStaticExecutables (haskell.lib.generateOptparseApplicativeCompletion "cabal2nix" haskell.packages.ghc882.cabal2nix);

   cabal2nix = symlinkJoin {
     inherit (cabal2nix-unwrapped) name meta;

---

@madjar In https://github.com/NixOS/nixpkgs/pull/81125#issuecomment-591480366 we were discussing why cabal2nix was being pulled from a separate package set. I guess this is the reason? If cabal2nix is pulled from a different package set, then it won't cause these circular dependencies.

Could you send a PR fixing this? (Probably just applying my patch from above...)

[Ptival]

Maybe also add a comment above that line so that the next person knows not to change it! :-)

[madjar]

@cdepillabout Correct me if I'm wrong, but using a different package set doesn't really solve the problem, it just moves it to a different package set (which is less used, so the problem is less likely to trigger). An actual solution (that has the benefit of not bringing another ghc) would be to have cabal2nix come from the non-overriden package set. Is such a thing possible?

[cdepillabout]

@madjar

using a different package set doesn't really solve the problem, it just moves it to a different package set (which is less used, so the problem is less likely to trigger)

That is correct.

An actual solution (that has the benefit of not bringing another ghc) would be to have cabal2nix come from the non-overriden package set. Is such a thing possible?

This is also correct. And it should be possible, but I don't believe we have any code currently doing something like this.

---

Could you first send a PR just changing cabal2nix to come from a different package set (basically anything other than haskellPackages)? I'd like to merge this in as soon as possible so that the chance people run into these infinite recursion issues is lowered. You could use something like my diff from above.

After we merge that in, if you'd like to come up with a cleaner solution that creates a separate package set specifically for cabal2nix, we could evaluate whether we want to use that here in nixpkgs.

[JBetz]

You should also be able to workaround this by creating a separate namespace for your overridden Haskell package set, so that cabal2nix can use the old one while your stuff uses the new one.

E.g.,

let
  config = {
    packageOverrides = pkgs: rec {
      overriddenHaskellPackages = pkgs.haskellPackages.override rec {
        overrides = new: old: rec {
          semigroups = new.callCabal2nix "semigroups" (pkgs.fetchFromGitHub
            {
              owner = "ekmett";
              repo = "semigroups";
              rev = "2c662cd60b49927d1ac76c657bc0bf0350869386";
              sha256 = "18hm4sykwf89z63299drq23ybl8qm5zbcs5a35bcviqcxkayq6c4";
            }) {};
        };
      };
    };
  };
  pkgs = import <nixpkgs> { inherit config; };
in
  pkgs.mkShell {
    inputsFrom = [
      pkgs.overiddenHaskellPackages.semigroups
    ];
  }

If you replace overridenHaskellPackages with haskellPackages it will throw the infinite recursion error just like the example.