Nix could support evaluating attribute sets more lazily, such that e.g. the following works:
(throw "" // { x = 0; }).x
-> 0
Currently this doesn't work because attribute sets are evaluated strictly in all names, meaning all //
operations and co. will have to be applied to figure out all names, making the above code throw in the current version.
Making attribute names lazy in that way has the potential to solve many existing problems:
.extend
attributes of package scopes won't necessarily have to evaluate all package attributes. Similarly .override
, etc. of packages won't have to call the derivation
primop multiple times anymoretypes.lazyAttrsOf
might not be needed anymoreself.lib
for the attribute definitions if lib
is added in a later overlaylib.mkIf
might not be necessary for the module system anymoreHowever this also changes semantics, so this should be thought out well.
I have implemented an initial prototype that supports above evaluation in https://github.com/Infinisil/nix/commit/afeda4fda1214f285083acad64e1d5425e8b42cb. I'll continue with this if I have time.
Ping @edolstra @Profpatsch
Gave this another implementation try in https://github.com/Infinisil/nix/commit/5dbf67cab037b3bb8302052fdda6534e5cf23d9b, which works much better than the previous one. For example:
let
lib = import ./lib;
set1 = lib.makeExtensible (self: builtins.trace "set1" {
foo = 1;
});
set2 = set1.extend (self: super: builtins.trace "set 2" {
bar = 2;
});
set3 = set2.extend (self: super: builtins.trace "set 3" {
baz = 3;
});
set4 = set3.extend (self: super: builtins.trace "set 4" {
qux = 4;
});
in set4
Evaluating this in a current Nix version would print
trace: set1
trace: set1
trace: set 2
trace: set1
trace: set 2
trace: set 3
trace: set1
trace: set 2
trace: set 3
trace: set 4
{ __unfix__ = <LAMBDA>; bar = 2; baz = 3; extend = <CODE>; foo = 1; qux = 4; }
Whereas with above commit it just prints:
trace: set1
trace: set 2
trace: set 3
trace: set 4
{ __unfix__ = <LAMBDA>; bar = 2; baz = 3; extend = <CODE>; foo = 1; qux = 4; }
Therefore not evaluating all the overlays multiple times! Unfortunately there is a bug with this implementation which causes it to go into an infinite loop with nixpkgs derivations. Also, this implementation copy-pasted a bunch of code around, and probably leaks some memory and does additional unnecessary allocations here and there. Definitely not ready, but it's getting there :)
Turns out there was no infinite recursion in the last version, but instead it was so slow that I didn't think it would ever finish!
I worked more on this, with the latest version being in https://github.com/Infinisil/nix/tree/lazy-attr-names-v3. It's still about 40% slower than stable (tested on a firefox instantiation), but it's usable now! There are some other things to do though. I still have high hopes that it can improve the speed of Nix in general, though it might be pretty hard to get there.
Opened a draft PR to track further development: https://github.com/NixOS/nix/pull/4154
Most helpful comment
Turns out there was no infinite recursion in the last version, but instead it was so slow that I didn't think it would ever finish!
I worked more on this, with the latest version being in https://github.com/Infinisil/nix/tree/lazy-attr-names-v3. It's still about 40% slower than stable (tested on a firefox instantiation), but it's usable now! There are some other things to do though. I still have high hopes that it can improve the speed of Nix in general, though it might be pretty hard to get there.