Nix: Nix flakes should not cache evaluation errors

[Mic92]

Describe the bug

Currently error messages are only showed the first time they appear.
After that the error message looks like this:

$ sudo nixos-rebuild switch
building the system configuration...
error: --- EvalError -------------------------------------------------------------------------- nix
cached failure of attribute 'nixosConfigurations.turingmachine.config.system.build.toplevel.drvPath'
````

The problem is that the error message of the first time might be no longer visible,
so users would get stuck not knowing what its going on. 

**Steps To Reproduce**

1. Create an invalid nixos configuration i.e. add two conflicting options (but valid nix syntax)
2. Run `nixos-rebuild switch`:

error: --- ThrownError --------------------------------------------------------------------------- nix
The option system.nixos.versionSuffix' has conflicting definitions, in' and `'.
(use '--show-trace' to show detailed location information)

2. Run `nixos-rebuild switch` again:

building the system configuration...
error: --- EvalError -------------------------------------------------------------------------- nix
cached failure of attribute 'nixosConfigurations.turingmachine.config.system.build.toplevel.drvPath'

**Expected behavior**

It should not cache evaluation failures and show me the message agin

**`nix-env --version` output**

```console
$ nix-env --version
nix-env (Nix) 2.4pre20200721_ff314f1

[edolstra]

No, evaluation errors must be cached, because otherwise an operation like nix search nixpkgs has to evaluate stuff even in the cached case (since nixpkgs contains a lot of attributes that don't evaluate). However, we should probably store the error message in the cache (though that might require a lot of extra disk space).

[Mic92]

It also looks like evaluation errors in flake.nix files have no file name / line number.

[Mic92]

Evaluation error messages tend to be small. If no stack is included.

[edolstra]

They're pretty big, e.g.

Package ‘cudatoolkit-10.2.89’ in /home/eelco/Dev/nixpkgs-stable/pkgs/development/compilers/cudatoolkit/common.nix:214 has an unfree license (‘unfree’), refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

(use '--show-trace' to show detailed location information)

[Mic92]

How about an option that by-passes cached evaluation errors that is internally used by nixos-rebuild?

[Kloenk]

Normally that cache works just fine. Everytime I change something the cached failure is gone. The problem I have is, how do I remove the cached failure. Because when I play with the daemon I cache failures, where the fixed comes in a changed daemon and so no changes in the nix files.

[domenkozar]

What if the error message was also cached?

[edolstra]

@Kloenk Only EvalErrors should be cached, so daemon errors shouldn't be.

[Kloenk]

@Kloenk Only EvalErrors should be cached, so daemon errors shouldn't be.

I got my wopAddToStoreNar (I think it was) to fail. This made it impossible to include sources in the eval step

[edolstra]

Hm, that still shouldn't be cached because such an error is not a subclass of EvalError.

[Kloenk]

The daemon did not response with an error to addToStore, so the eval failed with no such file or directory

[colemickens]

Can we --force in the meantime? Or is there already such a feature I've missed? It's a daily occurrence that I add a "1" somewhere to bust the cache, and similarly frequent that I forget to remove it.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flakes-with-unfree-licenses/9405/1