Nixpkgs: haskell-cryptonite does not work on older CPUs

[peti]

The Haskell package cryptonite uses the AES-NI CPU extension [1] by default. This means that the binary won't work properly on older CPUs that don't support that extension. We actually have that problem with some of Hydra's build slaves. Currently, we see occasional test suite failures [2] when the build is scheduled to a build slave that lacks support for AES-NI, because the test suite runs a "broken" binary on that machine.

How should we remedy that issue? There are two obvious choices:

1. Disable AES-NI support by default to make sure our binaries works on all CPUs, even older ones. This comes at the expense of performance.
2. Keep AES-NI enabled. This means we'll have full performance, but the binary will crash on older CPUs, probably without a good explanation of what is going on.

Which one is preferable? Are there other options?

[1] https://en.wikipedia.org/wiki/AES_instruction_set
[2] https://github.com/haskell-crypto/cryptonite/issues/314

[turboMaCk]

I personally like stuff to be as optimal as it can be by default. In this case, despite obvious complications, I think we should stick to building faster binary that works on most machines. That way all users unaware of this issue will get the fast version which will most likely work for them. For users with older CPUs, they will need to opt-out from optimization and probably rebuild a lot of packages but at least it would be explicit action one takes so such users will be aware of the trade-off.

If we decide to go with disabling AES-NI I will personally define overlay. Not because I necessary believe this would have a great impact on performance but because I don't want to have this as a variable when ever I'm debugging performance related issues.

As for the hydra specifically, I think this is unfortunate but there likely be similar issues with packages outside of Haskell set as well. Maybe we can bring more maintainers together and find some general approach that works for all such issues?

[codygman]

I personally like stuff to be as optimal as it can be by default. In this case, despite obvious complications, I think we should stick to building faster binary that works on most machines

EDIT: whoops, realized I wasn't on the cryptonite repo but in nixpkgs. Oh well, question is still there but maybe I should copy in the linked issue.

Can we conditionally disable AES-NI with CPP?

I think @ocheron here implies we can "remove global cc-options like -mssse3" if the CPU doesn't support them, presumably with CPP?

[cdepillabout]

Can we conditionally disable AES-NI with CPP?

I don't think this will necessarily help with Nixpkgs.

If a hydra builder builds it with AES-NI and that package gets into the shared nix cache, users without recent CPUs will get hard-to-debug errors when trying to use it.

However, if a hydra builder builds it witout AES-NI and that package gets into the shared nix cache, users with recent CPUs will have worse performance.

Nixpkgs should probably default to either disabling or enabling it by default, not letting it be decided by the CPU on the hydra builder that just happens to get to it.

---

Here are a couple other packages I found by grepping that deal with AES-NI:

• https://github.com/NixOS/nixpkgs/blob/4f69262c190845ca36de255e857d8b54e2faf4b4/pkgs/tools/misc/cpuminer-multi/default.nix#L2

  This defaults to compiling with aes-ni support.

• https://github.com/NixOS/nixpkgs/blob/4f69262c190845ca36de255e857d8b54e2faf4b4/pkgs/tools/networking/i2pd/default.nix#L4

  This defaults to compiling without aes-ni support.

• https://github.com/NixOS/nixpkgs/blob/4f69262c190845ca36de255e857d8b54e2faf4b4/pkgs/tools/networking/strongswan/default.nix#L75

  This enables aes-ni if on x86_64.

[volth]

If the option can configured only at compile time, the natural solution would be a condition on nixpkgs.localSystem.gcc.arch

[volth]

Nixpkgs should probably default to either disabling or enabling it by default,

nixpkgs default should't rely on advanced CPU features, there are still tons of old CPU in the wild.

It is not only the old hydra builders.
"QEMU Virtual CPU version 2.5+" - the default qemu's (and most cloud's) CPU does not support AES-NI

[stale[bot]]

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

[cdepillabout]

Still relevant, although I still haven't heard of any sort of policy in nixpkgs about what do to for these CPU flags by default.