Nixpkgs: Does runInLinuxVM really need "kvm" in systemFeatures?

[christian-marie]

I'm trying to build docker images on a hydra build server that doesn't have KVM support; the build job is never scheduled as "kvm" is propagated as a requiredSystemFeature via dockerTools.buildImage.runAsRoot: https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/vm/default.nix#L302

One year ago the same build server was successfully building .ovf artefacts, which I assume used runInLinuxVM; I suspect that qemu falls back to emulation if missing KVM support.

I can successfully build the docker layers locally after removing all kvm kernel modules and removing /dev/kvm. This was surprising to me as I was under the impression that the systemFeatures came from here: https://github.com/NixOS/nix/blob/fb35aaa42295940ca8dbfef17b081729636f9775/src/libstore/globals.cc#L101

Nevertheless, this local test along with the previous year's successful .ovf build leades me to believe that the KVM requirement may be superflous. Is there some way I can work around this without patching build-support to remove the requirement?

[eadwu]

You can probably do another lib.overrideDerivation on the result of runInLinuxVM to override requiredSystemFeatures and args. The reason why kvm support is needed (from what I've seen) is due to the fact qemuCommandLinux [1] will end up calling -enable-kvm [2].

[1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/vm/default.nix#L195
[1] https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/qemu-flags.nix#L19

[christian-marie]

Thanks for the helpful suggestion.

I tried this, and could indeed remove kvm as a requiredSystemFeature from the top-level derivation returned from dockerTools.buildImage.

Unfortunately that derivation still depends on a separate layer as the call to runInLinuxVM is nested [1]. I don't know of a way to override any "deeper".

Would it make sense to make the usage of kvm optional and configurable? I could produce a patch given some guidance on the user interface.

[1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/docker/default.nix#L164

[eadwu]

If you were to fix it without touching nixpkgs, you can copy the file over and fix the runInLinuxVM call there while redefining runWithOverlay = (import ./file.nix {}).runWithOverlay;.

Patching it would probably need an arg (something like enableKVM) propagated throughout all the calls which defaults to true to maintain the current behavior.

[christian-marie]

Again, thanks. I'll try to use this approach as a temporary workaround.

It seems [1] that @edolstra added kvm as a requiredSystemFeature almost a decade ago. I wonder if anyone remembers why. If there are no further insights then I plan to return with a patch that retains the current behaviour by default.

[1] https://github.com/NixOS/nixpkgs/commit/3cdc3c4abe820380ed0f66a87123d58589f67cf1

[7c6f434c]

It seems [1] that @edolstra added kvm as a requiredSystemFeature almost a decade ago. I wonder if anyone remembers why.

Well, if a builder set contains both KVM-capable and non-KVM-capable ones, the performance benefit of using the former is clear.

[christian-marie]

Well, if a builder set contains both KVM-capable and non-KVM-capable ones, the performance benefit of using the former is clear.

Yes, no argument there - I still want to use KVM where avaliable. I'm questioning if kvm in requiredSystemFeatures is needed for kvm support. I suspect oppourtunistic kvm use with a graceful fallback is possible. I'm probably missing something.

[7c6f434c]

I'm questioning if kvm in requiredSystemFeatures is needed for kvm support.

I don't think Nix has support for picking KVM-capable builders if they exist and ignoring the requirement otherwise.

[christian-marie]

Right, I'm making the assumption that the results are identical for emulated vs kvm; this may not hold in the presence of bugs so I'm behind making this an explicit builder option that changes the output hash. I'd lose the oppourtunistic usage of kvm, but it's a reasonable trade-off for better determinism guarantees.

I've just tested removing the requiredSystemFeature on hydra via importing a customised build-suport/docker, like so:

  runWithOverlay = x: lib.overrideDerivation (runWithOverlay' x) (a: { requiredSystemFeatures = []; });

And it works, so I'll be back with a patch. Thanks to all for the input.

[sokoow]

@christian-marie did you get anywhere patching this ? I'm also trying to build docker images without kvm and found your ticket, also hit the same wall I guess..

[christian-marie]

Hi, @sokoow I'm still using the locally patched version for our build; I haven't gotten around to making a patch, sorry.

[CMCDragonkai]

@christian-marie Can you show your patch here? We'd also like to make use of this.

Specifically I want to know what to override to remove the kvm requirement.

[fiksn]

This is quite annoying if you are using dockerTools.buildImage.runAsRoot in a pipeline that is being run on Travis. Apparently a few years ago everything worked fine, now it just fails due to missing server with KVM.

Patching it would probably need an arg (something like enableKVM) propagated throughout all the calls which defaults to true to maintain the current behavior.

I believe either this issue should be reopened or a new created to track that this should eventually be fixed. Perhaps you should have to explicitly set some flag but the end result should be that you can use runAsRoot without requiring KVM on the machine.