Nixpkgs: VirtualBox: Virtual machines fail to start.

[ambrop72]

Describe the bug

On the latest nixos-unstable (b5f5c97f7d67a99b67731a8cfd3926f163c11857), starting any virtual machine gives me error:

The virtual machine 'Fedora' has terminated unexpectedly during startup because of signal 6.
Result Code: | NS_ERROR_FAILURE (0x80004005)
Component: | MachineWrap
Interface: | IMachine {5047460a-265d-4538-b23e-ddba5fb84976}

To Reproduce
Steps to reproduce the behavior:

1. Use VirtualBox on nixos-unstable
2. Start a virtual machine

Screenshots

Additional context
Seems to have been caused by updating.

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: linuxPackages.virtualbox
# a list of nixos modules affected by the problem
module: virtualbox

[oxalica]

I've met the same issue with "signal 6". Rebooting does not help.

But I can run VM with --type=headless successfully. Seems there is something wrong with display.

[flokli]

I digged a bit.

Virtualbox seems to have problems when loading VBoxSharedCrOpenGL:

#0  0x00007fbbd957ba0f in crHashtableWalkUnlocked (hash=0x0, walkFunc=walkFunc@entry=0x7fbbd954c3a2 <renderspuVBoxCompositorClearAllCB>, dataPtr2=dataPtr2@entry=0x0) at /build/VirtualBox-6.0.8/src/VBox/GuestHost/OpenGL/util/hash.c:497
#1  0x00007fbbd954c526 in renderspuVBoxCompositorClearAll () at /build/VirtualBox-6.0.8/src/VBox/HostServices/SharedOpenGL/render/renderspu.c:1139
#2  0x00007fbbd954dd6b in renderspuCleanupBase (fDeleteTables=fDeleteTables@entry=true) at /build/VirtualBox-6.0.8/src/VBox/HostServices/SharedOpenGL/render/renderspu_init.c:491
#3  0x00007fbbd954dee3 in renderSPUCleanup () at /build/VirtualBox-6.0.8/src/VBox/HostServices/SharedOpenGL/render/renderspu_init.c:536
#4  0x00007fbbd967c4d8 in crSPUUnloadChain (headSPU=headSPU@entry=0x7fbbb4130180) at /build/VirtualBox-6.0.8/src/VBox/GuestHost/OpenGL/spu_loader/spuload.c:283
#5  0x00007fbbd967c744 in crSPULoad (child=child@entry=0x0, id=0, name=0x7fbbd968eb8f "render", dir=dir@entry=0x0, server=server@entry=0x7fbbd96cc680 <cr_server>) at /build/VirtualBox-6.0.8/src/VBox/GuestHost/OpenGL/spu_loader/spuload.c:166
#6  0x00007fbbd967c806 in crSPULoadChain (count=count@entry=1, ids=ids@entry=0x7fbbd975718c, names=names@entry=0x7fbbd9757180, dir=dir@entry=0x0, server=server@entry=0x7fbbd96cc680 <cr_server>) at /build/VirtualBox-6.0.8/src/VBox/GuestHost/OpenGL/spu_loader/spuload.c:201
#7  0x00007fbbd95e3fbf in crServerSetVBoxConfigurationHGCM () at /build/VirtualBox-6.0.8/src/VBox/HostServices/SharedOpenGL/crserverlib/server_config.c:327
#8  0x00007fbbd95cc904 in crVBoxServerInit () at /build/VirtualBox-6.0.8/src/VBox/HostServices/SharedOpenGL/crserverlib/server_main.c:582
#9  0x00007fbbd95bbb66 in VBoxHGCMSvcLoad (ptable=<optimized out>) at /build/VirtualBox-6.0.8/src/VBox/HostServices/SharedOpenGL/crserver/crservice.cpp:1562
#10 0x00007fbbdbe2ea7b in HGCMService::loadServiceDLL (this=this@entry=0x7fbbb80037f0) at /build/VirtualBox-6.0.8/src/VBox/Main/src-client/HGCM.cpp:348
#11 0x00007fbbdbe2ecb4 in hgcmServiceThread (pThread=0x7fbbb8003950, pvUser=0x7fbbb80037f0) at /build/VirtualBox-6.0.8/src/VBox/Main/src-client/HGCM.cpp:637
#12 0x00007fbbdbe2ceaf in hgcmWorkerThreadFunc (hThreadSelf=<optimized out>, pvUser=0x7fbbb8003950) at /build/VirtualBox-6.0.8/src/VBox/Main/src-client/HGCMThread.cpp:200
#13 0x00007fbbea3e0146 in rtThreadMain (pThread=pThread@entry=0x7fbbb8003b80, NativeThread=NativeThread@entry=140444783970048, pszThreadName=pszThreadName@entry=0x7fbbb8004460 "ShCrOpenGL") at /build/VirtualBox-6.0.8/src/VBox/Runtime/common/misc/thread.cpp:719
#14 0x00007fbbea48a619 in rtThreadNativeMain (pvArgs=0x7fbbb8003b80) at /build/VirtualBox-6.0.8/src/VBox/Runtime/r3/posix/thread-posix.cpp:327
#15 0x00007fbbea763ef7 in start_thread () from /nix/store/n2wvg362fsilqh3z3262argzxk681qpz-glibc-multi-2.27/lib/libpthread.so.0
#16 0x00007fbbea69422f in clone () from /nix/store/n2wvg362fsilqh3z3262argzxk681qpz-glibc-multi-2.27/lib/libc.so.6

I guessed it might be opengl not finding mesa or some x11 libraries, so I tried adding addOpenGLRunpath $out/libexec/virtualbox/VBoxSharedCrOpenGL.so as described in https://github.com/NixOS/nixpkgs/pull/60985 but seems this wasn't enough, and I'm not sure if that's the cause of the bug at all.

There's more libraries that seem to run dlopen, which might need patching too…

I'll try to bisect what broke it, maybe this will help tracking down what caused it.

[flokli]

I finished bisecting, and found the following commit causing it to break:

f79fd2e826dd95b3b64839d3e0bec8ae1dfab17e is the first bad commit
commit f79fd2e826dd95b3b64839d3e0bec8ae1dfab17e
Author: Thomas Tuegel <[email protected]>
Date:   Fri Jul 5 10:41:41 2019 -0500

    wrapQtAppsHook: wrap Qt applications for runtime dependencies

:040000 040000 a23c095c35d4c774f7ee64d5262888ead078a784 097d2fc2a69cf74da570a0371fd2cfb647ed60fc M nixos
:040000 040000 dca1fe8cd6b2e856fe9ab6136f0814ecb2d78316 5e6859cc779955cf034a32979bca4f314e9b0970 M pkgs
bisect run success

[root@flokli:~/nixpkgs]# git bisect log
git bisect start
# bad: [984851a9bfa3a7b5dacb436d7686f2f09b5e2e85] Merge pull request #66509 from emilazy/add-git-revise
git bisect bad 984851a9bfa3a7b5dacb436d7686f2f09b5e2e85
# good: [72216e12de053dc488707fb070e56047b5c91020] Merge #62393: virtualboxHeadless: Fix build
git bisect good 72216e12de053dc488707fb070e56047b5c91020
# good: [d0453ef9ce9f14e6f54a1a19ef3101d64d2f4240] bcachefs: 2019-07-13 (#64732)
git bisect good d0453ef9ce9f14e6f54a1a19ef3101d64d2f4240
# bad: [0a1bf4734fb93ebc7d2240a9d98babdacfcf6100] libtensorflow: add binary build and add automatic generation
git bisect bad 0a1bf4734fb93ebc7d2240a9d98babdacfcf6100
# bad: [8e04c3cbf10a206ad6a3a01d70725a281924c1f3] lolcat: 99.9.69 -> 99.9.99
git bisect bad 8e04c3cbf10a206ad6a3a01d70725a281924c1f3
# bad: [d7cdd895fa65334dc063ae6cd2e01d48e962bc3b] rsyslog: 8.1905.0 -> 8.1907.0
git bisect bad d7cdd895fa65334dc063ae6cd2e01d48e962bc3b
# bad: [325d2e2352c621c227edcbf6217eb5935261cc0f] Merge pull request #64912 from r-ryantm/auto-update/python3.7-texttable
git bisect bad 325d2e2352c621c227edcbf6217eb5935261cc0f
# good: [713a45ecf79a4b4c632819f1c898d3e66c77bdd2] python37Packages.Wand: 0.5.4 -> 0.5.5 (#64914)
git bisect good 713a45ecf79a4b4c632819f1c898d3e66c77bdd2
# good: [066491c2e18d6277e0765a3647068c93650fcd19] Merge pull request #63999 from dingxiangfei2009/appamor-cross-compile
git bisect good 066491c2e18d6277e0765a3647068c93650fcd19
# bad: [5bf68e1354d2a6ad3663c829675ddf90b6996e24] Merge #64742: firefox 67 -> 68, and related updates
git bisect bad 5bf68e1354d2a6ad3663c829675ddf90b6996e24
# bad: [6049c39bdb3d8bfc01e2a86d2fc6ef6e46f9b0be] keybase: fix eval after merge mistake
git bisect bad 6049c39bdb3d8bfc01e2a86d2fc6ef6e46f9b0be
# bad: [a88e319591f072afd109239237e5f3927654dc74] python36: 3.6.8 -> 3.6.9
git bisect bad a88e319591f072afd109239237e5f3927654dc74
# bad: [3adc9d04870605ca20969a98fe820535fc7a88a5] doc/languages-frameworks/qt.xml: Update for wrapQtAppsHook
git bisect bad 3adc9d04870605ca20969a98fe820535fc7a88a5
# bad: [51d78034a1db78f8ce0ea3ba6fa20be590ed0ca2] wrapQtAppsHook: Remove ad hoc Qt wrappers
git bisect bad 51d78034a1db78f8ce0ea3ba6fa20be590ed0ca2
# bad: [f79fd2e826dd95b3b64839d3e0bec8ae1dfab17e] wrapQtAppsHook: wrap Qt applications for runtime dependencies
git bisect bad f79fd2e826dd95b3b64839d3e0bec8ae1dfab17e
# first bad commit: [f79fd2e826dd95b3b64839d3e0bec8ae1dfab17e] wrapQtAppsHook: wrap Qt applications for runtime dependencies

I made sure to only mark commits bad that were really failing the virtulabox test with the specific error message posted above by using the following script for git bisect run:

#!/usr/bin/env bash
nixBuild() {
  DRV=$(nix-instantiate "$@");
  nix-build $DRV --builders "" --no-out-link;
  S=$?;
  if [ $S -ne 0 ]; then
    # trigger a git bisect bad if the bad line was found
    nix log $DRV | grep -i "has terminated unexpectedly during startup" && exit 42;
    # else trigger a skip (some other random build failure)
    exit 125;
  fi;
  # build was successful
  exit 0
}
nixBuild nixos/tests/virtualbox.nix -A simple-cli

So it's indeed some problems in how qt applications are wrapped cc @ttuegel / https://github.com/NixOS/nixpkgs/issues/65399

[flokli]

Wrapping simply all qt binaries via wrapQtAppsHook won't work, as there's some suid wrappers being used.

[worldofpeace]

@flokli I did https://github.com/NixOS/nixpkgs/commit/2bd649b152895199fb5cc22a715f1ef5a3c34330 recently.
Would that change the current state of things, or did you test with this commit?

[flokli]

@worldofpeace See the bisect log - I tested with latest master from some hours ago (was at 984851a9bfa3a7b5dacb436d7686f2f09b5e2e85 at this time, so your commit was included) - it didn't fix that failure.

[samueldr]

Had a look, see what the issue could be. I think the issue is that VirtualBoxVM (maybe among others) in libexec would need to be wrapped... Except that wrapping them breaks them.

Here's why I think it's related:

@santaslittlehelper ~ $ VirtualBoxVM --startvm "Tiny Core Linux"
Qt WARNING: Could not find the Qt platform plugin "xcb" in ""
Qt FATAL: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Aborted

---

diff --git a/pkgs/applications/virtualization/virtualbox/default.nix b/pkgs/applications/virtualization/virtualbox/default.nix
index 1a6ba5ac527..836c5e5b65a 100644
--- a/pkgs/applications/virtualization/virtualbox/default.nix
+++ b/pkgs/applications/virtualization/virtualbox/default.nix
@@ -183,6 +183,7 @@ in stdenv.mkDerivation {

   preFixup = optionalString (!headless) ''
     wrapQtApp $out/bin/VirtualBox
+    wrapQtApp $out/libexec/virtualbox/VirtualBoxVM
   '';

   passthru = {

Once wrapped, the suid wrapper seems to not work as expected

@santaslittlehelper ~ $ VirtualBoxVM --startvm "Tiny Core Linux"
VirtualBoxVM: Error -10 in SUPR3HardenedMain!
VirtualBoxVM: Effective UID is not root (euid=1000 egid=100 uid=1000 gid=100)

VirtualBoxVM: Tip! It may help to reinstall VirtualBox.

And, just as a test, tried to run the wrapped binary through sudo:

@santaslittlehelper ~ 1 $ sudo /nix/store/84kwxvr5wv7ikfb53ywynwvl435jvic5-virtualbox-6.0.8/libexec/virtualbox/VirtualBoxVM
[sudo] password for samuel:
VirtualBoxVM: supR3HardenedVerifySameFile: "/nix/store/84kwxvr5wv7ikfb53ywynwvl435jvic5-virtualbox-6.0.8/libexec/virtualbox/.VirtualBoxVM-wrapped" isn't the same as "/nix/store/84kwxvr5wv7ikfb53ywynwvl435jvic5-virtualbox-6.0.8/libexec/virtualbox/VirtualBoxVM"

So, there's a bit more to it than simply wrapping it.

[ambrop72]

Does simply not wrapping it make it work? If not, why did it work before?

[ambrop72]

Anyway, I think that we need to make sure that VirtualBoxVM is not wrapped, and hack it however we need to make it run, e.g. by inserting calls to QCoreApplication::addLibraryPath to main for all plugins that are needed.

[worldofpeace]

Last part where you ran it with sudo appears to trigger a check in a hardening function which obviously finds our shell wrapper suspicious.

@ambrop72 It does appear that in this case a patch may also be needed.

[samueldr]

Does simply not wrapping it make it work? If not, why did it work before?

There recently were changes in how Qt applications are packaged. The change (or an equivalent change) is required to fix an issue with mixed Qt applications in a user's environment. The issue this fixes is way worse (and hard to debug) than the issues it causes in the end.

Why did it work before? It worked because Qt would discover libraries (e.g. .../lib/qt-.../plugins/platforms/libqxcb.so) by peeking at any paths in the PATH, which could in theory bring the wrong versions in, e.g. versions from another incompatible Qt. The fixes makes all checks more strict, which breaks what seemed to work flawlessly beforehand. Most packages don't end up sanitizing the environment for Qt at runtime, though it looks like this is what Qt does (and rightly so due to hardening).

---

Anyway, I think that we need to make sure that VirtualBoxVM is not wrapped, and hack it however we need to make it run, e.g. by inserting calls to QCoreApplication::addLibraryPath to main for all plugins that are needed.

It would work, though the patch would need to be dynamic, as the full library paths would need to be used.

Adding to Qt's library path is what #44047 was doing to solve the issue. (Though it was doing so at initialization-time, by looking into a "registry" file of paths to add for the derivation.)

[ambrop72]

Testing this might be easier when setting virtualisation.virtualbox.host.enableHardening = false, (e.g. you can gdb it and it should respect QT_PLUGIN_PATH for testing which plugins are needed).

[flokli]

In case this helps somebody:

I added https://github.com/NixOS/nixpkgs/pull/66725. After using wrapQtAppsHook to wrap every executable (and only those, not the .so files present in $out/libexec), I tried to run the virtualbox tests again (with hardening disabled), see https://github.com/flokli/nixpkgs/commits/wrapqtappshook-virtualbox .

The tests seem to not crash immediately anymore, but the virtual machines still didn't boot up. I did not yet start a machine interactively to investigate.

[jcumming]

In case this helps somebody:

I added #66725. After using wrapQtAppsHook to wrap every executable (and only those, not the .so files present in $out/libexec), I tried to run the virtualbox tests again (with hardening disabled), see https://github.com/flokli/nixpkgs/commits/wrapqtappshook-virtualbox .

The tests seem to not crash immediately anymore, but the virtual machines still didn't boot up. I did not yet start a machine interactively to investigate.

This works for me. Thanks!

[flokli]

@jcumming can you elaborate?

You tried to manually run Virtualbox and start a VM through the GUI? Did you enable hardening?

[flokli]

ping @jcumming

[B4dM4n]

I got the Virtualbox VM Gui running by wrapping the VirtualBoxVM binary and disabling hardening.

--- a/pkgs/applications/virtualization/virtualbox/default.nix
+++ b/pkgs/applications/virtualization/virtualbox/default.nix
@@ -183,6 +183,7 @@ in stdenv.mkDerivation {

   preFixup = optionalString (!headless) ''
     wrapQtApp $out/bin/VirtualBox
+    wrapQtApp $out/libexec/virtualbox/VirtualBoxVM
   '';

   passthru = {

Enabling hardening doesn't work because the wrapper doesn't carry over the correct permissions and probably changes the expected binary path:

$ /run/wrappers/bin/VirtualBoxVM --startvm 51253cab-5d4a-4911-9296-8781ad765ee2
VirtualBoxVM: Error -10 in SUPR3HardenedMain!
VirtualBoxVM: Effective UID is not root (euid=1000 egid=1000 uid=1000 gid=1000)

[flokli]

@B4dM4n Yeah, I also got the qt app to run. But were you able to actually start a VM, too?

[B4dM4n]

Yes, without hardening the VM GUI starts fine.

I also noticed that wrapping VirtualBoxVM doesn't make a difference without hardening, because the VirtualBox binary seems to start the VM's without invoking VirtualBoxVM itself.

[ambrop72]

I've opened pull request #67968 which fixes this.