Nixpkgs: automatically download virtualbox oracle extpack if allowUnfree = true

[cdepillabout]

Issue description

If would be nice for the Virtualbox package to automatically download and install the Oracle Extension pack if nixpkgs.config.allowUnfree is set to true.

That is to say, with the following settings in /etc/nixos/configuration.nix, the Oracle Virtualbox Extension pack should be downloaded and installed automatically:

virtualisation.virtualbox.host.enable = true;
nixpkgs.config.virtualbox.enableExtensionPack = true;
nixpkgs.config.allowUnfree = true;

Currently, with the above options set, nixos-rebuild build produces the following message and fails to build the virtualbox package:

In order to use the extension pack, you need to comply with the VirtualBox Personal Use
and Evaluation License (PUEL) available at:

https://www.virtualbox.org/wiki/VirtualBox_PUEL

Once you have read and if you agree with the license, please use the
following command and re-run the installation:

nix-prefetch-url http://download.virtualbox.org/virtualbox/5.1.26/Oracle_VM_VirtualBox_Extension_Pack-5.1.26-117224.vbox-extpack

It looks like the maintainers of the virtualbox package are @flokli and @sanders. (I'm not sure if it is appropriate to ping maintainers like this, or if it will happen automatically.)

Technical details

• system: "x86_64-linux"
• host os: Linux 4.9.78, NixOS, 17.09.2875.c2b668ee726 (Hummingbird)
• multi-user?: yes
• sandbox: no
• version: nix-env (Nix) 1.11.16
• channels(root): "nixos-17.09.2875.c2b668ee726"
• channels(illabout): ""
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs

[FRidh]

Often the license does not permit us to automatically download such files. In this case, the license at

https://www.virtualbox.org/wiki/VirtualBox_PUEL

states

PLEASE READ THE FOLLOWING ORACLE VM VIRTUALBOX EXTENSION PACK PERSONAL USE AND EVALUATION LICENSE CAREFULLY BEFORE DOWNLOADING OR USING THE ORACLE SOFTWARE. THESE TERMS AND CONDITIONS CONSTITUTE A LEGAL AGREEMENT BETWEEN YOU AND ORACLE.

If we provide the means to automatically download the file, then at no point the user is presented with the fact that they should agree to the license first. Therefore, we should not automate this.

IANAL, but one solution that might be possible is a configuration flag for accepting the license which has a description that includes this line.

It looks like the maintainers of the virtualbox package are @flokli and @sanders. (I'm not sure if it is appropriate to ping maintainers like this, or if it will happen automatically.)

That's fine.

[cdepillabout]

@FRidh I'm definitely not a lawyer either. I don't know what the wording of PLEASE READ THE FOLLOWING means legally.

However, I'm used to Arch Linux, where there is an easy-to-use package called virtualbox-ext-oracle that will automatically download and install the Oracle Extension Pack.[^1]

It seems unfortunate that a great, hands-off distro like NixOS would require a manual step like it currently does.

IANAL, but one solution that might be possible is a configuration flag for accepting the license which has a description that includes this line.

This would work for me. :+1:

Although in the case of the virtualbox package, it seems like it should be enough for the user to specify the nixpkgs.config.virtualbox.enableExtensionPack flag. The description for the enableExtensionPack flag should have a link to the Oracle PUEL.

[^1] However, this is a user-provided package, not officially supported by Arch Linux.

[flokli]

This was already discussed previously in #672.

Personally, I'm more in favor of the Gentoo way of solving this, require the user to explicitly ack the PUEL (via ACCEPT_LICENSE configuration in this case), and then just allowing it to build. As we don't build unfree packages on Hydra, this would still mean each user will download their own copy, and we don't redistributethe binaries.

[cdepillabout]

@flokli Thanks for the pointer to #672.

I'm more in favor of the Gentoo way of solving this, require the user to explicitly ack the PUEL (via ACCEPT_LICENSE configuration in this case), and then just allowing it to build.

This would definitely work for me.

Although, it does seem somewhat silly that the nixpkgs.config.virtualbox.enableExtensionPack flag wouldn't imply acceptance of the PUEL. I can't think of a time when I would want nixpkgs.config.virtualbox.enableExtensionPack to be true but I don't want to accept the PUEL.

As we don't build unfree packages on Hydra, this would still mean each user will download their own copy, and we don't redistribute the binaries.

This also works for me.

However, would it be possible to do something similar to Arch Linux, where Virtualbox with the extension pack is a completely separate package? There could be a virtualbox-with-extension-pack package that takes the normal virtualbox package as input, and runs the command to install the extension pack. This would produce a virtualbox-with-extension-pack package very similar to the virtualbox package, just with the extension pack installed. This would allow the user to download a binary version of the Virtualbox package.

I'm not sure if this is possible with nix, though.

[flokli]

So first steps would be a extpack derivation, probably while introducing a virtualbox-puel license with free = false attribute.

Afterwards, this can be used as an input for the virtualbox derivation, so unattended installation would be possible if nixpkgs.config.allowUnfree is set explicitly (but each user would still download by himself, as we don't build unfree on hydra).

Regarding avoiding to compile virtualbox again when enabling extpack:
The current method installs the extpack somewhere inside $out of virtualbox, which of course would be read-only if the virtualbox package was already built.

I'm not sure if its possible to specify a searchpath to virtualbox, to discover extpacks at runtime.

In case thats possible, we could load the extpack similar to what is done for flash inside the firefox wrapper

[jlelse]

Wouldn't it be also a good idea to specify a writable path as alternative for the $out folder, so you can just install Virtualbox and install the Ext Pack when needed?

[flokli]

Well, this all depends on how virtualbox discovers the path to the extpack - didn't dig into the sources yet ;-)

[jlelse]

@flokli It's set here I think: https://github.com/NixOS/nixpkgs/blob/b3af2cd9627d717820c96259a878f24687066cf1/pkgs/applications/virtualization/virtualbox/default.nix#L127

[flokli]

Actually, VBoxExtPackHelperApp install simply extracts the oracle extpack to $out/(share/|libexec/)virtualbox/ExtensionPacks: Oracle_VM_VirtualBox_Extension_Pack (depending on whether hardening is enabled or not):

https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/virtualization/virtualbox/default.nix#L180

However, there seems to be Oracle_VBoxDTrace_Extension_Pack in there, too (and we want to keep that extpack intact - so we can't just point it to another location.

In general, that extension path seems to be a concatenation of RTPathAppPrivateArch and VBOX_EXTPACK_INSTALL_DIR ("ExtensionPacks") - so only one hardcoded path baked into the binary and no logic to look in multiple directories, requiring that extpack to be an input of virtualbox and then triggering rebuilds of virtualbox with unfree extpacks being enabled.

I'm not sure what's the best way to fix that properly - a patch allowing to pass the extpack path as environment variable, and adding a wrapper package in front of virtualbox might help.
In case extpacks would change, we'd then only need to rebuild the wrapper, not whole virtualbox.

Most likely we then also want to move extpacks and tools like VBoxExtPackHelperApp into separate outputs, to clean that up a bit.

@xeji, what are your thoughts on that?

[xeji]

Maybe we can create a separate virtualbox-with-extension-package that contains everything from virtualbox, plus the extension pack. All files except the extension pack can be either copied or symlinked form the original virtualbox package (which is built by hydra) so the build would be trivial and fast.

[jlelse]

"I'm not sure what's the best way to fix that properly - a patch allowing to pass the extpack path as environment variable, and adding a wrapper package in front of virtualbox might help.
In case extpacks would change, we'd then only need to rebuild the wrapper, not whole virtualbox."

That sounds like a great idea, because this would allow one to manually download the extpack file from Oracle's page and install it, right?

[flokli]

Maybe we can create a separate virtualbox-with-extension-package that contains everything from virtualbox, plus the extension pack. All files except the extension pack can be either copied or symlinked form the original virtualbox package (which is built by hydra) so the build would be trivial and fast.

@xeji Tried that out - unfortunately, the path where to look is hardcoded
inside the binaries, so no luck.

I'm not sure what's the best way to fix that properly - a patch allowing to pass the extpack path as environment variable, and adding a wrapper package in front of virtualbox might help.
In case extpacks would change, we'd then only need to rebuild the wrapper, not whole virtualbox."
That sounds like a great idea, because this would allow one to manually download the extpack file from Oracle's page and install it, right?

If I'm not mistaken, the logic to look for extpacks is in
src/VBox/Main/src-all/ExtPackManagerImpl.cpp@ExtPackManager::initExtPackManager,
loads the (hard-coded) base dir into szBaseDir, appends
VBOX_EXTPACK_INSTALL_DIR, initializes some data structure, then goes hunting
for extensions to load.

We could either watch out for a env variable being present, and use that one
when set, or probably start looking for multiple locations in $PATH style
(which would be nicer and maybe more upstream-compatible, even though we could
very easily assemble a single directory containing all extpacks using Nix).

In addition, src/VBox/Main/src-helper-apps/VBoxExtPackHelperApp.cpp would need
to be patched too, as it also verifies the extpacks to install are inside the
(hard-coded) base dir before doing anything. There's a comment to be strict "for
now", so maybe upstream intended to change behaviour anyhow at some point.

Thoughts? Somebody who wants to write a patch? :-D

[flokli]

To make navigating through the source simpler, I updated my git-svn mirror of virtualbox: https://github.com/flokli/virtualbox

[cdepillabout]

I sent a PR implementing the first step of @flokli's plan from above:

https://github.com/NixOS/nixpkgs/pull/44896

This adds the Virtualbox PUEL as a license, and adds Virtualbox Oracle Extension Pack as a separate derivation. As long as the end-user sets allowUnfree = true, they should be able to install the Oracle Extension Pack without any manual steps. However, this will still require the end-user to compile VirtualBox from scratch.

Hopefully this can get merged in time for 18.09. And then at some point in the next couple months someone will send a PR to VirtualBox (or just create a small patch for nixpkgs) that allows the Extension Pack to be installed without having to completely rebuild the VirtualBox derivation (as described above in detail by @flokli).

[cdepillabout]

PR was merged in.

This means that now you can set the following configuration options in your /etc/nixos/configuration.nix in order to have the Oracle VirtualBox ExtensionPack installed automatically:

{
  nixpkgs.config.allowUnfree = true;
  virtualisation.virtualbox.host.enable = true;
  virtualisation.virtualbox.host.enableExtensionPack = true;
}

This will, however, cause VirtualBox to be compiled from scratch.

It would be nice if someone sent a PR to VirtualBox allowing the location of the Extension Pack to be specified by an environment variable, as described above.

[flokli]

Just as a heads-up: it's highly unlikely I'll spend time on this - see https://github.com/NixOS/nixpkgs/pull/74503.

[stale[bot]]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[cdepillabout]

This PR was originally about updating the virtualbox derivation so the extpack could be downloaded automatically.

This was implemented in #44896.

flokli was also talking about modifying the virtualbox derivation so that it doesn't have to be rebuilt when you enable the extpack. This hasn't been implemented yet.

[flokli]

yeah, I'd consider this issue to be solved.

The idea from https://github.com/NixOS/nixpkgs/issues/34796#issuecomment-406372825 didn't really work, as Virtualbox has suid wrappers and we (rightly) don't dlopen random .so files from weird places (what extpacks are), so people using the unfree extpack would need to recompile virtualbox.