Minikube: Document work arround for "VirtualBox/Hyper-V conflict: WHvSetupPartition failed - VERR_NEM_VM_CREATE_FAILED"

Created on 25 Jun 2019  ·  86Comments  ·  Source: kubernetes/minikube

  • 执行命令
    minikube start --memory 1024 --cpus 1 --disk-size 40GB --vm-driver virtualbox --bootstrapper=kubeadm

  • 控制台输出

    minikube v1.2.0 on windows (amd64) Creating virtualbox VM (CPUs=1, Memory=1024MB, Disk=40000MB) ...
    E0625 18:36:59.036469 1740 start.go:559] StartHost: create: creating: Unable to start the VM: C:\Program Files\Oracle\VirtualBox\VBoxManage.exe startvm minikube --type headless failed: VBoxManage.exe: error: Call to WHvSetupPartition failed: ERROR_SUCCESS (Last=0xc000000d/87) (VERR_NEM_VM_CREATE_FAILED) VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole Details: 00:00:00.855203 Power up failed (vrc=VERR_NEM_VM_CREATE_FAILED, rc=E_FAIL (0X80004005)) X Unable to start VM: create: creating: Unable to start the VM: C:\Program Files\Oracle\VirtualBox\VBoxManage.exe startvm minikube --type headless failed: VBoxManage.exe: error: Call to WHvSetupPartition failed: ERROR_SUCCESS (Last=0xc000000d/87) (VERR_NEM_VM_CREATE_FAILED) VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole Details: 00:00:00.855203 Power up failed (vrc=VERR_NEM_VM_CREATE_FAILED, rc=E_FAIL (0X80004005))**:

  • 日志输出
    getting ssh client for bootstrapper: Error dialing tcp via ssh client: dial tcp 127.0.0.1:22: connectex: No connection could be made because the target machine actively refused it

  • 操作系统信息
    Win10 家庭中文版

chyperv kinbug lifecyclfrozen long-term-support needs-problem-regex owindows prioritbacklog

Most helpful comment

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

All 86 comments

This appears to be due to VirtualBox attempting to use HyperV as a backend, but failing for some reason: https://forums.virtualbox.org/viewtopic.php?f=6&t=92260

I had no idea that this was even possible with HyperV. I wonder if anyone else has gotten this configuration to work?

你使用的是什么版本的virtualbox?

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

bcdedit /set hypervisorlaunchtype off works for me

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

果然关闭hyper-v就可以了 不是说virtualBox6.0+可以直接运行的吗

bcdedit /set hypervisorlaunchtype off works for me

This worked for me as well!

Ricky

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

Thx a lot,it fixed everything!

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

This worked for me.
Thanks!

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

This works.

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

This works. thanks a lot

now that there is a solution ! I suggest this be added to the solution message !

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

Anyone using WSL2 after running this commmand? Does WSL2 still work?

Anyone using WSL2 after running this commmand? Does WSL2 still work?

Works for me with Ubuntu even when installing Debian.

WSL2 doesn't seem to work for me anymore after doing so. :/

WSL2 doesn't seem to work for me anymore after doing so. :/

WSL 2 is now using virtualization technology (based on Hyper-V) so it won't work as well after shutting down hyper-v support😟

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

This works.

Thanks, This worked!!

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

Thanks this worked!

Even though I deactivated Hyper-v in "Turn on/off windows features" I still had to run "bcdedit /set hypervisorlaunchtype off" in CMD. Thank god I found this discussion, or I'd have to setup my dev env all over again ...

So, when I put in the bcdedit /set hypervisorlaunchtype off thing, I get "The boot configuration data store could not be opened.
Access is denied."

@TheDarkestPlays Same issue here.

So, when I put in the bcdedit /set hypervisorlaunchtype off thing, I get "The boot configuration data store could not be opened.
Access is denied."

You have to run CMD as administrator (right click, Run as administrator)

@TheDarkestPlays @ericchansen
Win+R -> cmd -> Ctrl+Shift+Enter -> bcdedit /set hypervisorlaunchtype off -> reboot your system -> startup your vbox vms

bcdedit /set hypervisorlaunchtype off works for me

then how to off?

For posterity's sake: I had a student experiencing the "vbox: WHvSetupPartition failed" error. She should have been running via Hypervisor, but may have been experiencing a conflict between Docker Desktop and Docker Toolbox. Uninstalling VirtualBox and all things Docker via Windows Add/Remove programs may have been what fixed her issue.

In my meddling, I had run the command above (bcdedit /set hypervisorlaunchtype off), which caused Docker Desktop to not start. Even though virtualization was enabled in BIOS, and all the windows settings pointed to hypervisor being enabled, Docker still wasn't working.

I had to undo my mistake with bcdedit /deletevalue hypervisorlaunchtype, and restart, then everything began to work as expected.

Also, FYI to future searchers who land on this page, bcdedit /enum lists current settings.

bcdedit /set hypervisorlaunchtype off works for me

This worked for me. Thanks!

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

Works for me

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

Thanks works for me!!

In my case this was because I had Windows Sandbox enabled. Disabling this in _'Turn Windows features on or off'_ and restarting my machine solved the issue.

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

Thank god,it wooooorks for me, what happened here?

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

Thanks a lot, it works for me.

The bcedit thing did not work for me. Actually it was due to the core isolation security.
I disabled it and Virtualbox now works like a charm.

Windows Defender Security Center> Device Security> Core isolation > Memory integrity : off

https://forums.virtualbox.org/viewtopic.php?p=417512&sid=dab40dbc19d96086d6009ffae8554b92#p417512

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

This worked for me.
Thanks!

Thank you so much!!!!

CMD > bcdedit /set hypervisorlaunchtype off

did NOT work for me :( (it def ran successfully).

I have Windows 10 enterprise, 1903, build 18362.418

In windows features there is nothing even remotely related to VM's enabled yet i still get;

Call to WHvSetupPartition failed: ERROR_SUCCESS (Last=0xc000000d/87) (VERR_NEM_VM_CREATE_FAILED).
Result Code: | E_FAIL (0x80004005)
-- | --
Component: | ConsoleWrap
Interface: | IConsole {872da645-4a9b-1727-bee2-5585105b9eed}

Please help!

running the VirtualBox installation again in repair mode will fix it again. I had success results with VirtualBox 6.

running the VirtualBox installation again in repair mode will fix it again. I had success results with VirtualBox 6.

that worked thanks!

What worked for me:
I uninstalled the hyperV feature first and rebooted. Then I noticed the 'Call to WHvSetupPartition failed'.
On reboot; I disabled everything in BIOS concerning Virtualization, but on startup windows 10 froze.
Hard shutdown and enabled the Virtualization back on again in BIOS.
After startup I could install a linux OS in VMBox.

What worked for me:
I uninstalled the hyperV feature first and rebooted. Then I noticed the 'Call to WHvSetupPartition failed'.
On reboot; I disabled everything in BIOS concerning Virtualization, but on startup windows 10 froze.
Hard shutdown and enabled the Virtualization back on again in BIOS.
After startup I could install a linux OS in VMBox.
I wouldn't recommend on using hyperV to anyone, because it has some stupid limitations I didn't know about.

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

it solved my windows 10 problem

There are a ton of people saying regarding disabling Hyper-V #MeToo "that worked" "fixed it" "this is the solution" etc. This is a lot of unnecessary spam.

But the incompatibility of Hyper-V is an ongoing issue. The fact that VirtualBox recently began supporting creating and running VMs while Hyper-V is enabled should clue all of you in that this is a fixable problem, and they haven't gotten this far yet.

As such, turning off Hyper-V is not a solution. It is a workaround, and apparently the only option available for now, since the solution is outside of the scope of kubernetes / minkube but rather between Microsoft and Oracle (Windows Hyper-V and VirtualBox).

The solution of executing in CMD (Admin) bcdedit /set hypervisorlaunchtype off did not help me.

But after reading @tag's response, I decided to do the following steps that got me running VB VMs

  1. CMD (Admin) > bcdedit /deletevalue hypervisorlaunchtype
  2. Uninstall Docker
  3. Windows Feature > Disable 'Containers'
  4. Windows Feature > Disable 'Hyper-V'
  5. Restart
  6. Attempt to start VB VMs.

As per the Virtualbox changelog (VirtualBox 6.0.0 onwards): Added support for using Hyper-V as the fallback execution core on Windows host, to avoid inability to run VMs at the price of reduced performance
https://www.virtualbox.org/wiki/Changelog-6.0

And looking at the official documentation: Oracle VM VirtualBox can be used on a Windows host where Hyper-V is running. This is an experimental feature.
No configuration is required. Oracle VM VirtualBox detects Hyper-V automatically and uses Hyper-V as the virtualization engine for the host system. The CPU icon in the VM window status bar indicates that Hyper-V is being used.

https://docs.oracle.com/cd/E97728_01/F12469/html/hyperv-support.html

Has anyone had success booting a VM in VirtualBox which internally uses Hyper-V as virtualization engine?

It work for me
RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

Its work

You could run VirtualBox on top of Hyper-V.
But some additional actions needed.

  1. need to enable "Windows Hypervisor Platform" in OptionalFeatures.exe
  2. set "VBoxInternal/NEM/UseRing0Runloop" for virtual machine. See here how:
    https://superuser.com/questions/1480214/virtualbox-6-doesnt-run-with-hyper-v-enabled-on-windows-10-1903

But seams minikube not able to create new VM!
Only reuse.

bcdedit /set hypervisorlaunchtype off works for me. Thanks :+1:

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.
did not work for me :((
C:\WINDOWS\system32>bcdedit /set hypervisorlaunchtype off
The boot configuration data store could not be opened.
The system cannot find the file specified.

Thanks for the help all.

//Disable hyper-V in Powershell
PS> Disable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V-All

//This has to also be set!
RUN > CMD > bcdedit /set hypervisorlaunchtype off

Reboot.

Edit: don't know why the thumbs down. I just had to do it again now after a Windows update. And again it worked.

And yes, run as administrator - always run system changes as administrator!

RUN> CMD> Right Click> Run as Administrator> bcdedit /set hypervisorlaunchtype off

then reboot host machine.

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

worked for me too!

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

it worked Thanks!!!

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

it worked for me thank you so much

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

that worked for me 👍 👍 👍 👍 👍 👍 👍 👍 👍

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

My system is weird -- while the setting sticks in the BCD the boot manager ignores it somehow and Hyper-V is still started. I have tried all the online solutions, now my next move would be to call Microsoft Support.

Odd question but how do you re-enable if you need to? bcdedit /set hypervisorlaunchtype on returns an error:
The integer data is not valid as specified. Run "bcdedit /?" for command line assistance. The parameter is incorrect.

Odd question but how do you re-enable if you need to? bcdedit /set hypervisorlaunchtype on returns an error:

The integer data is not valid as specified.
Run "bcdedit /?" for command line assistance.
The parameter is incorrect.```

It should be bcdedit /set hypervisorlaunchtype auto.

Thanks @broomerr who also linked it. I'm just pulling the commands here so it can be easyly found.

You could run VirtualBox on top of Hyper-V.
But some additional actions needed.

  1. need to enable "Windows Hypervisor Platform" in OptionalFeatures.exe
  2. set "VBoxInternal/NEM/UseRing0Runloop" for virtual machine. See here how:
    https://superuser.com/questions/1480214/virtualbox-6-doesnt-run-with-hyper-v-enabled-on-windows-10-1903

For those like me who doesn't want to disable Hyper-V, you can try to run a command while beeing in VirtualBox installation directory.

To enable Hyper-V for a single VM:
VBoxManage setextradata "<VM NAME>" "VBoxInternal/NEM/UseRing0Runloop" 0

To enable Hyper-V for all VMs
VBoxManage setextradata global "VBoxInternal/NEM/UseRing0Runloop" 0

This fixed my problem and is a way better solution than disabeling hyper-v since it let you continue to use WSL2.

It appears that https://github.com/kubernetes/minikube/issues/4587#issuecomment-511230925 and https://github.com/kubernetes/minikube/issues/4587#issuecomment-511754366 are workarounds for this issue

Since this issue is out of scope for minikube/k8s I think the best we can do is clearly document it for future users. If anyone would be interested in contributing to the docs, I think it would be really helpful to include this fix in our hyperv documentation

https://github.com/kubernetes/minikube/blob/master/site/content/en/docs/drivers/hyperv.md

It would also be great to have a solutions message in the code (sample PR here: #4257)

If anyone is interested in contributing, feel free to take this issue by commenting /assign

bcdedit /set hypervisorlaunchtype off works for me

You Legend. I love you. It worked

How to enable Hyper V back after this command
bcdedit /set hypervisorlaunchtype off ??

How to enable Hyper V back after this command
bcdedit /set hypervisorlaunchtype off ??

You'd use bcdedit /set hypervisorlaunchtype auto, unless you're on a system like mine which ignores this flag.

this issue still needs a solution message so we automatically advice the user on this problem instead of errors. if anyone interested please read priya's comment https://github.com/kubernetes/minikube/issues/4587#issuecomment-617480167

Can't this tool use Virtualbox 6.1.4 or newer? Since this version, Virtualbox can use a Hyper-V backend just fine.

no conflict in Virtualbox 6.1.4 or higher

RUN > CMD > bcdedit /set hypervisorlaunchtype off
-worked with VirtualBox 6.0.4 :) Thanks!

It would be nice if this one actually upgraded to a new enough version of Virtualbox (6.1.4 or newer) that works with Hyper-V. Maybe someone else will also be as unlucky as I was for that setting not to work (somehow Hyper-V was starting in spite of me having done the hypervisorlaunchtype off thingy, it was very weird)

Everyone, this is now working with Virtualbox 6.1.x (although with deplorable speed). Just confirmed on my 2004 Windows.

I was a bit too quick to judge. The Virtualbox VM starts up, however it only goes until where the windows bootlogo appears. After that, the CPU usage spikes and nothing happens.

I also have a FreeBSD test VM, that manages to start up.

~Everyone, this is now working with Virtualbox 6.1.x (although with deplorable speed). Just confirmed on my 2004 Windows.~

I was a bit too quick to judge. The Virtualbox VM starts up, however it only goes until where the windows bootlogo appears. After that, the CPU usage spikes and nothing happens.

I also have a FreeBSD test VM, that manages to start up.

The implementation in Virtualbox is dog slow, but I think I managed to boot up some light weight Linux machines that were already installed. VMware works fine on top of Hyper-V. Let's hope a future Virtualbox update improves performance...

I must say that it's not just VirtualBox. I have converted my WSL to WSL2 to test how it is, and it is very slow as well.

I have now reverted everything back, uninstalled Hyper-V functionality, and converted my WSL2 back to WSL.

I must say that it's not just VirtualBox. I have converted my WSL to WSL2 to test how it is, and it is very slow as well.

I have now reverted everything back, uninstalled Hyper-V functionality, and converted my WSL2 back to WSL.

WSL2 being this slow is unusual. What CPU do you have? Maybe Hyper-V is only good when it comes to Intel and not to AMD virtualization, for example. For me WSL2 was as fast as any other virtualization, Hyper-V based or based on other platforms. VMware was also similarly fast on top of Hyper-V (no _perceived_ difference from native VMware). The ONLY situation that is dogshit slow for me is Virtualbox on top of Hyper-V.

Here's an excerpt from my sysinfo:

OS Name Microsoft Windows 10 Pro
Version 10.0.19041 Build 19041
Other OS Description    Not Available
OS Manufacturer Microsoft Corporation
System Manufacturer System manufacturer
System Model    System Product Name
System Type x64-based PC
System SKU  SKU
Processor   Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz, 3408 Mhz, 4 Core(s), 8 Logical Processor(s)
BIOS Version/Date   American Megatrends Inc. 3801, 2018-03-14
SMBIOS Version  3.0
Embedded Controller Version 255.255
BIOS Mode   UEFI
BaseBoard Manufacturer  ASUSTeK COMPUTER INC.
BaseBoard Product   MAXIMUS VIII HERO ALPHA
BaseBoard Version   Rev 1.xx
Platform Role   Desktop
Secure Boot State   On
PCR7 Configuration  Binding Not Possible
Windows Directory   C:\WINDOWS
System Directory    C:\WINDOWS\system32
Boot Device \Device\HarddiskVolume2
Locale  United States
Hardware Abstraction Layer  Version = "10.0.19041.1"
Time Zone   W. Europe Daylight Time
Installed Physical Memory (RAM) 64.0 GB
Total Physical Memory   63.9 GB
Available Physical Memory   55.1 GB
Total Virtual Memory    73.4 GB
Available Virtual Memory    62.2 GB
Page File Space 9.50 GB
Page File   C:\pagefile.sys
Kernel DMA Protection   Off
Virtualization-based security   Not enabled
Device Encryption Support   Reasons for failed automatic device encryption: TPM is not usable, PCR7 binding is not supported, Hardware Security Test Interface failed and device is not Modern Standby, Un-allowed DMA capable bus/device(s) detected, TPM is not usable
Hyper-V - VM Monitor Mode Extensions    Yes
Hyper-V - Second Level Address Translation Extensions   Yes
Hyper-V - Virtualization Enabled in Firmware    Yes
Hyper-V - Data Execution Protection Yes

Nothing specific, the hardware itself is 4 years old. I have no problems running software with it, for the most part.

Looks like Hyper-V is disabled FWIW. What is the performance of VMware and Virtualbox in this situation? You cannot expect Hyper-V (and thus WSL2) to be faster, since these other two are _pretty optimized_.

"Hyper-V is disabled": If you mean in windows then yes, Hyper-V is disabled because I disabled it intentionally, for it being too slow. Virtualbox now uses VT-x/AMD-V virtualization, with which I get decent speeds, just like with my WSL that's converted back from WSL2.

For WSL, the speed difference was more than twofold.
For my VirtualBox VMs, they hanged at start.

WSL1 doesn't use virtualization. I use 2 because 1 fails to provide me some essential features for my work.

Virtualbox 6.1.4 and VMware 15.5.5 still use VT-x/AMD-v, just indirectly, via an API offered by Hyper-V. Similarly, Hyper-V itself uses VT-x/AMD-v.

Did you try VMware 15.5.5 with Hyper-V enabled? On my system (CPU: i5-8250u, RAM: 8GB, main storage is a nameless nVME SSD, I don't think anything else matters) it's pretty much as fast as native VMware.

Yep, I too miss some functionality from WSL1 but it's still way faster than WSL2 so I'll be sticking with it for the time being.

I don't use VMware as VirtualBox provided me with better functionality back in the day (not to mention that it was free whereas VMware wasn't), so I stuck with it.

Yep, I too miss some functionality from WSL1 but it's still way faster than WSL2 so I'll be sticking with it for the time being.

I don't use VMware as VirtualBox provided me with better functionality back in the day (not to mention that it was free whereas VMware wasn't), so I stuck with it.

VMware Player is free (not free beer/open source, mind you, but still free in the sense that the only payment is providing your mail address to make an account on their site). It benefits from the exact same improvements as Workstation.

VirtualBox without Hyper-V is good for Windows XP and GUI-less Linux guests. Windows 7 graphics only work well on VMware on my system. And VMware also performs well on top of Hyper-V (after updating to 15.5.5, older versions are incompatible).

Anyway, this started becoming slightly off-topic. Minikube isn't gonna use VMware Player because, while it's free for consumers, it's not really reusable in this project. Virtualbox is. So Oracle needs a wake-up call that their implementation on top of Hyper-V leaves a lot to be desired and a commercial product has some improvements so they can't just cop out by saying it's impossible.

Thanks @broomerr who also linked it. I'm just pulling the commands here so it can be easyly found.

You could run VirtualBox on top of Hyper-V.
But some additional actions needed.

  1. need to enable "Windows Hypervisor Platform" in OptionalFeatures.exe
  2. set "VBoxInternal/NEM/UseRing0Runloop" for virtual machine. See here how:
    https://superuser.com/questions/1480214/virtualbox-6-doesnt-run-with-hyper-v-enabled-on-windows-10-1903

For those like me who doesn't want to disable Hyper-V, you can try to run a command while beeing in VirtualBox installation directory.

To enable Hyper-V for a single VM:
VBoxManage setextradata "<VM NAME>" "VBoxInternal/NEM/UseRing0Runloop" 0

To enable Hyper-V for all VMs
VBoxManage setextradata global "VBoxInternal/NEM/UseRing0Runloop" 0

This fixed my problem and is a way better solution than disabeling hyper-v since it let you continue to use WSL2.

This solution really worked for me, the above ones were not working.

Use Case-> I was trying to launch kubenetes cluster using minikube.

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

I have tried a ton of googled solutions.
And finally, this works perfectly.
Rock u!

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

Bcedit command not found what do I do now

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

I have tried a ton of googled solutions.
And finally, this works perfectly.
Rock u!

Bcedit command not found what do I do now

hello by-maxx
Could you please give me a screenshot?

correct command is : bcdedit

RUN > CMD > bcdedit /set hypervisorlaunchtype off
then reboot host machine.

果然关闭hyper-v就可以了 不是说virtualBox6.0+可以直接运行的吗

It's work for me

bcdedit /set hypervisorlaunchtype off seems to be the workaround to fix this issue. I'm going to leave this issue open for visibility.

we could add this or our Regex Problem Matching to give a solution message to user with link to this issue

Was this page helpful?
0 / 5 - 0 ratings