Vagrant: Unable to modify VM with Virtualbox 6, boot timeout

[SteveAyre]

Host: MacOS 10.14.2 (Mojave)
Guest: Ubuntu Xenial 64bit
Vagrant 2.2.2
Virtualbox 6.0.0r127566

Expected behavior

VM is modified, boots normally.

Actual behavior

Hangs on boot with 'SSH auth method: private key' as last output, before timing out with 'Timed out while waiting for the machine to boot' error message. Provisioners do not run.

Steps to reproduce

Having either of these in the Vagrantfile will exhibit the behaviour when running 'vagrant up'

current.vm.provider "virtualbox" do |v|
v.customize [ "storageattach", :id, "--storagectl", "SCSI", "--port", "1", "--device", "0", "--type", "hdd", "--medium", "extradisk.vmdk"]
end

current.vm.provider "virtualbox" do |v|
v.customize [ "modifyvm", :id, "--uartmode1", "disconnected" ]
end

References

dizeee mentions the same issue in a comment on #9794

[briancain]

Hello! Thanks for opening an issue. When opening a new issue on the Vagrant project, we ask that everyone provide a gist containing the behavior you're seeing with the --debug flag included. Also please provide us with the Vagrant box name that was used. This helps us figure out what's actually going on. Thanks!

[briancain]

I also just tried this with Virtualbox 6.0, Vagrant box bento/ubuntu-18.04, and Vagrant version 2.2.2 and it worked just fine:

Vagrant.configure("2") do |config|
  config.vm.define "ubuntu" do |b|
    b.vm.box = "bento/ubuntu-18.04"
    b.vm.provider "virtualbox" do |v|
      v.customize [ "modifyvm", :id, "--uartmode1", "disconnected" ]
    end
  end
end

vagrant@vagrant:~/test$ vagrant up ubuntu
Bringing machine 'ubuntu' up with 'virtualbox' provider...
==> ubuntu: Box 'bento/ubuntu-18.04' could not be found. Attempting to find and install...
    ubuntu: Box Provider: virtualbox
    ubuntu: Box Version: >= 0
==> ubuntu: Loading metadata for box 'bento/ubuntu-18.04'
    ubuntu: URL: https://vagrantcloud.com/bento/ubuntu-18.04
==> ubuntu: Adding box 'bento/ubuntu-18.04' (v201812.27.0) for provider: virtualbox
    ubuntu: Downloading: https://vagrantcloud.com/bento/boxes/ubuntu-18.04/versions/201812.27.0/providers/virtualbox.box
    ubuntu: Download redirected to host: vagrantcloud-files-production.s3.amazonaws.com
==> ubuntu: Successfully added box 'bento/ubuntu-18.04' (v201812.27.0) for 'virtualbox'!
==> ubuntu: Importing base box 'bento/ubuntu-18.04'...
==> ubuntu: Matching MAC address for NAT networking...
==> ubuntu: Checking if box 'bento/ubuntu-18.04' version '201812.27.0' is up to date...
==> ubuntu: Setting the name of the VM: test_ubuntu_1547057842847_51347
Vagrant is currently configured to create VirtualBox synced folders with
the `SharedFoldersEnableSymlinksCreate` option enabled. If the Vagrant
guest is not trusted, you may want to disable this option. For more
information on this option, please refer to the VirtualBox manual:

  https://www.virtualbox.org/manual/ch04.html#sharedfolders

This option can be disabled globally with an environment variable:

  VAGRANT_DISABLE_VBOXSYMLINKCREATE=1

or on a per folder basis within the Vagrantfile:

  config.vm.synced_folder '/host/path', '/guest/path', SharedFoldersEnableSymlinksCreate: false
==> ubuntu: Clearing any previously set network interfaces...
==> ubuntu: Preparing network interfaces based on configuration...
    ubuntu: Adapter 1: nat
==> ubuntu: Forwarding ports...
    ubuntu: 22 (guest) => 2222 (host) (adapter 1)
==> ubuntu: Running 'pre-boot' VM customizations...
==> ubuntu: Booting VM...
==> ubuntu: Waiting for machine to boot. This may take a few minutes...
    ubuntu: SSH address: 127.0.0.1:2222
    ubuntu: SSH username: vagrant
    ubuntu: SSH auth method: private key
    ubuntu:
    ubuntu: Vagrant insecure key detected. Vagrant will automatically replace
    ubuntu: this with a newly generated keypair for better security.
    ubuntu:
    ubuntu: Inserting generated public key within guest...
    ubuntu: Removing insecure key from the guest if it's present...
    ubuntu: Key inserted! Disconnecting and reconnecting using new SSH key...
==> ubuntu: Machine booted and ready!
==> ubuntu: Checking for guest additions in VM...
    ubuntu: The guest additions on this VM do not match the installed version of
    ubuntu: VirtualBox! In most cases this is fine, but in rare cases it can
    ubuntu: prevent things such as shared folders from working properly. If you see
    ubuntu: shared folder errors, please make sure the guest additions within the
    ubuntu: virtual machine match the version of VirtualBox you have installed on
    ubuntu: your host and reload your VM.
    ubuntu:
    ubuntu: Guest Additions Version: 5.2.22
    ubuntu: VirtualBox Version: 6.0
==> ubuntu: Mounting shared folders...
    ubuntu: /vagrant => /home/vagrant/test

And your first customize command is not valid:

vagrant@vagrant:~/test$ vagrant up ubuntu
Bringing machine 'ubuntu' up with 'virtualbox' provider...
==> ubuntu: Importing base box 'bento/ubuntu-18.04'...
==> ubuntu: Matching MAC address for NAT networking...
==> ubuntu: Checking if box 'bento/ubuntu-18.04' version '201812.27.0' is up to date...
==> ubuntu: Setting the name of the VM: test_ubuntu_1547058009294_23208
==> ubuntu: Clearing any previously set network interfaces...
==> ubuntu: Preparing network interfaces based on configuration...
    ubuntu: Adapter 1: nat
==> ubuntu: Forwarding ports...
    ubuntu: 22 (guest) => 2222 (host) (adapter 1)
==> ubuntu: Running 'pre-boot' VM customizations...
A customization command failed:

["storageattach", #<Object:0x0000000001996a28>, "--storagectl", "SCSI", "--port", "1", "--device", "0", "--type", "hdd", "--medium", "extradisk.vmdk"]

The following error was experienced:

#<Vagrant::Errors::VBoxManageError: There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["storageattach", "#<Object:0x0000000001996a28>", "--storagectl", "SCSI", "--port", "1", "--device", "0", "--type", "hdd", "--medium", "extradisk.vmdk"]

Stderr: VBoxManage: error: Could not find a registered machine named '#<Object:0x0000000001996a28>'
VBoxManage: error: Details: code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001), component VirtualBoxWrap, interface IVirtualBox, callee nsISupports
VBoxManage: error: Context: "FindMachine(Bstr(a->argv[0]).raw(), machine.asOutParam())" at line 335 of file VBoxManageStorageController.cpp
>

Please fix this customization and try again.

[SteveAyre]

Corrected the first example.

[SteveAyre]

Vagrantfile:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(2) do |config|

  config.vm.box_check_update = false

  config.vm.box = "ubuntu/xenial64"

  config.vm.provider "virtualbox" do |vb|
    vb.customize [ "modifyvm", :id, "--uartmode1", "disconnected" ]
  end

end

Works on Virtualbox 5.2

Steves-MBP:test steve$ vagrant -v ; VBoxManage -v ; vagrant up
Vagrant 2.2.2
5.2.22r126460
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'ubuntu/xenial64'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: test_default_1547073150023_90342
==> default: Fixed port collision for 22 => 2222. Now on port 2200.
==> default: Vagrant has detected a configuration issue which exposes a
==> default: vulnerability with the installed version of VirtualBox. The
==> default: current guest is configured to use an E1000 NIC type for a
==> default: network adapter which is vulnerable in this version of VirtualBox.
==> default: Ensure the guest is trusted to use this configuration or update
==> default: the NIC type using one of the methods below:
==> default: 
==> default:   https://www.vagrantup.com/docs/virtualbox/configuration.html#default-nic-type
==> default:   https://www.vagrantup.com/docs/virtualbox/networking.html#virtualbox-nic-type
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2200 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2200
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection reset. Retrying...
    default: 
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default: 
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default: 
    default: Guest Additions Version: 5.1.38
    default: VirtualBox Version: 5.2
==> default: Mounting shared folders...
    default: /vagrant => /Users/steve/Desktop/test
Steves-MBP:test steve$ vagrant destroy -f
==> default: Forcing shutdown of VM...
==> default: Destroying VM and associated drives...

After upgrading to virtualbox 6.0...

Steves-MBP:test steve$ vagrant -v ; VBoxManage -v ; vagrant up
Vagrant 2.2.2
6.0.0r127566
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'ubuntu/xenial64'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: test_default_1547075013903_84243
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
Timed out while waiting for the machine to boot. This means that
Vagrant was unable to communicate with the guest machine within
the configured ("config.vm.boot_timeout" value) time period.

If you look above, you should be able to see the error(s) that
Vagrant had when attempting to connect to the machine. These errors
are usually good hints as to what may be wrong.

If you're using a custom box, make sure that networking is properly
working and you're able to connect to the machine. It is a common
problem that networking isn't setup properly in these boxes.
Verify that authentication configurations are also setup properly,
as well.

If the box appears to be booting properly, you may want to increase
the timeout ("config.vm.boot_timeout") value.

[SteveAyre]

--debug gist at https://gist.github.com/SteveAyre/8b63790bcfa3f9d2cd9767f93f38cb05

[briancain]

From what I can see, this _seems_ like an issue with the ubuntu box. When I used the ubuntu/xenial64 box, I ran into this timeout issue. However with the bento/ubuntu-18.04 box I was able to bring up a vm with provider customizations just fine.

[SteveAyre]

It may well be specific to 16.04, but at the same time the box works fine with Virtualbox 5.2.

[SteveAyre]

Playing around a little more it looks like the storageattach example is different and a non-issue. --port 0 and --port 1 are used by the vmdk for the disks in the box and so using those values replaces one of the disks needed to boot. --port 2 allows the box to boot with Virtualbox 6.0. I'd previously been testing with a trusty64 box where it only has one vmdk attached by default, and copied that config over without increasing the port number.

The "--uartmode1 disconnected" behaviour looks like a real issue, but could be specific to 16.04. I was trying that command as I'd seen it suggested elsewhere to speed up Xenial's boot time and stop it logging the console log file on the host. I'm guessing that there is some difference between 5.2 and 6.0 in how the device state appears to the guest that means Ubuntu happily boots on one and not the other; which likely makes it a Virtualbox/Ubuntu/box issue rather than vagrant.

[briancain]

I also just tried with bento/ubuntu-16.04 on VirtualBox 6.0 and it had no issues :man_shrugging: I am 90% sure it is likely an issue with Ubuntus box itself.

[chrispelzer]

I'm running into this issue with Virtualbox 6.0 and my current development box which is built with bento/ubuntu-18.04. Switching back to Virtualbox 5.2.22 and it works perfectly fine.

I currently have

current.vm.provider "virtualbox" do |v|
v.customize [ "modifyvm", :id, "--uartmode1", "disconnected" ]
end

in my box as well. I'll try to build a new box later and see if I get the same issue.

[chrisroberts]

Does the guest hang when the customize entries are removed?

[chrispelzer]

@chrisroberts It looks like

    # Disable console log from being required
    vb.customize [ "modifyvm", :id, "--uartmode1", "disconnected" ]

is the cause. I just tried to build my box again against bento/ubuntu-18.04 with Virtualbox 6.0.

With the customize removed, it gets past SSH auth method: private key, otherwise it will timeout.

The reason I had the customize entry was due to the ubuntu-bionic-18.04-cloudimg-console.log file needing to be available otherwise the built box will error, that it's missing.

[cjelger]

I have the same problem with ubuntu/xenial64 on MacOS Mojave 10.14.2 and VirtualBox 6.0.2r128162. The VM boots extremely slowly and even with a 10mn timeout the vagrant boot times out. It used to work fine with VirtualBox 5.2.

If I use bento/ubuntu-16.04 instead of ubuntu/xenial64 it works just fine.

The Vagrantfile I use also has vb.customize ['modifyvm', :id, '--uartmode1', 'disconnected' ] ... in case this the main cause of the issue. I unfortunately didn't have time to try without that line with ubuntu/xenial64.

[rogusdev]

I tested 5 different Vagrantfiles with Vagrant 2.2.3 and VirtualBox 6.0.2 r128162:

ubuntu/bionic64 w uart, timed out:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/bionic64"
  config.vm.box_version = "20190119.0.0"
  config.vm.provider "virtualbox" do |vb|
    vb.customize [ "modifyvm", :id, "--uartmode1", "disconnected" ]
  end
end

ubuntu/bionic64 no uart, worked fine:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/bionic64"
  config.vm.box_version = "20190119.0.0"
  config.vm.provider "virtualbox" do |vb|
  end
end

ubuntu/xenial64 w uart, timed out:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/xenial64"
  config.vm.box_version = "20190118.0.0"
  config.vm.provider "virtualbox" do |vb|
    vb.customize [ "modifyvm", :id, "--uartmode1", "disconnected" ]
  end
end

ubuntu/xenial64 no uart, worked fine:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/xenial64"
  config.vm.box_version = "20190118.0.0"
  config.vm.provider "virtualbox" do |vb|
  end
end

bento/ubuntu-18.04 w uart, worked fine:

Vagrant.configure("2") do |config|
  config.vm.box = "bento/ubuntu-18.04"
  config.vm.box_version = "201812.27.0"
  config.vm.provider "virtualbox" do |vb|
    vb.customize [ "modifyvm", :id, "--uartmode1", "disconnected" ]
  end
end

Previously I used ubuntu/bionic64 w uart with virtualbox 5.2 with no problems, I am only here on this issue now because I upgraded to Virtualbox 6 today and instantly had these problems, even with existing VMs that I could no longer run without timeouts. So the issue was not a recent ubuntu addition.

Also note that I was able to "Show" these VMs that timed out in Virtualbox and watch them operating as expected, so the issue is somehow with vagrant and virtualbox's interactions in these particular images, only when uart mode is disconnected.

So I recognize that this is an obscure/rare issue -- but definitely not that rare! -- and I am definitely switching to bento right now, but figured I'd add more info in case that helps.

[arfs]

If anyone cares, I had the same exact issue after I upgraded to Virtualbox 6. However, I am running on Windows 10 instead of OSX like the OP.

I have been using ubuntu/xenial64 for a year now and it was working fine when I was on Virtualbox 5.x but after upgrading my boot up gets stuck at "SSH auth method: private key" as well.

I switched to bento/ubuntu-18.04 per discussion above and it seems to be working fine now.

[nunosousa-dev]

I just got the same issue after upgrade to VirtualBox 6 using ubuntu/bionic64.

Disabled the uartmode and everything seems fine now.

[chrisroberts]

Hi everyone,

I spent the afternoon yesterday digging into this and it looks to be related to an updated handling of serial devices introduced into VirtualBox. Using the ubuntu boxes, disconnecting the serial port, or even disabling it, produces the same resultant behavior which from the Vagrant side is a boot timeout. If you pull the GUI up for the guest, you will see that the guest is still running, just extremely slowly. Updating grub on the guest to remove serial output makes things work normally again, but this requires starting the guest first to disable and then reloading to disconnect/disable the serial port.

If you don't want the serial port enabled, and are using VirtualBox 6, you'll be better off using a bento box instead of the ubuntu box as bento is not built with grub configured for serial output.

Cheers!

[JuPlutonic]

Thanx for sharing knowhow bento/ubuntu-16.04

[AlexandreLepretre]

@chrisroberts thanks a lot for that, it helped massively. I've found how to do what you describe on Ask Ubuntu: https://askubuntu.com/questions/771871/16-04-virtualbox-vm-from-vhd-file-hangs-at-non-blocking-pool-is-initialized

1. Start the VM directly in Virtualbox (not Vagrant)
2. Wait minutes, hours, however long it takes for it to finally boot
3. Edit /etc/default/grub and all the config under /etc/default/grub.d/ to remove all mentions of console=ttyS0
4. Run sudo update-grub
5. Power off the VM
6. Restart the VM

[deeco]

+1 for the bento/ubuntu-18.04,
I had the same issue in Windows 10 , Ubuntu WSL. works fine without customization.
vb.customize [ "modifyvm", :id, "--uartmode1", "disconnected" ]

replaced "ubuntu/xenial64" for "ubuntu/xenial64" and now all works fine again

[dsloyer]

I am using WSL on Windows 10, deploying a cluster of several Xenial VMs. If I do not disconnect the UART, the VMs will not start, dumping this during vagrant up:

There was an error while executing VBoxManage, a CLI used by Vagrant for controlling VirtualBox. The command and stderr is shown below. Command: ["startvm", "9a544b92-0045-48fa-be6b-922059d5e1c8", "--type", "headless"] Stderr: VBoxManage.exe: error: RawFile#0 failed to create the raw output file /mnt/c/Users/dsloyer/dev/ukube/ubuntu-xenial-16.04-cloudimg-console.log (VERR_PATH_NOT_FOUND) VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole

Disconnecting the UART eliminates this problem, and works perfectly on VirtualBox 5.2.26. Upgrading to VirtualBox 6.0 causes the super-slow boot behavior reported here.

I am more or less stuck on Xenial, for now, and hope that this problem can be addressed in a newer version of VirtualBox 6.

If anyone knows how to solve the boot issue in Windows 10 WSL without disabling the UART, I'd love to hear it!

Thanks, Dave

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.