Vagrant: "[…] the ports in the auto-correction range are all also used."

[skyzyx]

This is a new one for me. I know about auto-correcting ports, but apparently the range has run out and I don’t know how to reset the pointer back to the beginning of the auto-correction range.

This is an existing Vagrant configuration (Vagrantfile, VMware provider, Vagrant version, host OS, etc.) that I've been using for months… as recently as this morning.

Vagrant version

Encountered error on both 1.8.4 and 1.9.1.

$ vagrant version
Installed Version: 1.8.4
Latest Version: 1.9.1

To upgrade to the latest version, visit the downloads page and
download and install the latest version of Vagrant from the URL
below:

  https://www.vagrantup.com/downloads.html

If you're curious what changed in the latest release, view the
CHANGELOG below:

  https://github.com/mitchellh/vagrant/blob/v1.9.1/CHANGELOG.md

Host operating system

macOS 10.11.6.

Guest operating system

CentOS 7.3.1611

Vagrantfile

Vagrant.require_version ">= 1.8.4"

environment_name = "Development Environment"
memsize = 2048
numvcpus = 2

Vagrant.configure("2") do | config |

  # Box
  config.vm.box = "skyzyx/centos7"
  config.vm.boot_timeout = 240

  # Networking
  config.vm.hostname = "dev-vm"
  config.vm.network :forwarded_port, guest: 80,  host: 80,  id: :http,  auto_correct: true # HTTP
  config.vm.network :forwarded_port, guest: 443, host: 443, id: :https, auto_correct: true # HTTPS
  config.vm.network :private_network, ip: "33.33.33.10"
  config.ssh.forward_agent = true

  # Allow 20 seconds to gracefully halt (instead of 60)
  config.vm.graceful_halt_timeout = 20

  # Cache the yum packages locally if we can
  if Vagrant.has_plugin?("vagrant-cachier")
    config.cache.scope = :machine
    config.cache.auto_detect = true
    config.cache.enable :yum
  end

  # Check for vbguest plugin
  if Vagrant.has_plugin?("vagrant-vbguest")
    config.vbguest.auto_update = true
    config.vbguest.no_remote = false
  end

  # Synced folders
  if Vagrant::Util::Platform.windows?
    config.vm.communicator = "winrm"
    config.vm.synced_folder "", "/vagrant", type: "smb"
    config.vm.synced_folder (File.expand_path '~'), "/home/vagrant/host", type: "smb"
  else
    config.vm.synced_folder "", "/vagrant", type: "nfs",  mount_options: ['rw', 'vers=3', 'tcp', 'fsc', 'actimeo=2']
    config.vm.synced_folder (File.expand_path '~'), "/home/vagrant/host", type: "nfs",  mount_options: ['rw', 'vers=3', 'tcp', 'fsc', 'actimeo=2']
    # If using Sublime Text on the Mac side, edit your preferences: https://www.jverdeyen.be/vagrant/speedup-vagrant-nfs/
  end

  # Oracle VirtualBox
  config.vm.provider :virtualbox do | vb |
    vb.name = environment_name
    vb.gui = false
    vb.linked_clone = true

    vb.memory = memsize
    vb.cpus = numvcpus
    vb.customize ["modifyvm", :id, "--ioapic", "on"]
  end

  # VMware Fusion
  config.vm.provider :vmware_fusion do | vm |
    vm.name = environment_name
    vm.gui = false
    vm.functional_hgfs = false
    vm.linked_clone = true

    vm.vmx["memsize"] = memsize
    vm.vmx["numvcpus"] = numvcpus
  end

  # Parallels Desktop
  config.vm.provider :parallels do | prl |
    prl.name = environment_name
    prl.update_guest_tools = true
    prl.linked_clone = true

    prl.memory = memsize
    prl.cpus = numvcpus
  end

  config.vm.provision :ansible do | ansible |
    ansible.playbook = "ansible/vm.yml"
    ansible.inventory_path = "ansible/inventories/dev-vm"
    ansible.limit = "vagrant"
    ansible.verbose = false
  end

  config.vm.post_up_message = "Welcome to the Development Environment."
end

Debug output

https://gist.github.com/skyzyx/a90fb1296646418dba0503c364abc5bf

Expected behavior

Vagrant should have been able to assign/auto-correct the port.

Actual behavior

Vagrant was unable to assign/auto-correct the port.

Steps to reproduce

1. vagrant up

Additional Info

Ports

$  sudo lsof -i :80 | grep -i LISTEN
vmnet-nat 76370        root   64u  IPv4 0xfd358e8f8bf28c1b      0t0  TCP *:http (LISTEN)

$ sudo lsof -i :443 | grep -i LISTEN
vmnet-nat 76370        root    9u  IPv4 0xfd358e8f7e533a2b      0t0  TCP *:https (LISTEN)

Global status

$ vagrant global-status
id       name   provider state  directory                           
--------------------------------------------------------------------
There are no active Vagrant environments on this computer! Or,
you haven't destroyed and recreated Vagrant environments that were
started with an older version of Vagrant.

This is after a fresh restart of the Host machine.

[skyzyx]

Attempted the same thing with a newer Ruby (2.4.0 from RVM), a simplified Vagrantfile, and the _official_ CentOS 7 image.

Vagrantfile

Vagrant.configure("2") do | config |

  # Box
  config.vm.box = "centos/7"

  # Networking
  config.vm.hostname = "dev-vm"
  config.vm.network :forwarded_port, guest: 80,  host: 80,  id: :http,  auto_correct: true # HTTP
  config.vm.network :forwarded_port, guest: 443, host: 443, id: :https, auto_correct: true # HTTPS
  config.vm.network :private_network, ip: "33.33.33.10"
  config.ssh.forward_agent = true
end

Output

$ vagrant up

Bringing machine 'default' up with 'vmware_fusion' provider...
==> default: Box 'centos/7' could not be found. Attempting to find and install...
    default: Box Provider: vmware_desktop, vmware_fusion, vmware_workstation
    default: Box Version: >= 0
==> default: Loading metadata for box 'centos/7'
    default: URL: https://atlas.hashicorp.com/centos/7
==> default: Adding box 'centos/7' (v1611.01) for provider: vmware_fusion
    default: Downloading: https://atlas.hashicorp.com/centos/boxes/7/versions/1611.01/providers/vmware_fusion.box
==> default: Successfully added box 'centos/7' (v1611.01) for 'vmware_fusion'!
==> default: Cloning VMware VM: 'centos/7'. This can take some time...
==> default: Checking if box 'centos/7' is up to date...
==> default: Verifying vmnet devices are healthy...
==> default: Preparing network adapters...
==> default: Fixed port collision for 80 => 80. Now on port 2250.
Vagrant found a port collision for the specified port and virtual machine.
While this port was marked to be auto-corrected, the ports in the
auto-correction range are all also used.

VM: default
Forwarded port: 443 => 443

[skyzyx]

Using the _thinner_ Vagrantfile, I tried it again with a different provider in an attempt to rule that out.

$ VAGRANT_DEFAULT_PROVIDER=virtualbox vagrant up

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'centos/7' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Loading metadata for box 'centos/7'
    default: URL: https://atlas.hashicorp.com/centos/7
==> default: Adding box 'centos/7' (v1611.01) for provider: virtualbox
    default: Downloading: https://atlas.hashicorp.com/centos/boxes/7/versions/1611.01/providers/virtualbox.box
==> default: Successfully added box 'centos/7' (v1611.01) for 'virtualbox'!
==> default: Importing base box 'centos/7'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'centos/7' is up to date...
==> default: Setting the name of the VM: dev-env_default_1484163717763_56930
==> default: Fixed port collision for 80 => 80. Now on port 2250.
Vagrant found a port collision for the specified port and virtual machine.
While this port was marked to be auto-corrected, the ports in the
auto-correction range are all also used.

VM: default
Forwarded port: 443 => 443

NOTE: I did NOT delete the .vagrant/ directory from cwd before running this.

[kikitux]

Hello

Can you run this command:

vagrant global-status

If you think some of those environments are gone, you can try then:

vagrant global-status --prune

And try again.

That should clean up the used/mapped ports.

[skyzyx]

I managed to work this out, and think this may be an opportunity for Vagrant to be improved to clean-up after itself a little better in the future.

1. Nuke the Vagrant VM:

   vagrant destroy
   rm -Rf .vagrant/
   echo -n "" > /etc/exports
   

2. The ports and port mappings appear to be stored (and not cleaned-up) in these files:

   sudo rm -f "/Library/Preferences/VMware Fusion/networking*"
   sudo rm -f "/Library/Preferences/VMware Fusion/vmnet8/nat.conf*"
   

3. Launch the UI for VMware Fusion to re-write the files we just deleted.

   open "/Application/VMware Fusion.app"
   

4. vagrant up works again.

This link was helpful. http://www.weatherhead.net/vmware-fusion-nat-port-forwarding-101/

UPDATE (2017-12-05): In the 11 months since I posted this, I've had to do this about once per quarter. I don't need to do it on my home VMs, but there appears to be something about my corporate network that causes the ports to run out. It also ignores my Vagrantfile setting:

config.vm.network :private_network, ip: "33.33.33.10"

Thus far, I have not been able to find a less destructive way of resolving this issue — so if anybody has some in-depth knowledge of Vagrant, VMware Fusion, or the Provider, it would be wonderful to find a way to simplify this resolution.

[fhenri]

Thanks @skyzyx - Running on VMWare Fusion 8, I also had to patch vmnet-natd from https://blogs.vmware.com/teamfusion/2016/01/workaround-of-nat-port-forwarding-issue-in-fusion-8-1.html

rather than closing/opening the app, I would recommend using the CLI

sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli ––stop
sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli ––start

It let you see if you have any errors

[skyzyx]

@fhenri: Yeah, that was a _different_ issue that I ran into back when VMware Fusion 8.1 came out. That was fixed in 8.1.1 or 8.1.2, so I haven't had that particular issue in a while. I'm running VMware Fusion 8.5.x these days.

But it's a good point about starting/stopping with the CLI. I'll have to give that a shot next time. 👍

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.