Vagrant: Port forward issues on macOS Big Sur with Vagrant VMware provider

[michalmiddleton]

Vagrant version

Vagrant 2.2.13
vagrant-vmware-desktop (2.1.5, global)
vagrant-vmware-utility 1.0.16
VMware Fusion Pro 12

Host operating system

macOS 12.0.1 Big Sur

Guest operating system

Linux (centos7) or Windows (2012r2)

Vagrantfile

Vagrant.configure(2) do |config|
    Vagrant.require_version ">= 2.1.5"
    config.vm.box = "tmcsvc/centos-7"

    config.vm.provider :vmware_desktop do |vmware|
        vmware.vmx['displayname'] = 'Linux-CentOS7-test_linux'
        vmware.linked_clone = true
        vmware.gui = false
        vmware.vmx["memsize"] = "1024"
        vmware.vmx["numvcpus"] = "2"
        vmware.vmx["ethernet0.pcislotnumber"] = "33"
        vmware.whitelist_verified = true
    end
end

Debug output

Regular vagrant up
$ vagrant up Bringing machine 'default' up with 'vmware_desktop' provider... ==> default: Cloning VMware VM: 'tmcsvc/centos-7'. This can take some time... ==> default: Checking if box 'tmcsvc/centos-7' version '7.8.1' is up to date... ==> default: Verifying vmnet devices are healthy... ==> default: Preparing network adapters... ==> default: Starting the VMware VM... ==> default: Waiting for the VM to receive an address... ==> default: Forwarding ports... default: -- 22 => 2222 ==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:2222 default: SSH username: vagrant default: SSH auth method: private key default: Warning: Connection refused. Retrying... default: Warning: Connection refused. Retrying... default: Warning: Connection refused. Retrying... default: Warning: Connection refused. Retrying... ...
netstat shows no open port 2222:
$ netstat -anvp tcp | grep LISTEN tcp6 0 0 *.53 *.* LISTEN 131072 131072 226 0 0x0100 0x00000006 tcp4 0 0 *.49682 *.* LISTEN 131072 131072 226 0 0x0100 0x00000006 tcp4 0 0 127.0.0.1.49227 *.* LISTEN 131072 131072 8366 0 0x0180 0x00000026 tcp4 0 0 127.0.0.1.8698 *.* LISTEN 131072 131072 6393 0 0x0100 0x00000026 tcp4 0 0 127.0.0.1.53 *.* LISTEN 131072 131072 5123 0 0x0180 0x0000000e tcp4 0 0 127.0.0.1.44444 *.* LISTEN 131072 131072 5111 0 0x0080 0x00000006 tcp4 0 0 127.0.0.1.9922 *.* LISTEN 131072 131072 382 0 0x0180 0x00000026 tcp4 0 0 *.59866 *.* LISTEN 131072 131072 507 0 0x0100 0x00000006 tcp6 0 0 *.49187 *.* LISTEN 131072 131072 432 0 0x0100 0x00000006 tcp4 0 0 *.49187 *.* LISTEN 131072 131072 432 0 0x0100 0x00000006 tcp46 0 0 *.18102 *.* LISTEN 131072 131072 79 0 0x0100 0x00000006 tcp6 0 0 *.61500 *.* LISTEN 131072 131072 245 0 0x0180 0x00000006 tcp4 0 0 *.61500 *.* LISTEN 131072 131072 245 0 0x0180 0x00000006 tcp4 0 0 127.0.0.1.8021 *.* LISTEN 131072 131072 1 0 0x0180 0x00000006 tcp6 0 0 ::1.8021 *.* LISTEN 131072 131072 1 0 0x0180 0x00000006

Full debug: https://gist.github.com/michalmiddleton/8441bef9c65abb697d38005207317052

Expected behavior

connect to the vm

Actual behavior

stuck on waiting to connect. Happens for both windows and linux boxes

Steps to reproduce

1. vagrant up
   2.
   3.

References

none

[fhenri]

I have same issue after upgrade

host: macOS bigsur
vm: centos
vagrant version: latest (2.2.13)
vagrant vmware: 2.1.5
vagrant utility : 1.0.16

did you check your file /Library/Preferences/VMware\ Fusion/vmnet8/nat.conf in [incomingtcp] section ? do you have an entry created for the VM ?

[GameGamer43]

@fhenri I'm having the same issue and have confirmed that the VM is in /Library/Preferences/VMware\ Fusion/vmnet8/nat.conf

As a temporary workaround you can use SSH to forward the port using something like the following command:

ssh -L [ssh port vagrant is mapping to 22]:[vm ip address]:22 vagrant@[vm ip address]

[michalmiddleton]

Same here, the [incomingtcp] entry is in place. So is this actually a Fusion issue?
I tried explicitly listing Fusion in my Firewall and allowed Full Disk access (because that was the only place that listed Fusion) but no luck.

[michalmiddleton]

It's listed as a Known Issue with the v12 Fusion release: https://docs.vmware.com/en/VMware-Fusion/12/rn/VMware-Fusion-12-Release-Notes.html
(╯°□°)╯︵ ┻━┻

[chrisroberts]

Hi there,

As noted above (https://github.com/hashicorp/vagrant/issues/12045#issuecomment-726872246), port forwarding to the loopback device is no longer functional in Fusion 12 on macOS Big Sur. This is a result of the move away from kexts and to using the vmnet framework provided by macOS. Since Vagrant connects to the guest via the port forward on the loopback, and the port forward isn't actually functional, Vagrant can never establish a connection. We worked for quite a bit to see if we could provide an addon solution to get the port forwarding to work until it's been resolved within Fusion itself, but it'll be a lower level fix that's required.

A workaround for this is to enable connecting to the guest via the assigned IP address directly using ssh_info_public:

Vagrant.configure("2") do |config|
  config.vm.provider :vmware_desktop do |v|
    v.ssh_info_public = true
  end
end

You can also automatically apply this globally to Vagrant projects by adding this to a Vagrantfile in the VAGRANT_HOME directory (~/.vagrant.d/Vagrantfile).

[michalmiddleton]

@chrisroberts Thank you! Alvaro (via Hashi support ticket) just replied with the same config change :) This works!!!

I opened a ticked with VMware to see what they say about it as well.

[fhenri]

yep working for me too

[ghsolutions]

works for me too

[rbmcgowan]

A workaround for this is to enable connecting to the guest via the assigned IP address directly using ssh_info_public:

Vagrant.configure("2") do |config|
  config.vm.provider :vmware_desktop do |v|
    v.ssh_info_public = true
  end
end

@chrisroberts, vagrant novice here, do we just paste that workaround into the Vagrantfile ? because I tried that without any success?

[rbmcgowan]

adding the following to the bottom of my Vagrantfile worked for me (using vmware_fusion rather than vmware_desktop).

Vagrant.configure("2") do |config|
  config.vm.provider : vmware_fusion do |v|
    v.ssh_info_public = true
  end
end

[lukepolo]

Seems to be the same problem with virtual box https://github.com/hashicorp/vagrant/issues/12047 , but not sure what the workaround is for that

[sengfai]

Hi everyone I solved it:

1. Update your virtualbox
2. Update your vagrant
3. Go to System Preference then privacy& security and Allow the permission in general tap.

[4levels]

The proposed solution of adding the ssh_info_public option throws an error when using virtualbox provider:

There are errors in the configuration of this machine. Please fix
the following errors and try again:
VirtualBox Provider:
* The following settings shouldn't exist: ssh_info_public

[chrisroberts]

Please note that this issue is specifically about the Vagrant VMware Desktop provider. This issue does not affect Virtualbox on macOS Big Sur.

[4levels]

We just experienced a very similar issue with the Virtualbox provider. Vagrant never manages to connect but connecting via ssh just works.
This issue prevents us from running our provision steps.
Given that #12047 was already closed I added my comment here.

FYI we did update everything to the latest version and made sure the OSX permissions were all set. We rebooted as well to make sure the kernel modules were loaded correctly.

We have now resorted to downgrading back to Catalina so I cannot currently
reproduce or run a debug session.

I sincerely hope by the time we all upgrade to Big Sur the transition will
be smoother.

[ryanmoon]

I'm not sure the suggestion for using ssh_info_public is the solution, as we build all of our vagrant boxes with that flag as part of their Vagrantfile.

For instance, a macOS Big Sur box built with:

Vagrant.configure("2") do |config|
  config.vm.communicator = "ssh"
  config.vm.network :forwarded_port, guest: 22, host: 2222, id: "ssh", auto_correct:true
  ["vmware_fusion", "vmware_workstation", "vmware_desktop"].each do |provider|
    config.vm.provider provider do |vmware|
      vmware.gui = true
      vmware.ssh_info_public = true
      vmware.whitelist_verified = true
    end
  end
end

Still has the issue, granted this is a VMware Fusion issue moreso than a vagrant issue.

Vagrant: 2.2.13
vagrant-vmware-desktop: 2.1.5
VMware Fusion: 12.0,0
Test Kitchen: 2.7.2
macOS 11.0.1

[4levels]

@chrisroberts are you suggesting I should open another issue since upgrading to BigSur also breaks the Virtualbox provider as we experienced first hand?

[chrisroberts]

@ryanmoon Are you experiencing issues with Vagrant being unable to complete a vagrant up? If so, are you using the latest vagrant-vmware-utility release (1.0.16)?

@4levels No, a new issue shouldn't be opened as #12047 seems to cover the issue you are describing (it has just been closed prematurely).

[michalmiddleton]

For those interested in VMware NAT issue - Fusion 12.1.0 came out and the issue is still present. I just received the following response from VMware support:

NAT related issues in Fusion 12.1.0 are being the ones focused on the next maintenance release. There is background work going on for Jumbo Packets, NAT issues and most of the networking related issues. However we don't have an ETA as of now since we have to rely on Apple's response on the same.

[craigontour]

What about winrm? I am trying to use kitchen with vagrant to converge a W2012 VM (previously used VBox no problems).
With VMWare Fusion 12 on Big Sur I just get:

==> default: Starting the VMware VM...
       ==> default: Waiting for the VM to receive an address...
       ==> default: Forwarding ports...
           default: -- 5985 => 55985
           default: -- 5986 => 55986
           default: -- 22 => 2222
       ==> default: Waiting for machine to boot. This may take a few minutes...
           default: WinRM address: 127.0.0.1:55985
           default: WinRM username: Administrator
           default: WinRM execution_time_limit: PT2H
           default: WinRM transport: negotiate
       Timed out while waiting for the machine to boot. This means that....

Is there a ssh_info_public equivalent for winrm ?

With 'winrm_info_public' I get error when i validate the ~/.vagrant.d/Vagrantfile:

VMware Desktop Provider:
* The following settings shouldn't exist: winrm_info_public