Vagrant: NFS issue in Catalina with secondary filesystems

[Mezzle]

The workaround in vagrant 2.2.6 for Catalina does not work when the shared folder is not on the boot volume.

I use a separate, case-sensitive partition for all my projects. It shows up on Catalina at /Volumes/Development as well as at /System/Volumes/Data/Volumes/Development.

Vagrant dutifully creates an entry in the exports file like this:

/System/Volumes/Data/Volumes/Development/example -alldirs -mapall=501:80 172.17.0.19

However, this gives an error with respect to nfs check exports

exports:2: exported dir/fs mismatch: /System/Volumes/Data/Volumes/Development/example /Volumes/Development

_Modified from a post originally posted by @oddnoc in https://github.com/hashicorp/vagrant/issues/10961#issuecomment-544623375_

[paoga87]

I have a similar issue and have tried many workarounds. This is the error I'm getting:

NFS is reporting that your exports file is invalid. Vagrant does
this check before making any changes to the file. Please correct
the issues below and execute "vagrant reload":

exports:3: exported dir/fs mismatch: /Users /System/Volumes/Data

However, in my case Vagrant is not even creating the entry in my exports file.

[nathan-fiscaletti]

Might be related to https://github.com/hashicorp/vagrant/issues/11134

[Mezzle]

They're ... slightly related, but not quite.

[oddnoc]

Ugh. In Catalina 10.15.1, the workaround of editing /etc/exports and restarting nfsd no longer works.

[hoskogmo]

Experiencing the same problem using MacOS Catalina 10.15 with Vagrant 2.2.6 when mounting a APFS volume that is case sensitive. Vagrant does prefix the paths as expected with /System/Volumes/Data after upgrading from 2.2.5. I have yet to find a workaround.

[jbeker]

So, in 10.15.1., nfsd is crashing. If you look in the console app into the system.log you will see entries like this:

Oct 30 09:30:27 hobbiton-wired com.apple.xpc.launchd[1] (com.apple.nfsd[1303]): Service exited due to SIGSEGV | sent by exc handler[1303]

I have submitted a ticket with Apple (ID: FB7418234). But if others can duplicate and report the issue that would be helpful.

[oddnoc]

I can't reproduce the crash that @jbeker is reporting. What I have found is that editing exports and restarting nfsd now works on 10.15.1 if and only if there is exactly one entry in the exports file. Sigh.

[jbeker]

I can't reproduce the crash that @jbeker is reporting. What I have found is that editing exports and restarting nfsd now works on 10.15.1 if and only if there is exactly one entry in the exports file. Sigh.

I have multiple directories in my exports line. When I have a chance I will try with only one.

[paoga87]

Here's a similar issue with NFS and Catalina (https://github.com/docksal/docksal/issues/1130), and there is a quick fix in there that worked for me for a Docksal application (https://github.com/docksal/docksal/issues/1130#issuecomment-546452903).

I'm still having issues with Vagrant though... I wonder if there is a fix that's similar to the one with Docksal?

[Mezzle]

I think the quick fix above just moves the mount location to a known place - and is fixed by the 2.2.6 fix... This issue only persists with secondary disks, however :(

[paoga87]

I can't reproduce the crash that @jbeker is reporting. What I have found is that editing exports and restarting nfsd now works on 10.15.1 if and only if there is exactly one entry in the exports file. Sigh.

I have multiple directories in my exports line. When I have a chance I will try with only one.

@jbeker I'm curious to know if you were able to try w/ just one entry on the exports file

[jbeker]

I can't reproduce the crash that @jbeker is reporting. What I have found is that editing exports and restarting nfsd now works on 10.15.1 if and only if there is exactly one entry in the exports file. Sigh.

I have multiple directories in my exports line. When I have a chance I will try with only one.

@jbeker I'm curious to know if you were able to try w/ just one entry on the exports file

Yes, so I tested with 2.2.6 and if I manually went in to the /etc/exports file and put each directory on its own line, it worked. Seems like that would be a simple solution for vagrant to just output its lines a different way.

So my exports now look like:

/System/Volumes/Data/Users/me/development/dev-icing -alldirs -mapall=501:20 192.168.33.11 /System/Volumes/Data/Users/me/development/dev-icing/db -alldirs -mapall=501:20 192.168.33.11 /System/Volumes/Data/Users/me/development/dev-icing/foo -alldirs -mapall=501:20 192.168.33.11

[Mezzle]

https://github.com/hashicorp/vagrant/pull/11150 <-- By the way, this PR should resolve the issue...

[molleafauss]

@jbeker I have a similar issue with multiple directories on same line.
This is due to the fact that my Vagrantfile is listing these directory to mount via NFS in a linux box:

    {
      '.' => '/vagrant',
      sources => '/code',
    }.each do |host_path, guest_path|
      dev.vm.synced_folder host_path, guest_path, :nfs => true
    end

point is that on my Catalina host, '.' is a direct subdirectories of sources (I see you get the same as they point to /dev-icing, /dev-icing/db, /dev-icing/foo), so I end up with those multiple directory in /etc/exports in the same line.
It _looks_ like the NFS from Catalina is not able to accept multiple directories in a config line? I see in fact a difference in /etc/exports between Mojave and Catalina:

Catalina:
/System/Volumes/Data/Users/me/dev -alldirs -mapall=501:20 192.168.192.168

Mojave:
"/Users/me/dev" 192.168.192.168 -alldirs -mapall=501:20

Directories are quoted and there's the IP before the options...

I did fix by removing the extra directory from the line in /etc/exports and that did the trick. Not sure if I can force that not to happen by modifying the Vagrantfile

[jbonhag]

Hi @Mezzle, and thanks for opening this issue!

I tried to recreate the issue on macOS Catalina 10.15.1 (19B88) using an NFS mount from a secondary physical disk to an Ubuntu guest, but I didn't see any issues with the mount.

My Vagrantfile I used and the output is here:
https://gist.github.com/jbonhag/fc28acd7e78d68cbf3a864fd930f8cb2

Would you be willing to share an example Vagrantfile that demonstrates the issue? Additionally, if you can run vagrant up --debug and post the result to a Gist, that will provide some additional information.

Cheers!

[molleafauss]

@jbonhag you might want to investigate my last comment and @jbeker ones.
It seems that this depends on the fact that multiple directories are added to the same line in the /etc/export.
It smells like vagrant is determining that all those are under a common "root" but it should be adding only the root and instead is adding them all.

Try adding in your Vagrantfile this second entry:

config.vm.synced_folder "/Volumes/Development/otherdir", "/home/vagrant/otherdir", type: "nfs"

It might lead the macos nfsd to crash.

[jbonhag]

Hi @molleafauss -- thank you so much for pointing that out. I can see the nfsd crash with the following Vagrantfile:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/trusty64"

  config.vm.network "private_network", ip: "172.17.0.19"

  config.vm.synced_folder "/Volumes/Development", "/home/vagrant/Development", type: "nfs"
  config.vm.synced_folder "/Volumes/Development/otherdir", "/home/vagrant/otherdir", type: "nfs"
end

This created the following in /etc/exports:

# VAGRANT-BEGIN: 501 4cfbc5d1-d146-4754-a2d8-b0ae6c7c5d4a
/System/Volumes/Data/Volumes/Development /System/Volumes/Data/Volumes/Development/otherdir -alldirs -mapall=0:80 172.17.0.19
# VAGRANT-END: 501 4cfbc5d1-d146-4754-a2d8-b0ae6c7c5d4a

At least for me, sudo nfsd checkexports does not return an error with this configuration. I don't think this is an issue with having multiple directories on the same line, because the exact same configuration works on macOS Mojave.

Thanks again for your feedback -- I'll take a look at @Mezzle's PR to see if that's the direction we want to go with this.

[jbeker]

@jbonhag So, in my testing, it is the multiple directories on one line. I say this because if I rewrite the exports file by hand and all I do is split them apart, nfs doesn't crash any more.

[JensBourgeois]

I can confirm the same:

https://github.com/hashicorp/vagrant/issues/10961#issuecomment-555777091

[Mezzle]

Seems like something somewhere has updated and fixed the issue I've been having, most likely from Apple's side, as I've not updated vagrant.

Not sure how relevant this now makes my PR...

[jbeker]

Unfortunately I am still seeing the issue and can reproduce 100%

With the following /etc/exports

# VAGRANT-BEGIN: 501 bfa946d5-9b4c-42ef-9fb3-87471212485e
/System/Volumes/Data/Users/gothmog/development/dev-icing /System/Volumes/Data/Users/gothmog/development/dev-icing/db /System/Volumes/Data/Users/gothmog/development/dev-icing/food52 -alldirs -mapall=501:20 192.168.33.11
# VAGRANT-END: 501 bfa946d5-9b4c-42ef-9fb3-87471212485e

I still get repeated:

Nov 21 08:26:14 hobbiton-wired com.apple.xpc.launchd[1] (com.apple.nfsd[3060]): Service exited due to SIGSEGV | sent by exc handler[3060]

If I manually change /etc/exports to this:

# VAGRANT-BEGIN: 501 bfa946d5-9b4c-42ef-9fb3-87471212485e
/System/Volumes/Data/Users/gothmog/development/dev-icing/food52 -alldirs -mapall=501:20 192.168.33.11
/System/Volumes/Data/Users/gothmog/development/dev-icing/db -alldirs -mapall=501:20 192.168.33.11
/System/Volumes/Data/Users/gothmog/development/dev-icing -alldirs -mapall=501:20 192.168.33.11
# VAGRANT-END: 501 bfa946d5-9b4c-42ef-9fb3-87471212485e

This is 100% reproducible on 10.15.1 (19B2106). This is with vagrant 2.2.6.

[jbonhag]

Hi all, I've just merged a PR into the master branch that makes the change suggested by @jbeker: https://github.com/hashicorp/vagrant/issues/11216. It should be fixed in the next release.

@Mezzle: thanks for the update! Feel free to close this issue if you feel that it's resolved. We can move the discussion about the exports file over to #11209.

[chrisroberts]

Closing this as I have confirmed installing Catalina update 1 fixes what ever the issue was with secondary volumes (either nfsd or firmlinks behavior).

Cheers!

[Jens-Bourgeois]

@chrisroberts
Any idea when the new release is coming out?
This is really annoying, I wish there was a way to easily install develop branch or hotfixes.

[nathan-fiscaletti]

@Jens-Bourgeois installing from source is actually fairly easy:
https://www.vagrantup.com/docs/installation/source.html

[molleafauss]

Would love to see going out, we've developers sidestepping this bug but we'd like to avoid installing from source.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.