Vagrant: support long paths on windows hosts, by rebinding the share to a UNC path

[Pomax]

Based on https://github.com/mitchellh/vagrant/issues/1953#issuecomment-60852879 and the workaround provided in https://www.virtualbox.org/ticket/11976#comment:2, it might make sense to make vagrant do the rebindng from "plain path" to UNC path during the creation step on windows platforms.

This would allow vagrant to actually be a viable technology on windows hosts for nodejs setups (npm installs are notorious for stupidly long dependency chains, with deduplication only happening _after_ the full, deep, installation of all dependencies)

[chernetsov0]

One up on this. @mitchellh I and whole lot of people just using Grunt or Gulp for any reason in the project would greatly appreciate this being fixed. At least as much as they appreciate Vagrant existing.

Ensuring that folder[:hostpath] is absolute, has correct slashes (\) and prepend it with \\?\ only on Windows hosts in VagrantPlugins::ProviderVirtualBox::Driver::Version_4_3.share_folders should be enough. The same for older driver versions.

[chernetsov0]

I had fixed that method in 1.6.3 locally like this:

def share_folders(folders)
  folders.each do |folder|
    args = ["--name",
      folder[:name],
      "--hostpath",
      '\\\\?\\' + folder[:hostpath].gsub(/[\/\\]/,'\\')]
    args << "--transient" if folder.has_key?(:transient) && folder[:transient]

    # Add the shared folder
    execute("sharedfolder", "add", @uuid, *args)

    # Enable symlinks on the shared folder
    execute("setextradata", @uuid, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/#{folder[:name]}", "1")
  end
end

and defined shared folder as:

  config.vm.synced_folder File.dirname(__FILE__) + '/projects', '/home/vagrant/projects', create: true

which fixed problem for me.

[Pomax]

fun aside: windows has accepted / as a valid path delimiter pretty much since window 1 (so, practically, 3/3.11), but most people don't know this because windows will _show_ them as \, and the command shell doesn't auto-complete things when you use it, and so a mistaken belief has set in that "windows paths _must_ use \". Extremely persistent myth: when accessing paths, the character / is fully legal and interpreted correctly, all the way up to the most modern version of the windows command shell, powershell _and_ windows explorer.

[chernetsov0]

@Pomax yeah, but looks like \\?\ is fixed character combination (as well as \\.\) so //?/ doesn't work (or maybe just did not for me). And while shells do work like you said, I'm not sure how for example CreateDirectoryW will work on / delimited path.

Also it's nicer to have consistent delimiting going on. I agree, it's not required but using .gsub(/[\/\\]/,'\\') makes it a bit cleaner IMO.

[Pomax]

Agreed. Uniformity trumps clever knowledge.

[jhnns]

:+1:

[Taytay]

:+1: Yes please - we just made this change locally, and it's awesome.

@chernetsov0, maybe opening a PR for the suggested change to plugins/providers/virtualbox/driver/version_4_3.rb would make it easier/faster for @mitchellh to accept?

This will let us use vagrant for node development, even with our Windows devs (and without weird workarounds). That's huge (and one of the main reasons we switched to Vagrant in the first place)

[vbarinov]

+1

[rweich]

+1

[Pomax]

@sethvargo I'd mark this a bug, not an enhancement. Shared dirs are unpredictably broken so we can't rely on them on windows, whereas we can rely on them in unix/linux/osx (you can't ask whether the dir you're going to write works. If you could, this would probably more an enhancement than bug)

[sethvargo]

@Pomax I'll defer to @mitchellh on that :smile:

[Pomax]

@mitchellh if you're looking in, can you confirm you've seen this bug please? There's a fair amount of activity but no signal that you've seen this makes it seem like we're talking amongst ourselves =)

[ClementParis016]

Would be great to see this issue fixed :+1:

In the meantime, could someone explain how to fix this temporarily while waiting an official update?

[celtric]

Can confirm that @chernetsov0 solution works on Windows 7, using Vagrant 1.7.2 and VirtualBox 4.3.20.

# Lines 495-510 - C:\HashiCorp\Vagrant\embedded\gems\gems\vagrant-1.7.2\plugins\providers\virtualbox\driver\version_4_3.rb
def share_folders(folders)
  folders.each do |folder|
    args = ["--name",
      folder[:name],
      "--hostpath",
-       folder[:hostpath]]
+      '\\\\?\\' + folder[:hostpath].gsub(/[\/\\]/,'\\')]
    args << "--transient" if folder.key?(:transient) && folder[:transient]

    # Enable symlinks on the shared folder
    execute("setextradata", @uuid, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/#{folder[:name]}", "1")

    # Add the shared folder
    execute("sharedfolder", "add", @uuid, *args)
  end
end

[sirlatrom]

+1

[jgoux]

Awesome @chernetsov0 and @celtric , thanks a lot for this hack !
I hope this will be added in the next release.

[nschubach]

I am not able to get this patch to work. I modified the version_4_3.rb file as noted above and changed my synced folder to be absolute:

Vagrant file:

Vagrant.configure(2) do |config|
    config.vm.box = "deimosfr/debian-jessie"
    config.vm.network "forwarded_port", guest: 8000, host: 8000
    config.vm.network "private_network", ip: "192.168.3.10"
    config.vm.synced_folder File.dirname(__FILE__), '/vagrant', create: true

    config.vm.provider "virtualbox" do |vb|
        vb.gui = true
        vb.memory = "1024"
    end

    config.vm.provision "shell", inline: <<-SHELL
        sudo apt-get update
        sudo apt-get install -y nodejs nodejs-legacy npm git
        cd /vagrant
        npm install gulp
    SHELL
end

Which fails pretty hard when it gets to the gulp install.

vagrant@debian-testing-amd64:/vagrant$ npm install gulp
npm ERR! Error: EPERM, open '/vagrant/node_modules/gulp/node_modules/liftoff/node_modules/findup-sync/node_modules/glob/
node_modules/minimatch/node_modules/brace-expansion/test/negative-increment.js'
npm ERR!  { [Error: EPERM, open '/vagrant/node_modules/gulp/node_modules/liftoff/node_modules/findup-sync/node_modules/g
lob/node_modules/minimatch/node_modules/brace-expansion/test/negative-increment.js']
npm ERR!   errno: 50,
npm ERR!   code: 'EPERM',
npm ERR!   path: '/vagrant/node_modules/gulp/node_modules/liftoff/node_modules/findup-sync/node_modules/glob/node_module
s/minimatch/node_modules/brace-expansion/test/negative-increment.js' }
npm ERR!
npm ERR! Please try running this command again as root/Administrator.

[nicekiwi]

@nschubach did you try running with 'sudo' as the feedback recommends?

[nicekiwi]

+1

[dtfinch]

chernetsov0's fix worked in my case, with Vagrant 1.7.2 and VirtualBox 4.3.22 on Windows 7 64-bit.

[nschubach]

@nicekiwi Yes, I've tried running sudo npm install and it still gives:

==> default: npm
==> default:
==> default: ERR!
==> default:   { [Error: EPERM, open '/vagrant/node_modules/gulp-ng-annotate/node_modules/gulp-util/node_modules/lodash.template/node_modules/lodash.escape
/node_modules/lodash._reunescapedhtml/package.json']
==> default: npm
==> default:
==> default: ERR!
==> default:    errno: 50,

Vagrant 1.7.2, VirtualBox 4.1.8 (checking updates states I'm at the latest version...) and Windows 7 64-bit. This is on an NTFS partition, on an SSD, with all users having full control over the folder where the source code is saved.

After downloading and installing Virtualbox 4.3.8 I'm getting interesting results ... still not quite there, but better.

[nicekiwi]

@nschubach why did you change your sync folder to absolute? does that directory actually exist? You may have configured that wrong.

[nicekiwi]

chernetsov0's first code worked for me too. Didn't need to change my sync dir.

[bencomeau]

chernetsov0 and celtric's changes worked for me too. Working with Vagrant 1.7.2, VirtualBox 4.3.22, Windows 7 Pro 64

[jfbibeau]

:+1: Please add this to the next Vagrant release. Has anyone opened a PR for this yet?

[fusionstrings]

:+1: The very point of using tools like Vagrant is to provide portable repos to developers regardless of the OS their machine is running. Infact using Vagrant has more justification for Windows users than any other platform. I myself not able to recommend Vagrant because of this issue. If this doesn't work then windows should not be in support list. That would make it easier.

Eagerly waiting for some fixes.

[Ingramz]

I am unable to produce a PR right now but I'm submitting my version of the path name here.

so instead of

folder[:hostpath]

I'm offering this line:

Vagrant::Util::Platform.windows? ? ("//?/" + File.expand_path(folder[:hostpath])).gsub("/","\\") : folder[:hostpath]

This way anyone using not windows will have the same experience and for windows paths relative ones are converted to absolute as relative paths are not supported in the UNC format. I do not know if there are any caveats to converting the path beforehand, so I'd like to hear your feedback.

You might also notice that I'm not a ruby programmer.

[BenLiyanage]

This worked for me, though performance seems to have tanked on the share now.

Windows 7
Vagrant 1.7.2
Virtual Box 4.3.26

I had to upgrade some stuff to get this to work correctly. Not sure which upgrades were necessary.

[voraciousdev]

I've spent about 16 hours working on this today (I wish I was exaggerating), and I finally have a solution to the Vagrant/Windows issue at hand. I have been working on a Node.js development environment built on Vagrant, and I have been relentlessly determined to get this working.

This is the commit that I implemented it in, but I'll go ahead and post the main snippet of code here:

config.vm.provider "virtualbox" do |v|
    v.customize ["sharedfolder", "add", :id, "--name", "www", "--hostpath", (("//?/" + File.dirname(__FILE__) + "/www").gsub("/","\\"))]
end

config.vm.provision :shell, inline: "mkdir /home/vagrant/www"
config.vm.provision :shell, inline: "mount -t vboxsf -o uid=`id -u vagrant`,gid=`getent group vagrant | cut -d: -f3` www /home/vagrant/www", run: "always"

In the code above, I am appending \\?\ to the current directory absolute path. This will actually force the Windows API to allow an increase in the MAX_PATH variable (normally capped at 260). Read more about max path. This is happening during the sharedfolder creation which is intentionally handled by VBoxManage and not Vagrant's "synced_folder" method. The last bit is pretty self-explanatory; we create the new shared folder and then make sure it's mounted each time the machine is accessed or touched since Vagrant likes to reload its mounts/shared folders on each load.

I absolutely hope this helps everyone that was struggling with this like I was!

[jfbibeau]

Created pull request #5495 to make it easier to get a fix in for this issue.

[iget-master]

There's any forecast for this fix release? This is an important bug!

[kaluznyt]

I struggled with this on win 10 + virtualbox 5.0 + vagrant 1.7.4, however I then installed the gulp + laravel-elixir globally (npm install --global) instead of in the app folder which cause problems because it's mapped to win dir, gulp command now works fine in the app context

[ezequieltejada]

On Windows 10, VirtualBox 5.0.1, and Vagrant 1.7.4

Once I made the change suggested by @celtric in C:HashiCorpVagrantembeddedgemsgemsvagrant-1.7.4pluginsprovidersvirtualboxdriverversion_5_0.rb (line 511) I get the following error.

Once I revert the change I still get the original error:

I'll try installing the packages using the "--global" parameter.

EDIT: Seems to be an issue opened about the error I'm getting: #5933

[Pomax]

Please understand that activity in closed issues is likely to be entirely ignored

Tempting as it is, don't keep commenting on this issue: it was for UNC support in pre-1.7.3, and we've passed that point. We're at 1.7.4 now, and if you are having problems with versions since this issue was resolved, file a new issue about that so that we can all discuss the problems you are having with it in the right place.

So again: do not comment on this thread anymore, this issue was _fixed and resolved_ with the release of 1.7.3. So, if you have problems with 1.7.3, or 1.7.4, either file a new issue or comment on an already existing one for those versions. Not here. It doesn't help the vagrant community to hide comments relating to bugs in current versions in a comment thread for a much older version.

[morokolo]

This is how we get the magic in our pants :

1. Using your windows explorer,Navigate to your vagrant shared folder (I am using scotchbox by the way) e.g C:scotchbox/public/gulpProject
2. in the address bar of the folder, type cmd and press enter
3. do your gulp installation npm install anything you want bitches...oh what is that ? No errors
4. Do the suck my duck dance!!!

[dantheman213]

Hey guys, I created a script to fix this issue. My solution was to execute the project in a part of the VM that wasn't being shared with Windows. The script can be run over and over again to run a project. It copies the base project to a temporary execution directory. If you're running it 2nd time + it checks for file differences and will run npm install if your "package.json" file has changed. Lastly it runs npm install inside the execution directory and voila! Workaround complete. :)

Check it out here guys:
https://gist.github.com/dantheman213/7a505004d26c480e8274

[berpcor]

Hello. I have the latest version of Vagrant, but the problem is still exists.
I have npm EACCES error. Too long file names for Windows.
Is there any patch?
Should I manually make corrections like said above?

[Ingramz]

@berpcor there would be no point because it has been already added to the latest version of Vagrant.

There are a couple of ways you can work around your issues and they are all mentioned in the PR starting from post https://github.com/mitchellh/vagrant/pull/6493#issuecomment-187139779

[berpcor]

My bad. I have already not the latest version of Vagrant. I read here that the latest version of Virtualbox can solve the problem. Reinstalled it, but my Vagrant stopped working (Looks like virtualbox 5 is not supported). Now, when you said about version, I checked mine... Need to install it...

[morokolo]

Open your project folder, I am using scothbox eg C:/scothbox/public/mywebapp

in the address bar, ctrl+a if not all selected then type cmd

from there do your npm install

[berpcor]

I've installed the last version of Vagrant and Virtualbox. Problem is still exists...

[Ingramz]

@berpcor read the workarounds I mentioned earlier. Try npm install --no-bin-links first and if that fails, you'll need to explain exactly what's the problem you are having.

[dantheman213]

I just make sure "node_modules/" is removed from my Node.JS project then
I create that folder at /root or somewhere else in my Vagrant that is
not shareable. I think create a symlink from
/home/user/myproject/node_modules to /root/node_modules and then run
"npm install" inside my project directory. This makes everything work great!

On 03/23/2016 12:54 PM, Indrek Ardel wrote:

@berpcor https://github.com/berpcor read the workarounds I mentioned
earlier. Try |npm install --no-bin-links| first and if that fails,
you'll need to explain exactly what's the problem you are having.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/mitchellh/vagrant/issues/4815#issuecomment-200517509