Pipenv: OSError when creating virtualenv symlinking in Vagrant

[Inveracity]

The issue

Using pipenv in a vagrant box is causing a little trouble with the symlinking.
It has been requested that --always-copy is added to pipenv when it creates the virtual environment see ( #1929) but it appears that is not gonna happen 😞

The error

Using /usr/bin/python3.6m (3.6.5) to create virtualenv…
Running virtualenv with interpreter /usr/bin/python3.6m
Using base prefix '/usr'
New python executable in /vagrant/project/thing/.venv/bin/python3.6m
Also creating executable in /vagrant/project/thing/.venv/bin/python
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/virtualenv.py", line 2349, in <module>
    main()
  File "/usr/local/lib/python3.6/dist-packages/virtualenv.py", line 712, in main
    symlink=options.symlink)
  File "/usr/local/lib/python3.6/dist-packages/virtualenv.py", line 927, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/usr/local/lib/python3.6/dist-packages/virtualenv.py", line 1395, in install_python
    os.symlink(py_executable_base, full_pth)
OSError: [Errno 71] Protocol error: 'python3.6m' -> '/vagrant/project/thing/.venv/bin/python'

Environment variables

export PIPENV_VENV_IN_PROJECT=1
export PIPENV_IGNORE_VIRTUALENVS=1

commands to fix it

pipenv sync # Fails with OSError listed above
virtualenv --always-copy .venv
pipenv sync # Success!

Seen in vagrant box: ubuntu/xenial64 and ubuntu/bionic64

I hope it helps someone who got stuck with this like I did

[uranusjr]

Seems like this is a problem with Vagrant running on Windows. Is this the case? This would be a legitimate use case worth looking into (for me).

[techalchemy]

@Inveracity I wouldn't say 'it appears its not gonna happen', the issue in question was closed because a simple solution was found that didn't require api changes on our end. If that's not true in this case we can revisit it, although I was under the impression that this is automatically toggled for windows virtualenv creation anyway...

Either way, you can also just set the environment variable VIRTUALENV_ALWAYS_COPY=1 and this will be toggled on at creation... closing for now, but let me know if this doesnt work

[Inveracity]

@uranusjr It is indeed Vagrant running on windows

@techalchemy it works! thank you so much!

tested it doing:

export VIRTUALENV_ALWAYS_COPY=1
pipenv sync # success
pipenv --rm
export VIRTUALENV_ALWAYS_COPY=0
pipenv sync # fail as expected

[uranusjr]

Probably best to add this to documentation or wiki for future reference. You won’t be the last having this problem.

[techalchemy]

Possibly we should just turn this on for windows users...

[uranusjr]

Problem is you can’t detect Windows inside a Vagrant box (it’s report itself as Linux). Otherwise virtualenv would have detected it and switch to copy mode automatically.

[techalchemy]

well vagrant uses a VMM + virtualization layer that shouldn't really care about whether it runs on a windows host, unless there is a bug in vagrant's actual OS emulation stack that prevents it from emulating the system call properly or something...

The only situation I can see this being relevant is if vagrant provides a way to share a filesystem folder to the guest, the same way as virtualbox does (I've never used vagrant on windows) and there was some attempt to create a virtualenv in that folder from within the VM. @Inveracity can you confirm if that's what was going on here?

[Inveracity]

In my setup I'm using Virtualbox as the virtualization layer, and I believe vagrant simply uses virtualbox to do the share.
I can confirm that I am making the virtualenv in the folder shared by the host OS.

The vagrantfile I use:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|

  config.vm.define "testbox" do |dev|
    dev.vm.box = "ubuntu/xenial64"
    dev.vm.host_name = "testbox"
    dev.vm.network :private_network, ip: "10.0.0.2"
    config.vm.provider :virtualbox do |vb|
        vb.customize ["modifyvm", :id, "--memory", "4096"]
        vb.customize ["modifyvm", :id, "--ioapic", "on"]
        vb.customize ["modifyvm", :id, "--cpus", "2"]
        vb.linked_clone = true
    config.vm.synced_folder ".", "/vagrant",
        type: "virtualbox",
        mount_options: ["dmode=775,fmode=775"]
    end
    config.vm.provision "shell", path: "bootstrap_testbox.sh"
  end
end

I'm wondering if my mount options are causing the issues I see.

[techalchemy]

Yeah this is definitely an issue of the folder being shared by the host => the guest would need to know that somehow or you'd have to do the creation on the host itself or else you just need to keep that environment variable set anytime you do virtualenv creation in this folder

[ynixon]

VIRTUALENV_ALWAYS_COPY

this didn't help me
I get:

         1: from C:/HashiCorp/Vagrant/embedded/gems/2.2.3/gems/net-scp-1.2.1/lib/net/scp/upload.rb:52:in `entries'
C:/HashiCorp/Vagrant/embedded/gems/2.2.3/gems/net-scp-1.2.1/lib/net/scp/upload.rb:52:in `open': Not a directory @ dir_initialize - C:/vagrant/./venv/bin/python (Errno::ENOTDIR)