tox freezes when pip installing from vcs in requirements.txt

[pytoxbot]

• Bitbucket: https://bitbucket.org/hpk42/tox/issue/286
• Originally reported by: @maiksensi
• Originally created at: 2015-11-12T11:50:29.939

I am using tox to pip install dependencies from a requirements.txt file.One of these dependencies is a repository. I am using pip's facility to install directly from the vcs:

http://pip.readthedocs.org/en/stable/reference/pip_install/#git

I noticed that when the environments are created everything is fine, but as soon as I run tox -r and the environment with the cloned dependency inside exists, tox freezes, because it tries to clone the repository again but fails (as it already exists).

Currently, the only solution to is to manually delete the environment (in my case the .tox folder).

Log output of tox -r -v (I removed the ignoreerrors=1 in make_emptydir):

#!bash

WindowsError: [Error 5] Access is denied: 'C:\\Users\\me\\repos\\myproject\\.tox\\py27\\src\\anotherproject\\.git\\objects\\pack\\pack-9620b5d62dba6f0eb22cf2f802db22244f3cdab7.idx'

[pytoxbot]

Original comment by @stefano-m

@maiksensi, feel free to submit a PR. I'm sure that the tox core devs would be happy too. ;-)

[pytoxbot]

Original comment by @maiksensi

Any news on this? I am happy to supply a PR if wanted.

[pytoxbot]

Original comment by @stefano-m

@maiksensi @The-Compiler you can see what pip does in this case at:

https://github.com/pypa/pip/blob/develop/pip/utils/init.py#L95

but beware of

https://github.com/pypa/pip/issues/2892

[pytoxbot]

Original comment by @maiksensi

Tested this a bit further and it seems to be a rmtree 'issue'. On Windows the .git directory is set as read only. So rmtree removes all but the .git directory and then pip tries to clone the repository again. Rmtree fails on these. I am not sure if this is something Tox should work around (I uess I could provide a patch) or something I should discuss in the rmtree issue tracker. There is a stackoverflow (http://stackoverflow.com/questions/4829043/how-to-remove-read-only-attrib-directory-with-python-in-windows) post suggesting a workaround with an onerror handler and I guess rmtree will not 'fix' this behavior, as one can handle this with the onerror handler @hpk42 do you have any opinion on this?

I haven't tested this with a minimal example (manually deleting directories, using pip directly, etc) but I will.

[pytoxbot]

Original comment by @The-Compiler

How can this be reproduced? I never had any issues with VCS installs and -r.

Can you run with -v and see where it hangs?

[obestwalter]

Windows is causing a lot of pain regarding deletion of folders for many reasons (e.g. just open an explorer window in a certain folder and then try to delete that folder from the command line -> no luck ...).

I would be interested to see how you would want to solve this particular problem from inside tox.

[hpk42]

I think http://stackoverflow.com/questions/4829043/how-to-remove-read-only-attrib-directory-with-python-in-windows#4829285 provides a way to remove read-only trees. I'd be happy about a PR which does this from tox so that we reduce the number of cases where we can't remove a dir on windows. If implementing this in tox i suggest to write a "rmtree" helper function and only test that function. It's not neccessary to write a full functional test IMO.

[kamichal]

In my case the problem was with pip stalling on downloading packages, because of proxy configuration.
Regularly I use proxy and have it configured via evnitonment variables (http_proxy, https_proxy, ect..).
I solved it by calling unset http_proxy, unset https_proxy before calling tox.

[maiksensi]

Actually, I couldn't reproduce it on my newer systems, so I am not sure this is still an issue. Also, I got rid of the repository install on all newer projects.

[obestwalter]

HI @maiksensi thanks for the update - let's close this then.