Tox: Issue on dependencies when pip install from Git repository

[kianxiongfoo]

Hi,

I am trying to import a package from my private git repository as mentioned in https://pip.pypa.io/en/stable/reference/pip_install/#git by adding this line to my requirement.txt:

git+https://bitbucket.org/my-repo/cool-project.git#egg=ProjectName

I am getting this error:

running install_lib
    copying build/lib/coolproject/tests/test_coolproject.py -> /lib/python3.6/site-packages/coolproject/tests
    error: could not delete '/lib/python3.6/site-packages/coolproject/tests/test_coolproject.py': Permission denied

It seems like it is trying to install the package globally in my Mac OS.

I have tried:

1. sudo , which is not a good practice and it does not work
2. running 'pip install -U' before running test at [command]
3. adding -e and --src when pip install to force the repo to install in testenv site-packages

I faced this issue before in my own virtualenv, and I fixed it by creating another virtualenv with python3 built-in venv:
https://docs.python.org/3/library/venv.html

Any idea?

[obestwalter]

Hi @kianxiongfoo, to me it looks like you are trying to use pip directly to install something into your system Python or a Python that wa installed via a package manager (/lib/...). So the pip command you are using is the one that is part of that system interpreter. This fails as you are not runnig the command as root and therefore get a permission error.

To make sure that the pip is used that you actually want (the one in your venv), make sure the venv is activated when you run pip.

If you would do that within tox, that would be taken care of by tox. So as far as I can see, this is not a tox related problem. If it is, please give us more information what you are trying to do.

[kianxiongfoo]

Hi,

I put the git+https://bitbucket.org/my-repo/cool-project.git#egg=ProjectName at my requirements.txt and run tox -r

I did not run it in the venv generated by tox.

Any idea?

[obestwalter]

I put the git+https://bitbucket.org/my-repo/cool-project.git#egg=ProjectName at my requirements.txt and run tox -r

If you run tox then you automatically run the commands section of your testenv in tox.ini inside a virtualenv. tox creates that virtualenv for you (in the .tox folder of your project), builds and installs the project and then runs your commands inside that prepared virtualenv.

I did not run it in the venv generated by tox.

Like I said: that is automatic as soon as you run tox. Or did I misunderstand something?

running install_lib
    copying build/lib/coolproject/tests/test_coolproject.py -> /lib/python3.6/site-packages/coolproject/tests
    error: could not delete '/lib/python3.6/site-packages/coolproject/tests/test_coolproject.py': Permission denied

Because of what I explained above: that output does not look like it came from a tox run, as tox would not try to install your project to /lib/... but rather <path/to/your/cool-project/sources>/.tox/....

Please note that tox by default creates virtualenvs not venvs, uness you have tox-venv installed.

Can you please post your tox.ini contents and the output of your tox run here (please run tox -rvv)? If your project is public you could also send a link to your project.

[kianxiongfoo]

My requirements.txt:
MessagingDispatcher
pymysql
coverage
mock
tox
moto
git+https://bitbucket.org/path/to/your/cool-project.git#egg=ProjectName`

tox.ini:
`
[tox]
envlist = py36
skipsdist = True

[testenv]
deps = -r{toxinidir}/requirements.txt
commands =
coverage run --source=. -m unittest discover -s tests/
`

running tox -rvv:
error: could not delete '/lib/python3.6/site-packages/coolframework/tests/test_customer_driver.py': Permission denied

Command "/Users/fookianxiong/Documents/Projects/lambda-functions/.tox/py36/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/private/var/folders/mr/tl0_sykj6hx_t_bb4m101xph0000gn/T/pip-build-6hjtkgy1/coolFramework/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/mr/tl0_sykj6hx_t_bb4m101xph0000gn/T/pip-xi9g5rc2-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/fookianxiong/Documents/Projects/lambda-functions/.tox/py36/bin/../include/site/python3.6/coolFramework" failed with error code 1 in /private/var/folders/mr/tl0_sykj6hx_t_bb4m101xph0000gn/T/pip-build-6hjtkgy1/coolFramework/
ERROR: could not install deps [-r/Users/fookianxiong/Documents/Projects/lambda-functions/requirements.txt]; v = InvocationError("'/Users/fookianxiong/Documents/Projects/lambda-functions/.tox/py36/bin/pip install -r/Users/fookianxiong/Documents/Projects/lambda-functions/requirements.txt'", 1)`

Note: I wish i could provide you the project link but it is not public. All the dependencies above "git+https://bitbucket.org/path/to/your/cool-project.git#egg=ProjectName" is successfully installed except for the git repo.

I have also tried to switch to the virtualenv created by tox and run "pip install git+https://bitbucket.org/path/to/your/cool-project.git#egg=ProjectName", but i still have the same issue.

Thanks.

[kianxiongfoo]

I faced this issue before in my own virtualenv, and I fixed it by creating another virtualenv with python3 built-in venv:
https://docs.python.org/3/library/venv.html

How do I force tox to create a venv like above instead of virtualenv?

Thanks a lot.

[obestwalter]

Thanks, but the output is not complete, what I see I don't understand. The correct Python Interpreter is used, but then pip ends up trying to install this into the system Python. How that happens I have no idea.

If you want to use venv, you can install it as a plugin for the tox you use via pip install tox-venv. tox --version should list it then as an installed plugin.

[kianxiongfoo]

Hi,

Yes, installing tox-venv fixed my issues.

Thanks for the support!!