Tox: "extras" is ignored for environment

[jonathanunderwood]

Thanks for submitting an issue!

If submitting a BUG please provide:

• [x] Minimal reproducible example or detailed description, assign "bug"
• [x] OS and pip list output

OS: Fedora 29, python3 virtualenv

$ pip list
Package        Version                       
-------------- ------------------------------
atomicwrites   1.2.1                         
attrs          18.2.0                        
coverage       4.5.2                         
filelock       3.0.10                        
lz4            2.1.6.dev15+g824be78.d20181219
mccabe         0.6.1                         
more-itertools 4.3.0                         
pip            18.1                          
pluggy         0.8.0                         
psutil         5.4.8                         
py             1.7.0                         
pycodestyle    2.4.0                         
pyflakes       2.0.0                         
pytest         4.0.2                         
pytest-cov     2.6.0                         
setuptools     40.4.3                        
six            1.12.0                        
toml           0.10.0                        
tox            3.6.0                         
virtualenv     16.1.0      

tox.ini file contents:

[tox]
envlist = py,flake8
minversion = 2.4.0

[testenv]
extras = tests
passenv = *
# setenv = PYTHONMALLOC = pymalloc
#     PYTHONMALLOCSTATS = 'yes'
usedevelop = True
commands=
    pytest --cov=lz4 --tb=long {posargs}

[testenv:flake8]
extras = flake8
passenv = *
skip_install = True
commands =
    flake8 lz4 setup.py tests

[flake8]
ignore = E501

In setup.py is this fragment:

...
    extras_require={
        'tests': tests_require,
        'flake8': [
            'flake8',
        ]
    },

However:

$ tox -e flake8
flake8 installed: lz4==2.1.6.dev12+g08d8f22.d20181219
flake8 run-test-pre: PYTHONHASHSEED='3162133725'
flake8 runtests: commands[0] | flake8 lz4 setup.py tests
ERROR: InvocationError for command could not find executable 'flake8'
_______________________________________________________________ summary ________________________________________________________________
ERROR:   flake8: commands failed

In other words, the extras directive in [testenv:flake8] is ignored. If I instead add deps = flake8, all is well.

[gaborbernat]

Extras are installed only at the first time the tox environment is created. Can you reproduce it with tox -rvve flake8? If yes please attach the log for that.

[jonathanunderwood]

Yep, reproduced:

$ tox -rvv -e flake8
  removing /home/jgu/Projects/python-lz4/.tox/log
using tox.ini: /home/jgu/Projects/python-lz4/tox.ini
using tox-3.6.0 from /home/jgu/Projects/python-lz4/venv/lib64/python3.7/site-packages/tox/__init__.py
skipping sdist step
flake8 start: getenv /home/jgu/Projects/python-lz4/.tox/flake8
flake8 cannot reuse: -r flag
flake8 recreate: /home/jgu/Projects/python-lz4/.tox/flake8
  removing /home/jgu/Projects/python-lz4/.tox/flake8
setting PATH=/home/jgu/Projects/python-lz4/.tox/flake8/bin:/home/jgu/Projects/python-lz4/venv/bin:/usr/libexec/python3-sphinx:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/jgu/.local/bin:/home/jgu/bin
  /home/jgu/Projects/python-lz4/.tox$ /home/jgu/Projects/python-lz4/venv/bin/python3 -m virtualenv --python /home/jgu/Projects/python-lz4/venv/bin/python3 flake8 >/home/jgu/Projects/python-lz4/.tox/flake8/log/flake8-0.log
flake8 finish: getenv after 2.51 seconds
flake8 start: finishvenv 
write config to /home/jgu/Projects/python-lz4/.tox/flake8/.tox-config1 as '539757cd4f25c547933a0a745002f52a /home/jgu/Projects/python-lz4/venv/bin/python3\n3.6.0 0 1 0'
flake8 finish: finishvenv after 0.00 seconds
flake8 start: envreport 
setting PATH=/home/jgu/Projects/python-lz4/.tox/flake8/bin:/home/jgu/Projects/python-lz4/venv/bin:/usr/libexec/python3-sphinx:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/jgu/.local/bin:/home/jgu/bin
  /home/jgu/Projects/python-lz4$ /home/jgu/Projects/python-lz4/.tox/flake8/bin/python -m pip freeze >/home/jgu/Projects/python-lz4/.tox/flake8/log/flake8-1.log
flake8 finish: envreport after 0.25 seconds
flake8 installed: lz4==2.1.6.dev12+g08d8f22.d20181219
flake8 start: run-test-pre 
flake8 run-test-pre: PYTHONHASHSEED='1720270556'
flake8 finish: run-test-pre after 0.00 seconds
flake8 start: runtests 
flake8 runtests: commands[0] | flake8 lz4 setup.py tests
setting PATH=/home/jgu/Projects/python-lz4/.tox/flake8/bin:/home/jgu/Projects/python-lz4/venv/bin:/usr/libexec/python3-sphinx:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/jgu/.local/bin:/home/jgu/bin
ERROR: InvocationError for command could not find executable 'flake8'
flake8 finish: runtests after 0.00 seconds
flake8 start: run-test-post 
flake8 finish: run-test-post after 0.00 seconds
_______________________________________________________________ summary ________________________________________________________________
ERROR:   flake8: commands failed

[gaborbernat]

you've set skip_install = True for the flake8 env, if you skip installation of the package, by extension the extras will not be installed either

[jonathanunderwood]

Yes, indeed, removing the skip_install directive fixed it, thanks. That needs to be made clearer in the docs I think. But actually, I don't think that's a very sensible behaviour.

[gaborbernat]

It's the sanest possible behaviour, I mean the extras is a way to specify additional dependencies of your package. If we don't install the package, none of the package, any of dependencies or any of additional dependencies will be installed. Furthermore, we have no way to install only extras (but not package/dependencies), this is a pip limitation. Feel free to bring it up the feature request with them if you really want to have it.

[jonathanunderwood]

I'm not sure that's right though: pip install .[flake8] doesn't install any other requirements.

[gaborbernat]

It installs both the package, it's install requirements plus the extras flake8.

[jonathanunderwood]

Yep, you're right.

[blueyed]

Furthermore, we have no way to install only extras (but not package/dependencies), this is a pip limitation.

Is this still the case?

Feel free to bring it up the feature request with them if you really want to have it.

Any pointers?