Setuptools: tests_require=['pytest', 'pytest-cov'] breaks setuptools.setup

[ghost]

Originally reported by: slashfoo (Bitbucket: slashfoo, GitHub: slashfoo)

---

(test)[20:24:14] [jamiel@moeka:masterskel/] $ python setup.py coverage
This command (coverage) requires the installation of pytest-cov
running coverage
Searching for pytest-cov
Reading https://pypi.python.org/simple/pytest-cov/
Best match: pytest-cov 1.6
Downloading https://pypi.python.org/packages/source/p/pytest-cov/pytest-cov-1.6.tar.gz#md5=6da54d74bde9d200de45068ba2ea637a
Processing pytest-cov-1.6.tar.gz
Writing /tmp/easy_install-xvMxzZ/pytest-cov-1.6/setup.cfg
Running pytest-cov-1.6/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xvMxzZ/pytest-cov-1.6/egg-dist-tmp-BGJIES

Installed /home/jamiel/dev/masterskel/pytest_cov-1.6-py2.7.egg
Searching for pytest
Best match: pytest cov-1.6
Downloading https://pypi.python.org/packages/source/p/pytest-cov/pytest-cov-1.6.tar.gz#md5=6da54d74bde9d200de45068ba2ea637a
Processing pytest-cov-1.6.tar.gz
Writing /tmp/easy_install-EweVbo/pytest-cov-1.6/setup.cfg
Running pytest-cov-1.6/setup.py -q bdist_egg --dist-dir /tmp/easy_install-EweVbo/pytest-cov-1.6/egg-dist-tmp-zxCrHa

Installed /home/jamiel/dev/masterskel/pytest_cov-1.6-py2.7.egg
Traceback (most recent call last):
  File "setup.py", line 203, in <module>
    zip_safe=False,
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/jamiel/.virtualenvs/test/local/lib/python2.7/site-packages/setuptools/command/test.py", line 130, in run
    self.distribution.fetch_build_eggs(self.distribution.tests_require)
  File "/home/jamiel/.virtualenvs/test/local/lib/python2.7/site-packages/setuptools/dist.py", line 264, in fetch_build_eggs
    replace_conflicting=True
  File "/home/jamiel/.virtualenvs/test/local/lib/python2.7/site-packages/pkg_resources.py", line 588, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: pytest

since 'pytest-cov' depends on 'pytest', making tests_require=['pytest-cov'] fixes the issue, same if it is tests_require=['pytest-cov', 'pytest'].

Setting a breakpoint on ${HOME}/.virtualenvs/slashpyenv/lib/python3.4/site-packages/pkg_resources.py:620 (replace for your path to pkg_sources.py) shows:

dist = best[req.key] = env.best_match(req, ws, installer)

after running on for req = Requirement.parse('pytest-cov'), when I run !env.best_match(Requirement.parse('pytest'), ws, installer) it returns the best match to be 'pytest-cov 1.6' and exits.

_note_: In the report you see 3.4 in the outputs ect... but this is also present in 2.7.

---

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/196

[ghost]

_Original comment by_ jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Issue #370 was marked as a duplicate of this issue.

[ghost]

_Original comment by_ jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

It looks like pytest-cov is being matched as the "cov" version of pytest, and so is fulfilling the requirement of "pytest". I seem to recall somewhere that setuptools discourages the use of dashes in package names, probably for this reason.

I don't yet know what to recommend here other than to follow the indicated workaround.

[ghost]

_Original comment by_ jkbr (Bitbucket: jkbr, GitHub: jkbr):

---

Got the same problem with

#!python

tests_require = [
   'pytest',
   'pytest-httpbin',

]

Reversing the order fixed it.

[ghost]

_Original comment by_ pbn (Bitbucket: pbn, GitHub: pbn):

---

I am seeing the same problem here with:

#!python

install_require = [
    'pytest',
    'pytest-instafail',
]


                    

[AlexanderMartynoff]

Hello everyone, who can say when this bug will be resolved?

[jaraco]

I don't think it will ever be fixed. Setup requires and easy install are deprecated in favor of PEP 518 in pip and tools like tox and pip. I recommend avoiding easy install and setup requires as much as possible, use the workaround where necesaary.

[astrojuanlu]

I recommend avoiding easy install and setup requires as much as possible, use the workaround where necesaary.

What about tests_require? Is there a PEP 518 equivalent yet?

[benoit-pierre]

Just encountered this issue today while looking at another bug. It looks like it's caused by reusing the easy_install command instead of creating a new one every time:

diff --git a/setuptools/dist.py b/setuptools/dist.py
index fa0b5eb4..6cd04d6e 100644
--- a/setuptools/dist.py
+++ b/setuptools/dist.py
@@ -482,36 +482,30 @@ class Distribution(Distribution_parse_config_files, _Distribution):

     def fetch_build_egg(self, req):
         """Fetch an egg needed for building"""
-
-        try:
-            cmd = self._egg_fetcher
-            cmd.package_index.to_scan = []
-        except AttributeError:
-            from setuptools.command.easy_install import easy_install
-            dist = self.__class__({'script_args': ['easy_install']})
-            dist.parse_config_files()
-            opts = dist.get_option_dict('easy_install')
-            keep = (
-                'find_links', 'site_dirs', 'index_url', 'optimize',
-                'site_dirs', 'allow_hosts'
-            )
-            for key in list(opts):
-                if key not in keep:
-                    del opts[key]  # don't use any other settings
-            if self.dependency_links:
-                links = self.dependency_links[:]
-                if 'find_links' in opts:
-                    links = opts['find_links'][1].split() + links
-                opts['find_links'] = ('setup', links)
-            install_dir = self.get_egg_cache_dir()
-            cmd = easy_install(
-                dist, args=["x"], install_dir=install_dir,
-                exclude_scripts=True,
-                always_copy=False, build_directory=None, editable=False,
-                upgrade=False, multi_version=True, no_report=True, user=False
-            )
-            cmd.ensure_finalized()
-            self._egg_fetcher = cmd
+        from setuptools.command.easy_install import easy_install
+        dist = self.__class__({'script_args': ['easy_install']})
+        dist.parse_config_files()
+        opts = dist.get_option_dict('easy_install')
+        keep = (
+            'find_links', 'site_dirs', 'index_url', 'optimize',
+            'site_dirs', 'allow_hosts'
+        )
+        for key in list(opts):
+            if key not in keep:
+                del opts[key]  # don't use any other settings
+        if self.dependency_links:
+            links = self.dependency_links[:]
+            if 'find_links' in opts:
+                links = opts['find_links'][1].split() + links
+            opts['find_links'] = ('setup', links)
+        install_dir = self.get_egg_cache_dir()
+        cmd = easy_install(
+            dist, args=["x"], install_dir=install_dir,
+            exclude_scripts=True,
+            always_copy=False, build_directory=None, editable=False,
+            upgrade=False, multi_version=True, no_report=True, user=False
+        )
+        cmd.ensure_finalized()
         return cmd.easy_install(req)

     def _set_global_opts_from_features(self):

Working on adding a test before making a PR.

[jaraco]

The fix was released with 36.4.0.

[bluenote10]

I can observe this issue with setuptools version 40.8.0 as well. Was this issue fully resolved or is there a regression?

[ryan-collingham]

I'm also seeing a simillar clash between the "Flask" and "flask_restplus" packages using setuptools 41.0.0 and python 3.7.3. Steps to reproduce and output:

root@112cc68fe288:/# cat setup.py
import setuptools

print('setuptools version = {}'.format(setuptools.__version__))
setuptools.setup(name='mypkg', install_requires=['Flask', 'flask_restplus'])

root@112cc68fe288:/# python setup.py install
setuptools version = 41.0.0
running install
running bdist_egg
running egg_info
writing mypkg.egg-info/PKG-INFO
writing dependency_links to mypkg.egg-info/dependency_links.txt
writing requirements to mypkg.egg-info/requires.txt
writing top-level names to mypkg.egg-info/top_level.txt
reading manifest file 'mypkg.egg-info/SOURCES.txt'
writing manifest file 'mypkg.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to install

creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying mypkg.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying mypkg.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying mypkg.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying mypkg.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying mypkg.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/mypkg-0.0.0-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing mypkg-0.0.0-py3.7.egg
Copying mypkg-0.0.0-py3.7.egg to /usr/local/lib/python3.7/site-packages
Adding mypkg 0.0.0 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/mypkg-0.0.0-py3.7.egg
Processing dependencies for mypkg==0.0.0
Searching for flask_restplus
Reading https://pypi.org/simple/flask_restplus/
Downloading https://files.pythonhosted.org/packages/29/65/a928450eed445995fb25f9e4fe1495d2e8d53d9ddcfe1d94a017bed9f74e/flask_restplus-0.12.1-py2.py3-none-any.whl#sha256=cdc27b5be63f12968a7f762eaa355e68228b0c904b4c96040a314ba7dc6d0e69
Best match: flask-restplus 0.12.1
Processing flask_restplus-0.12.1-py2.py3-none-any.whl
Installing flask_restplus-0.12.1-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
writing requirements to /usr/local/lib/python3.7/site-packages/flask_restplus-0.12.1-py3.7.egg/EGG-INFO/requires.txt
Adding flask-restplus 0.12.1 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/flask_restplus-0.12.1-py3.7.egg
Searching for Flask
Downloading https://files.pythonhosted.org/packages/bf/e2/785c4b07c9f6d18ce2b42ea888801a35cc798fef6baa5982450ac96cb92b/flask-restplus-0.12.1.tar.gz#sha256=3fad697e1d91dfc13c078abcb86003f438a751c5a4ff41b84c9050199d2eab62
Best match: flask restplus-0.12.1
Processing flask-restplus-0.12.1.tar.gz
Writing /tmp/easy_install-83luw62d/flask-restplus-0.12.1/setup.cfg
Running flask-restplus-0.12.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-83luw62d/flask-restplus-0.12.1/egg-dist-tmp-dtc0k7jr
warning: no previously-included files matching '*.pyc' found anywhere in distribution
removing '/usr/local/lib/python3.7/site-packages/flask_restplus-0.12.1-py3.7.egg' (and everything under it)
creating /usr/local/lib/python3.7/site-packages/flask_restplus-0.12.1-py3.7.egg
Extracting flask_restplus-0.12.1-py3.7.egg to /usr/local/lib/python3.7/site-packages
flask-restplus 0.12.1 is already the active version in easy-install.pth

Installed /usr/local/lib/python3.7/site-packages/flask_restplus-0.12.1-py3.7.egg
error: The 'Flask' distribution was not found and is required by mypkg

[benoit-pierre]

@ryan-collingham: that's #498, which is a similar issue, but with install_requires (not tests_require or setup_requires), and the fix/workaround for this issue does not apply.

[ryan-collingham]

@benoit-pierre Ah thanks, I didn't spot that this issue was specific to tests_require/setup_requires params.