Coveragepy: `coverage run` does not ignore .venv dir, so it tracks all dependencies and python internal files

Created on 3 Dec 2019 · 17Comments · Source: nedbat/coveragepy

Describe the bug
A clear and concise description of the bug.

To Reproduce
How can we reproduce the problem? Please be specific.

What version of Python are you using? Python 3.7.4
What version of coverage.py are you using? The output of coverage debug sys is helpful.

    -- sys -------------------------------------------------------
                version: 4.5.4
                coverage: /Users/h14384/Documents/work/cm-ch-engine/venv/lib/python3.7/site-packages/coverage/__init__.py
            cover_paths: /Users/h14384/Documents/work/cm-ch-engine/venv/lib/python3.7/site-packages/coverage
            pylib_paths: /Users/h14384/.pyenv/versions/3.7.4/lib/python3.7
                tracer: CTracer
    plugins.file_tracers: -none-
    plugins.configurers: -none-
            config_files: .coveragerc
                        setup.cfg
                        tox.ini
            configs_read: -none-
                data_path: /Users/h14384/Documents/work/cm-ch-engine/.coverage
                python: 3.7.4 (default, Aug 15 2019, 12:39:43) [Clang 10.0.1 (clang-1001.0.46.4)]
                platform: Darwin-18.7.0-x86_64-i386-64bit
        implementation: CPython
            executable: /Users/h14384/Documents/work/cm-ch-engine/venv/bin/python
                    cwd: /Users/h14384/Documents/work/cm-ch-engine
                    path: 
                        /Users/h14384/.pyenv/versions/3.7.4/lib/python37.zip
                        /Users/h14384/.pyenv/versions/3.7.4/lib/python3.7
                        /Users/h14384/.pyenv/versions/3.7.4/lib/python3.7/lib-dynload
                        /Users/h14384/Documents/work/cm-ch-engine/venv/lib/python3.7/site-packages
            environment: PYENV_SHELL = zsh
            command_line: /Users/h14384/Documents/work/cm-ch-engine/venv/bin/coverage debug sys
            source_match: -none-
        source_pkgs_match: -none-
            include_match: -none-
            omit_match: -none-
            cover_match: /Users/h14384/Documents/work/cm-ch-engine/venv/lib/python3.7/site-packages/coverage
            pylib_match: /Users/h14384/.pyenv/versions/3.7.4/lib/python3.7

What versions of what packages do you have installed? The output of pip freeze is helpful.

    astroid==2.2.5
    autopep8==1.4.3
    boto3==1.9.208
    botocore==1.12.208
    certifi==2018.11.29
    chardet==3.0.4
    coverage==4.5.4
    Django==2.1.3
    django-debug-toolbar==2.0
    django-silk==3.0.2
    djangorestframework==3.9.0
    docutils==0.14
    fsspec==0.4.4
    gprof2dot==2016.10.13
    idna==2.8
    isort==4.3.21
    Jinja2==2.10
    jmespath==0.9.4
    lazy-object-proxy==1.4.1
    MarkupSafe==1.1.0
    mccabe==0.6.1
    memory-profiler==0.55.0
    mysqlclient==1.4.4
    numpy==1.16.4
    pandas==0.25.0
    pbr==5.1.1
    pep8==1.7.1
    psutil==5.6.3
    psycopg2==2.8.3
    pyarrow==0.14.1
    pycodestyle==2.4.0
    Pygments==2.3.1
    pylint==2.3.1
    pylint-django==2.0.11
    pylint-plugin-utils==0.5
    pymongo==3.9.0
    PyMySQL==0.9.3
    python-dateutil==2.7.5
    pytz==2018.7
    requests==2.21.0
    rope==0.14.0
    s3fs==0.3.3
    s3transfer==0.2.1
    scipy==1.3.0
    selenium==3.141.0
    six==1.11.0
    sqlparse==0.2.4
    stevedore==1.30.0
    typed-ast==1.4.0
    urllib3==1.24.1
    virtualenv==16.1.0
    virtualenv-clone==0.4.0
    virtualenvwrapper==4.8.2
    wrapt==1.11.2

What code are you running? Give us a specific commit of a specific repo that we can check out.

A Django app

What commands did you run?

coverage run manage.py test

Expected behavior
Coverage for my project files only, not every file in the environment including the python standard library.

Additional context

    venv/lib/python3.7/site-packages/urllib3/util/request.py                                                                  45     36    20%
    venv/lib/python3.7/site-packages/urllib3/util/response.py                                                                 35     29    17%
    venv/lib/python3.7/site-packages/urllib3/util/retry.py                                                                   150    102    32%
    venv/lib/python3.7/site-packages/urllib3/util/ssl_.py                                                                    147    114    22%
    venv/lib/python3.7/site-packages/urllib3/util/timeout.py                                                                  59     29    51%
    venv/lib/python3.7/site-packages/urllib3/util/url.py                                                                     103     87    16%
    venv/lib/python3.7/site-packages/urllib3/util/wait.py                                                                     77     59    23%
    ------------------------------------------------------------------------------------------------------------------------------------------
    TOTAL                                                                                                                 221690 156052    30%

bug fixed

Source

ReagentX

👍2

All 17 comments

coverage.py should default to ignoring the same files listed in .gitignore.

ReagentX on 3 Dec 2019

While we discuss what could be done here, it will work for you to add venv to the [run] omit setting of your coverage configuration.

It's not obvious to me that .gitignore is always the right set of things to omit from coverage measurement.

nedbat on 3 Dec 2019

Creating .coveragerc with

omit=
    .venv/*

will work.

.gitignore may not be the complete right answer but default behavior of testing the entire Python standard library definitely makes no sense.

ReagentX on 3 Dec 2019

Definitely it doesn't make sense to test the stdlib. Not to be pedantic, but just to clarify what we are talking about: the libraries you show here are not stdlib modules, they are third-party packages installed in the virtualenv.

There is logic in coverage.py to avoid tracing the stdlib (see the pylib_match line in your output above). Perhaps we need to add something similar for third-party libraries.

nedbat on 3 Dec 2019

I am not using .venv, but poetry, and the dependencies folder is not in my project folder. However, I still get all the dependencies in my coverage report. Is there a way for me to exclude the dependencies?

therearesomewhocallmetim on 1 May 2020

👍1

I am running into this issue with a Django project using .venv. I tried adding omit = .venv/* to setup.cfg but the warning doesn't go away.

You should be able to reproduce it with cookiecutter-django

mschoettle on 1 May 2020

The following command could work:
coverage run --omit 'venv/*' -m unittest tests/*.py && coverage report -m

zees-dev on 30 Sep 2020

👍1

@therearesomewhocallmetim could you give me detailed instructions for how to reproduce the poetry scenario?

nedbat on 17 Mar 2021

@nedbat I faced the same issue as @therearesomewhocallmetim, and here are the details.

Prerequisite

Poetry should be installed on the system. Setup instructions can be found at https://python-poetry.org/docs/#installation.

Steps to reproduce the scenario

Run poetry init in a new python project (empty folder).
Please note that the virtual environment's location depends on the Poetry configuration virtualenvs.in-project, which should be set to false in this case using the command poetry config virtualenvs.in-project false --local. More details are available on the Poetry website.
To confirm that the config change took effect, run poetry config --list to list all configs.
Add Pytest as a dependency using poetry add pytest.
Add Coverage as a dependency using poetry add coverage.
Run poetry install.
Poetry would have created a virtual environment by now.
Run source $(poetry env info --path)/bin/activate to activate the virtual environment created by Poetry (which is not in the project directory we are working in).
After adding some sample code and corresponding tests to this project, run coverage using coverage run -m pytest
Run coverage report. This would show the coverage for the files created in this project as well as the libraries from the virtual environment present at the path poetry env info --path.

Please let me know if you are not able to reproduce this or if you need more information.

Thanks!

akc-code on 27 Mar 2021

@akc-code thanks for these instructions, they were perfect!

nedbat on 10 Apr 2021

👍1

This is now fixed in 0285af966a

Would you mind testing it out?

pip install git+https://github.com/nedbat/coveragepy.git@0285af966a#egg=coverage==5.6b1

nedbat on 11 Apr 2021

Not sure if this a problem with coverage or django_coverage_plugin but the combination of --source and using the django_coverage_plugin causes this warning as well.

Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/template/context.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/template/exceptions.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/template/base.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/template/library.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/template/engine.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/template/utils.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/template/__init__.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/template/defaultfilters.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/template/smartif.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/template/defaulttags.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/templatetags/__init__.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: <snip>/djangotest/.venv/lib/python3.8/site-packages/django/templatetags/i18n.py (already-imported)

If it is an issue with django_coverage_plugin I am happy to raise the issue in the corresponding repo.

mschoettle on 13 Apr 2021

@mschoettle what version of coverage.py are you using?

nedbat on 13 Apr 2021

@nedbat Sorry, should have mentioned this. I tested it with 5.6b1.

mschoettle on 13 Apr 2021

Hi both, I'm hitting the same issue with django-coverage-plugin even though my venv folder is not inside the --source folder.

I've filed #1150 to track it separately, including a minimal reproduction environment.