Coveragepy: Option to use relative file paths in .coverage file

[nedbat]

Originally reported by Andrei Fokau (Bitbucket: andreif, GitHub: andreif)

---

I test my project in a Docker container and then examine covered lines in PyCharm on my host machine. Currently PyCharm can only load XML reports in Cobertura format and does actually have an issue due to expecting absolute file paths there. JetBrains guys are going to fix the issue I believe, but I wonder if we can add an option in coverage.py to produce .coverage file with relative paths so that one could run tests in a Docker container and then produce report on host machine? If .coverage contains relative paths, PyCharm would hopefully be able to load them directly without producing intermediate XML files.

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/597

[nedbat]

Original comment by Andrei Fokau (Bitbucket: andreif, GitHub: andreif)

---

@nedbat No, I don't use combine. I'll try to debug it later. It seems that absolute/relative paths issue can be related also to Cobertura issue since the paths are relative there. If you don't plan to use absolute paths in Cobertura XML, then I guess .coverage could also use relative paths, or I am wrong here?

[nedbat]

Make sure this is right: "The first value must be an actual file path on the machine where the reporting will happen, so that source code can be found." Also, this is used when combining data files. Do you use "coverage combine"?

[nedbat]

Original comment by Andrei Fokau (Bitbucket: andreif, GitHub: andreif)

---

@nedbat Does the configuration look correct or I did it wrong? It's in my setup.cfg

[nedbat]

Original comment by Andrei Fokau (Bitbucket: andreif, GitHub: andreif)

---

I am not sure how to use it. I have now .coverage file with paths starting with /app/myproj/.... I added [paths] to my config file:

[coverage:run]
source = myproj
branch = True
omit =
    */migrations/*

[coverage:report]
fail_under = 67
skip_covered = True
show_missing = True

# Regex expressions for lines excluded from report
exclude_lines =
    pragma: no cover
    if __name__ == .__main__.:

[coverage:paths]
source =
    myproj
    /app/myproj
    /Users/andrei/myproj

coverage report still shows NoSource errors.

[nedbat]

This sounds like something the [paths] section of the .coveragerc was meant to solve.

[blueyed]

While the specific issue is about paths, I think allowing to handle relative paths in .coveragerc is a valid feature request though.

It would be helpful for when coverage gets run in a changed cwd with subprocess handling.

Currently you have to set an environment variable, e.g.

tox.ini:

setenv =
    coverage: COVERAGE_HOME={toxinidir}

coverage.rc:

[run]
source =
  $COVERAGE_HOME/src
  $COVERAGE_HOME/tests

Since this should be working in a backward compatible way, I suggest either
supporting a ./ prefix to enable it, or set some environment variable like
$COVERAGE_CONFIG_DIR before reading the config file.

With the first approach source = . would use the current working directory, but source = ./ would use the directory of the config file.

[mcaulifn]

@nedbat https://github.com/pytest-dev/pytest-cov/issues/329 should be solved by this feature

[Wonskcalb]

I've got a very similar use-case here as well, where the data file is generated at some point by a CI pipeline, and read & processed down the stream. But because those two steps can run on different workers, the absolute path (which contains the worker unique ID) renders the coverage file unusable.

I discovered and played a little with the combine command, but it does work only if I have a data file with the correct paths (which I don't because that's what I need, rather than have).

[LasseGravesen]

I'm having the same problem, the absolute path was /app/src in the docker container where coverage was run, and wasn't correct for the place where I needed to use the coverage for anything useful.
I fixed it by adding an environment variable ROOT_DIRECTORY that specified the 'correct' path, and then used the sed command to substitute the value for the correct one.

sed -i -e "s,<source>/app/,<source>$(ROOT_DIRECTORY)/,g" coverage.xml

I think this would probably be fixed if the coverage sources were relative instead of absolute.

[nedbat]

@Wonskcalb @LasseGravesen Would your problem be solved by writing a [paths] section in your configuration file, and then running "coverage combine"?

[LasseGravesen]

@nedbat I can't get that to do what you suggest.

If I'm running on a machine where the path is /app/app and I want to end up with a normalized path using coverage combine in the coverage report: /c/Users/username/Desktop/Workspace/cov-test/app,

should my config look something like this?

tox.ini:

[tox]
skipsdist = True
usedevelop = True
envlist = py37

[testenv]
setenv = 
    PYTHONDONTWRITEBYTECODE = 1
deps = -rrequirements.txt
commands =
    coverage run -m pytest
commands_pre =
    coverage erase
commands_post =
    coverage combine --append
    coverage report
    coverage xml

[coverage:run]
source = app
parallel = True

[coverage:paths]
source = 
        /app/app
        /c/Users/username/Desktop/Workspace/cov-test/app

The coverage.xml file ends up looking like this:

<?xml version="1.0" ?>
<coverage branch-rate="0" branches-covered="0" branches-valid="0" complexity="0" line-rate="1" lines-covered="4" lines-valid="4" timestamp="1572035007115" version="4.5.4">
    <!-- Generated by coverage.py: https://coverage.readthedocs.io -->
    <!-- Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd -->
    <sources>
        <source>/app/app</source>
    </sources>
    <packages>
        <package branch-rate="0" complexity="0" line-rate="1" name=".">
            <classes>
                <class branch-rate="0" complexity="0" filename="__init__.py" line-rate="1" name="__init__.py">
                    <methods/>
                    <lines/>
                </class>
                <class branch-rate="0" complexity="0" filename="utils.py" line-rate="1" name="utils.py">
                    <methods/>
                    <lines>
                        <line hits="1" number="3"/>
                        <line hits="1" number="4"/>
                        <line hits="1" number="6"/>
                        <line hits="1" number="7"/>
                    </lines>
                </class>
            </classes>
        </package>
    </packages>
</coverage>

Where I would want the line <source>/app/app</source> to be /c/Users/username/Desktop/Workspace/cov-test/app.

To fix this with sed, I would run the command: sed -i -e "s,<source>/app/,<source>/c/Users/username/Desktop/Workspace/cov-test/,g" coverage.xml

and then I'd end up with a file coverage.xml that reads:

<?xml version="1.0" ?>
<coverage branch-rate="0" branches-covered="0" branches-valid="0" complexity="0" line-rate="1" lines-covered="4" lines-valid="4" timestamp="1572035007115" version="4.5.4">
    <!-- Generated by coverage.py: https://coverage.readthedocs.io -->
    <!-- Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd -->
    <sources>
        <source>/c/Users/username/Desktop/Workspace/cov-test/app</source>
    </sources>
    <packages>
        <package branch-rate="0" complexity="0" line-rate="1" name=".">
            <classes>
                <class branch-rate="0" complexity="0" filename="__init__.py" line-rate="1" name="__init__.py">
                    <methods/>
                    <lines/>
                </class>
                <class branch-rate="0" complexity="0" filename="utils.py" line-rate="1" name="utils.py">
                    <methods/>
                    <lines>
                        <line hits="1" number="3"/>
                        <line hits="1" number="4"/>
                        <line hits="1" number="6"/>
                        <line hits="1" number="7"/>
                    </lines>
                </class>
            </classes>
        </package>
    </packages>
</coverage>

[nedbat]

@LasseGravesen I think you want this in your config file:

[coverage:paths]
source = 
        /c/Users/username/Desktop/Workspace/cov-test/app
        /app/app

The first path should be a real path that exists where the reporting will happen. Does this work for you?

[LasseGravesen]

@nedbat
Sorry for the late response.

The problem is that the real path is /app/app, in that the coverage is being run inside a container. I want to translate /app/app to /c/Users/username/Desktop/Workspace/coverage-test/app. The latter path doesn't exist in the container, so it fails to create a report.

Here's a test repo, that's currently set to report on /app/app, if you change in the tox.ini that the source is reversed, like you stated in your example, then it report won't be generated.

https://github.com/FalconSocial/coverage-test

[nedbat]

@LasseGravesen Thanks for the example, we are getting closer.

When I run your example, I have a .coverage file with /app/app/helpers.py in it. What is the step that fails after that? What are you doing that doesn't work?

[LasseGravesen]

@nedbat The problem is that I don't want a .coverage file with /app/app/helpers.py in it, because I need to use it as an input to another program that does not have its root in /app.
So specifically, I would like a .coverage file with /c/Users/username/Desktop/Workspace/coverage-test/app/helpers.py instead, or more generally a way to specify the root path that should be written to the .coverage file.

[nedbat]

What is this other program? .coverage files are not a public interface to the coverage data. I didn't expect anything else to be reading them.

Would running "coverage combine" on the "/c/Users/username" system work here to re-map the paths?

[mcaulifn]

Tools like Sonar are reading the coverage files and others like pytest report.

[frwickst]

This is an issue that we are running into as well. We are creating coverage reports in Docker that of course does not have the same path as the host. So the report has a source that does not correspond to the host and can not be read by PyCharm in our case.
Using tools such as sed is, of course, one way, but that feels like the hacky solution. Is there a way to support relative paths in this case instead? It feels like having to change the path is the wrong way to go about this.

[JoaRiski]

I would like to see a good argument for why absolute paths are better than relative paths in the first place, other than them being legacy. At least I can't immediately come up with anything that wouldn't be better off using relative paths instead.

[nedbat]

Just to be clear: I am not absolutely refusing to support relative paths. I am trying to understand the details of the problem. I don't know what the implications of relative paths would be throughout coverage.py, and I'm trying to not open new cans of worms at the moment.

Has anyone tried modifying the code to see the extent of the change?

[JoaRiski]

I think the biggest problem is that you might want to use the coverage in other contexts aside from the one it was generated in, this is especially true with various CI tooling and Docker for example.

In the case of a Docker development environment, it's not enough to substitute the path generated by coveragepy with the actual absolute path as suggested in https://github.com/nedbat/coveragepy/issues/597#issuecomment-547148116, since that will most likely not work on other developer's machines.

What would be needed is a way (a flag for example) to generate coverage that maps to the correct source files regardless of where the project is located at, as long as the project structure is the same. This could be achieved for example by looking for paths relative to the configuration file, or optionally allow project root to be defined manually with a CLI argument (which probably also should support relative paths).

[JoaRiski]

Maybe the most sane way to map the relative paths is to map them relative to the coverage output path, so as long as that file is in the correct place (as defined by the output configuration), it would map to the correct files.

[LasseGravesen]

@nedbat .coverage files are used to create coverage.xml files, and its these that tools like Sonarqube uses. Right now, as frwickst, I am forced to run sed commands to replace the source path in the coverage.xml file, or downstream tools like sonarqube will fail to record any coverage information for that projec.t

[LasseGravesen]

@nedbat

Would running "coverage combine" on the "/c/Users/username" system work here to re-map the paths?

I don't think so, but that might just be the fact that I'm not sure how to use that command.

(coverage-test-py37) ✘-1 16:07 ~/Workspace/coverage-test [master|✔] $ ls
README.rst  __pycache__  app  coverage.xml  docker-compose.yml  requirements-dev.txt  requirements.txt  tests  tox.ini
(coverage-test-py37) ✔ 16:09 ~/Workspace/coverage-test [master|✔] $ cat .coverage
!coverage.py: This is a private format, don't read it directly!{"lines":{"/app/app/__init__.py":[1],"/app/app/helpers.py":[1,4,9,11,6]}}
(coverage-test-py37) ✔ 16:07 ~/Workspace/coverage-test [master|✔] $ coverage combine
No data to combine
(coverage-test-py37) ✔ 16:09 ~/Workspace/coverage-test [master|✔] $ cat .coverage
!coverage.py: This is a private format, don't read it directly!{"lines":{"/app/app/__init__.py":[1],"/app/app/helpers.py":[1,4,9,11,6]}}

[nedbat]

OK, I've implemented experimental support for relative filenames. If you can install it from GitHub and give it a try, that would help me know whether to merge it to master.

1. Install coverage from GitHub:
   $ pip install git+https://github.com/nedbat/coveragepy.git@66ee365305cb
2. Add this setting to your .coveragerc:
   [run] relative_files = True
3. Run coverage as you usually do.

Let me know if it works for the Docker cases. I'll also be interested to hear if it works for non-Docker cases :)

[nedbat]

This is now available in coverage.py 5.0b2.

[JoaRiski]

Thank you for the change @nedbat!

[davidszotten]

thanks ned, this works great! (my use-case is also docker) for lines and branches, though not for dynamic_context = test_function (i get no contexts reported)

created a separate issue #900

[olivertso]

Thanks @nedbat , this worked very well for me.

[engstrom]

Apologies for replying to an old thread, but this was the top result when searching for an issue I was having with uploading coverage to Code Climate and the solution was to remove the prefix argument from my cc-test-reporter format-coverage command. That solved the Invalid path part \"/\" error I was getting. Hopefully this is helpful for someone else down the road.