Catch2: How to check code coverage by Catch Unit Tests?

[hassanfarid]

Description

This is more of a question, that how to check what section of code is not covered by Unit Tests?

[horenmar]

Your use-case is outside of scope for Catch, you will need a separate tool to check which lines were actually ran by your test suite.

One such tool is gcc's gcov, but you will need to do some research for your specific platform.

[ankit-shah2]

Hi @hassanfarid
Please share on how you measure coverage with catch2. I

[hassanfarid]

@ankit-shah2 - You can't do it with Catch framework.

[ankit-shah2]

@hassanfarid Ok. Then how to proceed with it? Can we integrate some external tool and proceed (like gcov mentioned above)? Which?
Also can you share tips on how to integrate.
I have already written UT's in catch and now I have been asked to get coverage percentage.

[Bu11etmagnet]

You need to build your unit tests with --coverage, run them, and then use gcov , or better yet, lcov to see which lines are being covered.

Here's a makefile snippet to get you started:

TEST_PROGRAMS := foo_test bar_test # example

coverage: checks
    lcov --directory . --capture --output-file my.covdata
    genhtml --legend --title my_coverage -o coverage_dir my.covdata

checks: prep
    ${MAKE} parallel-checks

prep:
    rm -f *.success *.fail # make sure they aren't left behind from a previous test
    rm -f *.gcda # this shouldn't be necessary!

## Phony target which depends on all the "run the test" targets.
## When parallel-checks is built, its "run the test" dependencies can be built in parallel.
parallel-checks: $(addsuffix -runq,${TEST_PROGRAMS} ${TEST_SCRIPTS})

## Utility target to build and run a test program.
## Faster turn-around time because not everything needs to be built
## and only one test is executed.
%-runq: %
    @echo "  RUN     " $<; rm -f $<.success $<.fail; \
    if ./$<  >$<.out 2>&1 || ./$<  >$<.out 2>&1; then \
        mv $<.out $<.success ; \
    else \
        mv $<.out $<.fail ; cat $<.fail; exit 1; \
    fi

Now, typing make coverage will build all unit tests, run them, and create nice HTML pages.

[horenmar]

Basically, collecting coverage needs either

• support from the compiler (gcov, llvm-cov, coverage-sanitizer), or
• hacky use of debugger (OpenCppCoverage).

Because Catch is a test framework, 1) is straight out (catch comes in after the compiler has already ran) and 2) would require us to create an out of process runner that understands platform specific debugger options.