Biopython: Improving test coverage

[peterjc]

We have for some time run the continuous integration tests on TravisCI via the Python tool coverage and uploaded the metrics to https://codecov.io/github/biopython/biopython/

For Biopython 1.69 the test coverage reported was about 80%, as we approach Biopython 1.70 we are now up to 84%.

This has been a combination of removing some deprecated code, and adding more tests - including efforts like #820 for moving the remaining self-tests blocks under Bio/ into our test framework under Tests/ where those tests will be run regularly via continuous integration.

Note that currently our TravisCI configuration deliberately DOES NOT include the online tests (we are using python run_tests.py --offline), and due to installation time overheads DOES NOT have all the optional dependencies installed. This means the numbers on CodeCov.io are in a sense under-reporting our potential true test coverage.

I would like us to improve the coverage further - the sunburst image from CodeCov.io which takes into account the lines of code and percentage coverages highlights particular targets including Bio.Nexus, Bio.codonalign, Bio.KEGG, Bio.PopGen.

[Gasta88]

I'm particularly interested in Bio.KEGG.REST thanks to the recent development with the dbfetch module.

From CodeCov.io it seems that the private method _q() is never covered. Likewise, most exceptions are not tested.

I could try to expand the test script Tests\test_KEGG_online.py to verify the content of the handles and perform some fuzzy testing. Does it sound OK?

[peterjc]

@Gasta88 Right now (deliberately) the TravisCI tests are all run in offline mode, so none of the network only code in Biopython will be reported as covered. Please run coverage locally before trying to improve the online test coverage, as otherwise you might be wasting some effort.

Regarding Bio.KEGG, apart from the (online) REST.py file there is poor coverage in Bio/KEGG/__init__.py (looks like helper functions for output), Bio/KEGG/Enzyme/__init__.py and Bio/KEGG/Compound/__init__.py. The KGML folder looks better percentage-wise.

[peterjc]

Good news: I've enabled AppVeyor for continuous integration under Windows:

https://github.com/biopython/biopython/commit/09ba3065d191d204e860d67954d7f1ef7bb4a1b0
https://ci.appveyor.com/project/biopython/biopython/history

Bad news: Some of the Bio.Cluster doctests in the Tutorial would not work, so have been disabled:

https://github.com/biopython/biopython/commit/c2e2b388f2adc62bd0b143687091d2dc469c5b33
https://ci.appveyor.com/project/biopython/biopython/build/0.0.1

This means the CodeCov report dropped from about 84.7% to 84.2% (numbers revised).

However, it ought to be fairly simple to use the doctests to write a few more unittest based tests which will not be as fragile as a doctest, and ought to be reliable cross-platform.

[MaxGreil]

@peterjc are there still particular targets left for improved test coverage in this issue?

[peterjc]

Looking over https://codecov.io/gh/biopython/biopython/tree/master/Bio the ones which catch my eye are Bio.UniProt, Bio.NMR and Bio.Pathway (also Bio.Entrez scores badly but that has lots of online only tests which are not being counted here).

[svalqui]

I'll have a look at Bio.NMR

[svalqui]

I'll have a look at Bio.Uniprot

[svalqui]

I'll have a look at extending coverage for Bio.UniGene a bit more.

[peterjc]

Having had a quick look at the figures, Bio.Geo is another good target - it looks like adding a test for the record's __str__ method would bump that up a lot:

https://codecov.io/gh/biopython/biopython/src/master/Bio/Geo/Record.py

[svalqui]

@peterjc something has happened on codecov that it doesn't highlight the code anymore; there is an issue raised here:
https://github.com/codecov/codecov-python/issues/207
but that project has not been updated much lately, I might be related to changes on this project:
https://github.com/codecov/codecov-bash
having a brief look into other projects, some are working and others aren't; so is unclear if is codecov or we are missing some configuration.
Any ideas?

[peterjc]

I'd noticed the CodeCov issue last week or so - I was hoping it was a temporary glitch. Fingers crossed they can fix it (or point people at any configuration change).

There are/were alternative platforms, but CodeCov was the nicest at the time we set this up.