Pytest: Show run-time (total or mean?) per module

Created on 14 Dec 2017 · 5Comments · Source: pytest-dev/pytest

In the latest version (3.3+ I think), pytest shows an overall percentage of completion in a column behind the test-module names.

I have a couple of modules which take quite a long time, and identifying slow-running test modules might help optimise the tests a bit. But I'm not always actively looking at the console output while the tests are running.

Having some kind of timing information might be interesting.

The existing --durations argument only shows the runtime of the top-n slowest unit-tests. But if you have a couple tests which cannot be improved for speed, those always show up in that list which may hide other slow-running tests.

For example, imagine a sub-optimal setup/teardown on a module. This can easily increase the overall run-time very quickly and can also be easy to remedy. Identifying modules where the unit-tests run on average slower than on other modules might help.

reporting enhancement proposal

Source

exhuma

Most helpful comment

Hey @exhuma,

I think the idea is good, but I think once we do that we might start to come up with other interesting measurements (for instance, sort by fixture setup/teardown instead of total module time) and that doesn't scale well to add to the core I think.

Perhaps creating a plugin (pytest-timings? 😉) might be a better approach in the long run, because it might grow interesting new measurements on its own.

nicoddemus on 14 Dec 2017

👍2

All 5 comments

GitMate.io thinks the contributors most likely able to help are @RonnyPfannschmidt, and @nicoddemus.

pytestbot on 14 Dec 2017

Hey @exhuma,

Perhaps creating a plugin (pytest-timings? 😉) might be a better approach in the long run, because it might grow interesting new measurements on its own.

nicoddemus on 14 Dec 2017

👍2

Agreed... also, adding too much information in the default console output might become.......... messy. And measurements which are interesting for one person might not be for another one, so you can't really make it right for everyone right off the bat.

exhuma on 15 Dec 2017

@exhuma
I think that this use-case falls well into pytest-harvest. it allows you to get synthetic information (status, duration, parameters, ... and even fixtures and results if you're interested) related to all test nodes in a given scope (session, module, specific test objects). See for example this table.