Hypothesis: Example Database Viewer/Manager

[whatevergeek]

Apologies if there's already a way to do this.

Is there a way we can view the contents of Example Database?
I want to know what cases were already run so far for a particular test method.

I understand that the advice currently is to write the examples explicitly in code and not to depend on the examples database as they will be overridden during upgrades, etc.

Perhaps, it's good to have a way to store examples and maintain there somewhere instead of hardcoding examples in the code.

Any plans on the roadmap for the following?

• Resilient Example Database (i.e. we can choose to maintain our examples here instead of hard coding in source code)
• Example Database Viewer/Manager

[DRMacIver]

Is there a way we can view the contents of Example Database?

Not really, no.

Resilient Example Database (i.e. we can choose to maintain our examples here instead of hard coding in source code)

No, this is essentially impossible - the example database simply can't be robust against changes in the data generation, partly because of the underlying mechanism and partly because this would violate the rule that everything (with the exception of things you have explicitly requested with @example and thus are very visible) satisfies the invariants of your generated data.

A "robust" example database would have to basically be serialised Python objects, which is brittle in all sorts of excitingly different ways. The Hypothesis example database isn't that, it's just a record of how to construct the object given the test's specific data generation.

Example Database Viewer/Manager

This is in the category of "it would be nice to have, but is a large project that it is currently unlikely that anyone is going to sink any time into without it being funded or contributed by someone" (and TBH there's currently not much capacity for funded work). I expect at some point we will have one, but there's no roadmap or timeline towards that point at present.

[Zac-HD]

A database viewer could be part of #172, but as David says it's not likely to happen soon unless someone writes it for us (in which case, awesome!).

I want to know what cases were already run so far for a particular test method.

Try turning up the verbosity setting, and optionally emit extra information with hypothesis.note(...). If you'd prefer to aggregate and summarise the output, try using hypothesis.event(...) with the --hypothesis-show-statistics command-line argument.

Perhaps it's good to have a way to store examples and maintain them somewhere instead of hardcoding examples in the code.

Updating Hypothesis must be able to change the meaning of DB entries, or we can't improve example generation. If you have tests that need to run every time, @example(...) is the best solution - or just don't use Hypothesis for those tests! Fantastic as I think it is, other tools including traditional unit tests can be the right choice too.

[whatevergeek]

Hi @DRMacIver and @Zac-HD ,
Thanks for the prompt response and insights. I'll be sharing about hypothesis tomorrow in a local meetup. These info are handy in case someone has similar questions tomorrow. :-)