Coveragepy: coverage.control.Coverage.data_suffix has been removed in 5.0.0, but is not in the release notes

[eric-wieser]

The .data_suffix attribute on Coverage instances has been removed. This change isn't documented in the release notes.

Causes https://github.com/computationalmodelling/nbval/issues/129

[nedbat]

I guess I should be more disciplined. I didn't consider any of the data attributes of Coverage to be part of the publicly supported interface, since they aren't documented.

In 5.0 they have underscore prefixes. I can clarify this.

[takluyver]

I didn't write the coverage code in nbval, but looking at it, it looks like we're using this attribute for two things:

1. To extend the default data_suffix to make our own suffix - to keep the coverage collected by our subprocesses separate until the end of the test run.
2. To check for 'parallel mode' (data_suffix is True), in which case the merging is taken care of for us (but we suppress some warnings).

Is it possible to achieve the same goals with public APIs?

[nedbat]

If you want to see the name being used for your data file, CoverageData.data_filename() will tell you the full file name.

You can use Coverage.get_option to find out if parallel mode is set.

[nedbat]

I added some warnings and admonitions in 3875113f0e8a8ec333b61000fbe038a2725903f4

[vidartf]

@nedbat I originally wrote the code in nbval, and while it is long enough ago that I forgot the details, I did write a note in the code at the time:

Note about coverage data/datafiles:
When pytest is running, we get the pytest-cov coverage object.
This object tracks its own coverage data, which is stored in its
data file. For several reasons detailed below, we cannot use the
same file in the kernel, so we have to ensure our own, and then
ensure that they are all merged correctly at the end. The important
factor here is the data_suffix attribute which might be set.
Cases:
1. data_suffix is set to None:
   No suffix is used by pytest-cov. We need to create a new file,
   so we add a suffix for kernel, and then merge this file into
   the pytest-cov data at teardown.
2. data_suffix is set to a string:
   We need to create a new file, so we append a string to the
   suffix passed to the kernel. We merge this file into the
   pytest-cov data at teardown.
3. data_suffix is set to True:
   The suffix will be autogenerated by coverage.py, along the lines
   of 'hostname.pid.random'. This is typically used for parallel
   tests. We pass True as suffix to kernel, ensuring a unique
   auto-suffix later. We cannot merge this data into the pytest-cov
   one, as we do not know the suffix, but we can just leave the data
   for automatic collection. However, this might lead to a warning
   about no coverage data being collected by the pytest-cov
   collector.
Why do we need our own coverage data file?
Coverage data can get lost if we try to sync via load/save/load cycles
between the two. By having our own file, we can do an in-memory merge
of the data afterwards using the official API. Either way, the data
will always be merged to one coverage file in the end, so these files
are transient.

Any thoughts if there are better ways for achieving those goals other than what was outlined above? :)