Pytest: Check 6.0.0rc1 against Fedora packages

[nicoddemus]

That makes it simple to just look at the milestone and see the outstanding issues for the release.

_Originally posted by @nicoddemus in https://github.com/pytest-dev/pytest/issues/7528#issuecomment-662411170_

Fedora PR with comments: https://src.fedoraproject.org/rpms/pytest/pull-request/17

[nicoddemus]

@hroncok as I mentioned, just created this issue to give this some visibility regarding the 6.0 milestone. Please don't take this trying to create pressure of any kind from our side, take your time or even go ahead and close this if you are OK with 6.0.0rc1 or won't have the time to continue testing other Fedora packages. We definitely appreciate all your efforts on testing pytest and helping out the project. 😁

[hroncok]

What I'd like to see addressed before the final 6.0.0 release:

https://github.com/pytest-dev/pytest-xdist/pull/554#issuecomment-660981785
https://github.com/pytest-dev/pytest-forked/issues/45
https://github.com/pytest-dev/pytest-xdist/pull/558 release
https://github.com/pytest-dev/pytest/issues/7534

[hroncok]

The analysis is basically done. I can dig deeper if you think so.

[The-Compiler]

I'd be interested in what happened for the python-sphinxcontrib-programoutput failure (logs). I can't reproduce it locally (and I note upstream seems to not use pytest for their tests). Here's the relevant part of the logs:

+ cd sphinxcontrib-programoutput-0.14
+ PYTHONPATH=build/lib/
+ py.test-3.9 -v build/lib/sphinxcontrib -k 'not test_standard_error_disabled'
============================= test session starts ==============================
platform linux -- Python 3.9.0b5, pytest-6.0.0rc1, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /builddir/build/BUILD/sphinxcontrib-programoutput-0.14
collecting ... collected 0 items / 5 errors

==================================== ERRORS ====================================
__ ERROR collecting build/lib/sphinxcontrib/programoutput/tests/test_cache.py __
ImportError while importing test module '/builddir/build/BUILD/sphinxcontrib-programoutput-0.14/build/lib/sphinxcontrib/programoutput/tests/test_cache.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'sphinxcontrib.programoutput'

No idea why the import would fail here, given that PYTHONPATH is set in a way that should allow it to be imported. Also no idea what could've changed in pytest here, and as I can't reproduce locally I can't bisect.

[hroncok]

Note that running the tests on build/lib/sphinxcontrib is a hack. The problem is we have no way to actually install the package in the sphinxcontrib namespace and run the tests on installed, during rpmbuild.

[The-Compiler]

One down, three to go:

• [x] Deprecation warning with xdist: https://github.com/pytest-dev/pytest-xdist/pull/554#issuecomment-660981785 (causing failures for some testsuites using xdist and -Werror) - fixed via https://github.com/pytest-dev/pytest-xdist/pull/563
• [x] xdist release after the above and pytest-dev/pytest-xdist#558
• [x] pytest-dev/pytest-forked#46 (and release?)

Should we wait a bit more until those are taken care of before releasing the final 6.0?

[hroncok]

Since many CI setups ignore prereleases and will start to pick up 6.0.0 once released, I'd advise to hold on with the release until the issues are fixed (and released).

[nicoddemus]

Since many CI setups ignore prereleases and will start to pick up 6.0.0 once released, I'd advise to hold on with the release until the issues are fixed (and released).

Agreed, thanks for the list @hroncok!

I will try to get to those this weekend. 👍

[webknjaz]

@The-Compiler

• [x] pytest-dev/pytest-forked#46 (and release?)

Fixed

[The-Compiler]

Yay! So I think the only thing left is a pytest-xdist release now, correct?

[nicoddemus]

pytest-xdist-1.34.0 released. 😄

Closing this then, thanks everyone!