Pytest: testing: migrate pytest's own tests from the testdir fixture to the pytester fixture

[bluetech]

pytest has many tests for testing itself, under the testing/ directory in this repo. Many of the tests invoke pytest and check its output and outcomes. pytest provides a fixture called pytester which aids in this, you can find its documentation here.

The pytester itself is new in pytest 6.2 (unreleased yet when this is written), but its functionality is also provided by an older plugin called testdir. The two fixtures have almost the same API, the only difference is that testdir takes and returns py.path.local objects, while pytester is based on pathlib.Path, which we are trying to transition to.

Almost all of pytest's test are currently written using testdir. This issue is about migrating to use pytester instead. The instructions for converting are:

• Pick a file under testing/ which uses testdir.
• Go over each test which uses tesdir and change it to pytester. Beyond just replacing the name, some adjustments might be needed for using pathlib Paths instead of py.path.local paths.
• Bonus points: add a type annotation for the pytester argument.

Example:

Before:

def test_rootdir_wrong_option_arg(testdir):
    result = testdir.runpytest("--rootdir=wrong_dir")
    result.stderr.fnmatch_lines(
        ["*Directory *wrong_dir* not found. Check your '--rootdir' option.*"]
    )

After:

# At this import at the top of the file.
from _pytest.pytester import Pytester

def test_rootdir_wrong_option_arg(pytester: Pytester) -> None:
    result = pytester.runpytest("--rootdir=wrong_dir")
    result.stderr.fnmatch_lines(
        ["*Directory *wrong_dir* not found. Check your '--rootdir' option.*"]
    )

[symonk]

Thanks @bluetech I will start with testing/test_mark.py

Should this be branched off somewhere else than master for 6.2? or be destined for somewhere else? (I assume master is ok for both).

Completed (pending review):

• test_mark.py
• test_warning_types.py

[nicoddemus]

Should this be branched off somewhere else than master for 6.2? or be destined for somewhere else? (I assume master is ok for both).

Yes master is fine. 👍

[symonk]

test_stepwise next on the hit list, i've been refactoring the plugin so I want to follow up and port that over

[cmecklenborg]

I'm working on test_setuponly now, then will go back and try to finish test_assertrewrite

[pillemer]

Hi, I'd love to help here. I'm new to contributing so let me know if I should be doing anything differently.
I can do test_runner_xunit.py, if that's ok?

[symonk]

Thanks @pillemer & @cmecklenborg! (I had a skim of your branch @pillemer and it looks good for a PR) Welcome to the project!

[duthades]

Hi, I would like to contribute. I am working on testing/test_session.py.

[itsmegarvi]

hi , I am a newbie , first time contributing . I would like to contribute to test_setupplan.py

[JosiasAurel]

Hi, I am new to contributions and will like to contribute to testing/test_faulthandler.py

[JosiasAurel]

Here is my PR.

[locknic]

I am new to contributions and would like to contribute to testing/acceptance_test.py

[bot2x]

Hello people. Starting my journey of contributing to open source project here. I would like to contribute by working on test_warnings.py

[bluetech]

@bot2x Let us know if you need any help!

[bot2x]

@bot2x Let us know if you need any help!

Thank you for being so welcoming. I made the necessary changes and have created a pull request - https://github.com/pytest-dev/pytest/pull/8104#issue-533166082. I was not sure how else to reference it here.

[antonblr]

👋🏻 I would like to updatetest_capture.py and, perhaps test_config.py

[nicoddemus]

@antonblr please go ahead! I would suggest to start with the smallest one and open a PR with that. 👍

[bluetech]

I'm happy to say this issue is complete.

Thanks to all who contributed: @antonblr @bluetech @bot2x @cmecklenborg @crricks @duthades @itsmegarvi @JosiasAurel @locknic @nicoddemus @pillemer @proggga @symonk

It is very useful as a part of the effort the clean up the legacy py.path from pytest.

Hope to see you more in the future!