Pycodestyle: Convert existing unit tests from unittest to pytest

[ArcTanSusan]

This gist contains the abundantly verbose output of python pycodestyle.py --testsuite testsuite.

It's really hard to tell which of the test methods has failed because of style problems:

https://gist.github.com/ArcTanSusan/c8b3bfcb12d393121c3708ebf4486b7c#L339

[sigmavirus24]

You'll want to look at https://github.com/PyCQA/pycodestyle/blob/master/testsuite/support.py#L182 to see where it goes from there. That's roughly how --testsuite works. Frankly, I've never understood why that was in the tool to begin with.

[IanLee1521]

With the new ideas I have of a refactor, I suspect I will migrate the test functionality out of the tool and turn to something more like nose or py.test to handle the running of the tests instead.

[sigmavirus24]

@IanLee1521 personally, I'd like to collaborate with you on this because I think Flake8 3.0 will want to adopt and contribute back to a repository of said test cases (for it's functional test suite that is still not a thing but will be!). I suspect once we have these in a separate repository, we may also be able to entice pylint to contribute and use them as well.

[sigmavirus24]

To be clear, I'm thinking of these test cases as something similar to Cryptography's Vectors (https://github.com/pyca/cryptography/tree/master/vectors).

[ArcTanSusan]

If I understand the order of operations for refactoring tests, these are the steps in summary:

1. Refactor testsuite/support.py to output test failures in a coherent read-able way on stdout.
2. Refactor testsuite to switch entirely to either pytest or nosetests.
3. In long-term future, create a new repo of functional tests that which pycodestyle, pylint, flake8 can all share as single source of Truth.

[sigmavirus24]

My vote would be for pytest. I'll let @IanLee1521 make the final call though.

[The-Compiler]

FWIW the nose documentation recommends not using nose :wink:

[IanLee1521]

Ah, I hadn't noticed that nose was not being maintained. I mostly just use it as the test runner of my unittest[2] tests. Guess it's time to learn py.test... ;)

[ArcTanSusan]

I'll update the title of this issue. I can attempt to do the initial conversions to pytest.

[The-Compiler]

Let me know if you need help! Unfortunately I don't have time for doing the grunt work right now (sorry!), but I'm happy to help if you get stuck somewhere.

[sigmavirus24]

Flake8 3.0 is using py.test so I can help too if necessary.

Also note, that to reduce the number of changes, you can continue using unittest practices with py.test and make a first pass be switching the test runner to py.test. Then you can incrementally move the tests to py.test practices (this will keep the diffs small and easy to review and merge).