Phpunit: testdox does not work with filters

[bendavies]

| Q | A
| --------------------| ---------------
| PHPUnit version | master
| PHP version | 7.2.14
| Installation Method | git

reproduction:

git glone [email protected]:sebastianbergmann/phpunit.git
cd phpunit
composer install
./phpunit --testdox ## works
./phpunit --testdox --filter CliTestDoxPrinter

when adding a filter, a spinner is shown, testdox format is not printed.

[epdenouden]

@bendavies Thanks for the bug report, I have reproduced it here and will implement a fix.

If the spinner gets in your way, you can disable the spinner for now with either the --no-interaction CLI or noInteraction='true' configuration option.

[bendavies]

thanks!

[epdenouden]

@bendavies The fix has been released 🚚

./phpunit --testdox --filter CliTestDoxPrinter                                                                                                   ✔  15:13:07 
PHPUnit 8.1-g1ce7a827f by Sebastian Bergmann and contributors.

Runtime:       PHP 7.2.14 with Xdebug 2.6.0
Configuration: /Users/ewout/proj/phpunit/phpunit.xml

PHPUnit\Util\TestDox\CliTestDoxPrinterColor
 ✔ Colorizes diff in failure message  11 ms

PHPUnit\Util\TestDox\CliTestDoxPrinter
 ✔ Prints the class name of the test class  11 ms
 ✔ Prints the prettified method name  10 ms
 ✔ Prints check mark for successful test  9 ms
 ✔ Does not print additional information for successful test  11 ms
 ✔ Prints cross for test with error  12 ms
 ✔ Prints additional information for test with error  10 ms
 ✔ Prints warning triangle for test with warning  10 ms
 ✔ Prints additional information for test with warning  10 ms
 ✔ Prints cross for test with failure  11 ms
 ✔ Prints additional information for test with failure  11 ms
 ✔ Prints empty set symbol for test with failure  11 ms
 ✔ Does not print additional information for incomplete test by default  10 ms
 ✔ Prints additional information for incomplete test in verbose mode  11 ms
 ✔ Prints radioactive symbol for risky test  10 ms
 ✔ Does not print additional information for risky test by default  10 ms
 ✔ Prints additional information for risky test in verbose mode  11 ms
 ✔ Prints arrow for skipped test  10 ms
 ✔ Does not print additional information for skipped test by default  10 ms
 ✔ Prints additional information for skipped test in verbose mode  10 ms

[bendavies]

thanks!

[bendavies]

@epdenouden have just tried this on my project and doesn't seem fixed to me

vendor/bin/phpunit --filter PaymentToReceiptProfileTest --testdox
PHPUnit 8.0.5 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.2.14
Configuration: /Users/ben/Code/scotam/abacus-webapp/phpunit.xml.dist

 ◒ running tests

anything i can do to help you debug?

[epdenouden]

@bendavies Oh no, so annoying! I am sorry it has not been fixed for you. I will double check the fix has made it in 8.0.5 and works as expected.

• what terminal and environment do you try this on? We know the testdox-spinner can be a problem in the Gitlab terminal, for example, but you should see the filtered output
• is the project a public one and easy to install/container/test?

We'll get this bug fixed.

[bendavies]

@epdenouden ok, it seems the test results will be printed after ALL the tests have finished. until that time, the spinner will show. is that expected? that's completely different from phpunit 7 AFAIK?

this is reproducible in the phpunit testsuite, just add a few sleeps to make the tests slow. if the tests are fast it's hard to notice.

[epdenouden]

@bendavies Quick reply: yes, this is completely normal and indicates everything is working as expected. Also: yes! this is different from PHPUnit v7 and is a result of implementing a buffer in the TestDox printer to re-sequence the Testdox results for human readers.

[bendavies]

Ah I see, and that's a shame.
I use testdox to get quick feedback about which tests are running and how long they are taking to run.
Think of a functional test which takes 1+ minute to run with many tests.

I now get no feed back at all (until the end), so all the benefits i personally was getting from testdox are now gone.

thanks

[epdenouden]

@bendavies I was afraid you where going to mention this use case. This way of working is not supported by the upgraded TestDox printer. Removing things that are still useful to people is not a great feeling.

I'll propose a solution soon; let me know if you have any specific wishes or other bugs you noticed.

[epdenouden]

ah yes, #3567

[BafS]

Just to say, we have the same issue, I commented the group that we exclude as a workaround but it would be great if it can be fixed. Thanks for your efforts !