Phpunit: log-junit output missing files attribute in nested testsuite

[tburschka]

| Q | A
| --------------------| ---------------
| PHPUnit version | 7.1.5
| PHP version | 7.2.7
| Installation Method | Composer

I'm not sure if the reason is the format phpunit generates or the parser sonarqube using, but in the case of using dataProvider fo a test a new <testsuite> node will be opened as child node (instead of <testcase>). Sonar seems to support nested testsuites, but the file attribute is mandatory then, which is not generated.

Example:

<!-- ... -->
<testsuite name="Some\ImportantTest" file="/var/www/tests/Some/ImportantTest.php" tests="32" assertions="150" errors="0" failures="0" skipped="0" time="0.073943">
    <testcase name="test__construct" class="Some\ImportantTest" classname="Some.Tests.ImportantTest" file="/var/www/tests/Some/ImportantTest.php" line="178" assertions="17" time="0.003288"/>
    <testsuite name="Some\ImportantTest::testFormatValue" tests="7" assertions="11" errors="0" failures="0" skipped="0" time="0.007276">
        <testcase name="testFormatValue with data set #0" class="Some\ImportantTest" classname="Some.Tests.ImportantTest"  file="/var/www/tests/Some/ImportantTest.php" line="575" assertions="2" time="0.001235"/>
        <testcase name="testFormatValue with data set #1" class="Some\ImportantTest" classname="Some.Tests.ImportantTest"  file="/var/www/tests/Some/ImportantTest.php" line="575" assertions="2" time="0.001007"/>
        <!-- ... -->
    </testsuite>
</testsuite>
<!-- ... -->

[keradus]

according to JUnit v10 spec ( eg https://github.com/FriendsOfPHP/PHP-CS-Fixer/blob/2.12/doc/junit-10.xsd ), file attribute is optional.
Can you reference spec for Sonar? I don't know which JUnit version they follow

[tburschka]

I don't now the specification, but with the warning from SonarQube i suspected the format was wrong:

[SonarQube] WARN: Test cases must always be descendants of a file-based suite, skipping : Some\ImportantTest.someTestCase with data set #1 in Some\ImportantTest:someTestCase
[SonarQube] WARN: Test cases must always be descendants of a file-based suite, skipping : Some\ImportantTest.someTestCase with data set #2 in Some\ImportantTest:someTestCase

[keradus]

question here is what is the standard, is it wrong implementation of JUnit report in PHPUnit (according to v10, it is not), or too greedy SonarQube that does not respect the spec.
IMHO, later, or please provide JUnit spec that requires that attribute

[tburschka]

Since the report "works" while there is a warning SonarQube seems to respect the spec, but has trouble to locate the correct file for the nested testsuite. You're right that it is optional, while i didn't see the problem to add it to nested testsuites?

[keradus]

I'm thinking about bigger picture here, to fulfill the spec fully. This one attribute is just an example. Yet for that, we need to align on proper spec, that's why I asked, on the very beginning, for spec that Sonar is using

[sebastianbergmann]

There is no spec. The JUnit maintainers (and others) are trying to come up with one (for a (maybe) completely new logging format) in https://github.com/ota4j-team/opentest4j/issues/9. That is what https://github.com/sebastianbergmann/phpunit/issues/3162 is about.

[tburschka]

while it's nice to have a better specified and supported format, the question remains if the attribute can be added in the current junit xml?

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity within the last 60 days. It will be closed after 7 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity within the last 60 days. It will be closed after 7 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had activity since it was marked as stale. Thank you for your contributions.

[epdenouden]

Added this issue to my research notes for reimplementing the TestHook system