Infection: Using `defaultTestSuite` in phpunit.xml.dist generates phpunit configurations that do not run tests

Created on 15 Nov 2018 · 3Comments · Source: infection/infection

| Question | Answer
| ------------| ---------------
| Infection version | 0.9.3
| Test Framework version | PHPUnit/PhpSpec x.y.z
| PHP version | PHPUnit 6.5.13
| Platform | Ubuntu
| Github Repo | -

When using the defaultTestSuite option in our phpunit.xml.dist, Infection will generate an incorrect filtered testsuite (the .xmls created in /tmp).

Our source phphunit.xml.dist has:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
    colors="true"
    bootstrap="./tests/bootstrap.php"
    verbose="true"
    backupGlobals="false"
    defaultTestSuite="unit"
    beStrictAboutTestsThatDoNotTestAnything="true"
    beStrictAboutOutputDuringTests="true"
    beStrictAboutChangesToGlobalState="true"
    beStrictAboutTodoAnnotatedTests="true"
>

    <testsuites>
        <testsuite name="integration">
            <!-- omitted -->
        </testsuite>
        <testsuite name="unit">
            <!-- omitted -->
        </testsuite>
    </testsuites>

And the generated phpunitConfiguration.<code>.infection.xml looks like:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
        colors="false"
        bootstrap="/tmp/infection/interceptor.autoload.423b602b6e0091a29f84d236150f6771.infection.php"
        verbose="true"
        backupGlobals="false"
        defaultTestSuite="unit"
        beStrictAboutTestsThatDoNotTestAnything="true"
        beStrictAboutOutputDuringTests="true"
        beStrictAboutChangesToGlobalState="true"
        beStrictAboutTodoAnnotatedTests="true"
        stopOnFailure="true">
  <php>
    <ini name="memory_limit" value="1G"/>
  </php>
  <testsuites>
    <testsuite name="Infection testsuite with filtered tests">
            <!-- omitted -->
    </testsuite>
  </testsuites>

Note that the configured test suite name is now Infection testsuite with filtered tests, and combined with defaultTestSuite="unit" means the test won't run. Here's me running phpunit with that configuration:

$ vendor/bin/phpunit --configuration /tmp/infection/phpunitConfiguration.423b602b6e0091a29f84d236150f6771.infection.xml --stop-on-failure
PHPUnit 6.5.13 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.2.12-1+ubuntu16.04.1+deb.sury.org+1 with Xdebug 2.6.1
Configuration: /tmp/infection/phpunitConfiguration.423b602b6e0091a29f84d236150f6771.infection.xml



Time: 25 ms, Memory: 6.00MB

No tests executed!

This results in 100% covered code MSI when running vendor/bin/infection:

$ vendor/bin/infection 
You are running Infection with xdebug enabled.
    ____      ____          __  _
   /  _/___  / __/__  _____/ /_(_)___  ____ 
   / // __ \/ /_/ _ \/ ___/ __/ / __ \/ __ \
 _/ // / / / __/  __/ /__/ /_/ / /_/ / / / /
/___/_/ /_/_/  \___/\___/\__/_/\____/_/ /_/

Running initial test suite...

PHPUnit version: 6.5.13

   23 [============================] < 1 sec

Generate mutants...

Processing source code files: 9/9
Creating mutated files and processes: 13/13
.: killed, M: escaped, S: uncovered, E: fatal error, T: timed out

............S                                        (13 / 13)

13 mutations were generated:
      12 mutants were killed
       1 mutants were not covered by tests
       0 covered mutants were not detected
       0 errors were encountered
       0 time outs were encountered

Metrics:
         Mutation Score Indicator (MSI): 92%
         Mutation Code Coverage: 92%
         Covered Code MSI: 100%

Please note that some mutants will inevitably be harmless (i.e. false positives).

Bug Good First Issue / Easy Pick

Source