Phpunit: PHPDBG with process isolation does not generate code coverage

[Majkl578]

PHPUnit does not generate code coverage for tests run with PHPDBG in process isolation.

For the following class + test:

Foo.php

class Foo {
    public function bar() {
        return 123;
    }
}

FooTest.php

/**
 * @runTestsInSeparateProcesses
 * @preserveGlobalState disabled
 */
class FooTest extends PHPUnit_Framework_TestCase {
    public function testBar() {
        $this->assertSame(123, (new Foo)->bar());
    }
}

When I run:

$ phpdbg7.0 -qrr vendor/bin/phpunit FooTest.php --coverage-text --whitelist Foo.php

all I get is 0% code coverage. Coverage with XDebug is correct.

[Majkl578]

Hmm, actually, it also reports 0 assertions with PHPDBG while it does 1 assertion with XDebug...

So I did some digging, using this script:

$php = new PHPUnit_Util_PHP_Default;
var_dump($php->runJob('<?php echo 123; ?>'));

I ran it under standard CLI and under PHPDBG. The results are different.
For CLI:

$ php7.0 -n test.php
array(2) {
  ["stdout"]=>
  string(3) "123"
  ["stderr"]=>
  string(0) ""
}

For PHPDBG:

phpdbg7.0 -n -qrr test.php
array(2) {
  ["stdout"]=>
  string(0) ""
  ["stderr"]=>
  string(0) ""
}

So apparently the problem seems to be in phpunit/src/Util/PHP/eval-stdin.php where php://input is empty (but the input is in php://stdin).

PHPUnit 5.1.3, PHP 7.0.1 + 7.0.2RC1, Linux.

[raziel057]

I encounter the same problem with the same environment.

phpdbg -qrr /usr/local/bin/phpunit -v -c app/ MyTest.php --coverage-html cov

Runtime:       PHPDBG 7.0.0-5+deb.sury.org~trusty+1
Configuration: /srv/ic2m/novento/app/phpunit.xml.dist

Time: 2.2 seconds, Memory: 152.66Mb

OK (2 tests, 6 assertions)

phpdbg -qrr /usr/local/bin/phpunit -v -c app/ MyTest.php --coverage-html cov --process-isolation
PHPUnit 5.1.3 by Sebastian Bergmann and contributors.

Runtime:       PHPDBG 7.0.0-5+deb.sury.org~trusty+1
Configuration: phpunit.xml.dist

And if I run without --coverage-html I got a memory limit (only with --process-isolation)

phpdbg -qrr /usr/local/bin/phpunit -c app/ MyTest.php --process-isolation
PHPUnit 5.1.3 by Sebastian Bergmann and contributors.

mmap() failed: [12] Cannot allocate memory
[PHP Fatal error:  Out of memory (allocated 23756800) (tried to allocate 9622369213780901352 bytes) in phar:///usr/local/bin/phpunit/phpunit/Util/GlobalState.php on line 166]

[jmauerhan]

Same problem here. Was about to post this bug report. Assertion is not run, and coverage is not recorded. Running the exact same tests with xdebug enabled for coverage shows 100% coverage, and the assertion runs.

[jmauerhan]

I still see this happening, on PHPDBG 7.0.4, PHPUnit 5.5.0. I really don't want to switch back to xdebug :( Any ideas? I'd be happy to help with a PR, but I have no idea where the problem is.

[Wendtly]

I just switched from xdebug to phpdbg 4.8.27 on php 7.0.9 (AWS supported versions). While the speed improvements on unit test are great, on average I'm getting 10% less code coverage reported than I did with xdebug. Haven't taken the time to figure out what exactly isn't being reported that used to be.

[sebastianbergmann]

I do not use phpdbg and will not invest time to investigate issues related to it.

[Majkl578]

@sebastianbergmann So why do you support it in first place?
https://github.com/sebastianbergmann/phpunit/blob/6aac3946af58486dce4d03ce3f2230b011d36975/src/Util/PHP/AbstractPhpProcess.php#L218

Sad to see 2 years old issue being closed with "I don't care" reason rather than asking OSS community for help... :(

[courtney-miles]

I'm just starting with phpdbg and I'm not sure the result of code coverage yet. But I was getting errors with tests that run in a separate process and it was because they would be ran using php instead of phpdbg.

The solution was to set the full path to phpdbg in an environment variable named PHP_BINARY. I.e.

export PHP_BINARY=/usr/bin/phpdbg

PHPunit will then use this as the path for php when running tests in a separate process.

If you are not setting PHP_BINARY before running PHPUnit with phpdbg, then this may be why you are missing code coverage.