Pest: Missing support to Infection PHP

[nunomaduro]

Infection is a PHP mutation testing framework based on AST (Abstract Syntax Tree) mutations: https://infection.github.io/guide.

Problem: When running Infection against a test suite written in Pest, Infection will use the phpunit binary instead causing the following exception:

Pest\Exceptions\InvalidPestCommand: Please run `./vendor/bin/pest` instead
  of `/vendor/bin/phpunit`. in /Users/nunomaduro/laravel/vendor/pestphp/pest/
  src/TestSuite.php on line 106

How to fix this issue: Develop a Pest adapter and pull request it to the Infection repository.

[owenvoke]

I might take a quick look at this tomorrow, just to see how it's done. 👍

[owenvoke]

Right, I've taken a look and this is probably going to be complicated.

New adapters are created as an additional repository, I was thinking of a pestphp/infection-adapter or, if Infection want to have it, an infection/pest-adapter package. I've created a test version that is partially working, after making sure it uses the right executable, etc.

However it looks like the TestFrameworkConfigLocator class is hardcoded to use {path}/{frameworkName}.{extension}, whereas Pest uses the phpunit.xml/phpunit.xml.dist file. 😕 I can get round this by copying the phpunit.xml.dist to pest.xml.dist.

But after that, and some tweaking (hacks?)...

[nunomaduro]

@owenvoke I guess we can create our own repository for this. Let's call it pest-plugin-infection.

[nunomaduro]

This afternoon I going to create the template for that repository. Give me a couple hours.

[owenvoke]

It looks like the only changes we'd need to PR to Infection is to add a Pest constant and entry to the TYPES constant in TestFrameworkTypes. 🎉 (At least, in master)

[maks-rafalko]

Hello, Infection's creator here.

Thanks for looking into this and creating this issue.

@owenvoke can you please share an adapter you created for Pest? I wonder how you deal with TestFrameworkAdapter interface and determining whether the test failed or not.

Regarding adapter: I would suggest creating an official repository at infection/pest-adapter to be in line with already 3 implemented:

• infection/phpspec-adapter
• infection/codeception-adapter
• phpunit adapter (currently bundled into the core - infection/infection

You can be maintainers of this repo if that matters.

Why "official" repository is better? because

• we have a feature to automatically install adapters when the user runs Infection and adapter is not found (see https://github.com/infection/infection/blob/0e83204401a077924718b32e2323dbe335e75f98/src/TestFramework/AdapterInstaller.php#L48-L86)
• we have for example extensions autodiscovery feature (read here https://github.com/infection/extension-installer)
• I would avoid adding any constants to the Infection core for non-official adapters just because they are not under control and we can't guarantee they are up to date. Probably, if you decide not to move it there, we need to find another way

FYI, adapters should have a special composer package's type infection-extension to be autodiscovered and autoinstallable by our extensions-installer, e.g. see https://github.com/infection/codeception-adapter/blob/09dda58180bc3ba7ca3cb865ba0d0d84dfe3ef5a/composer.json#L4

Didn't dig into the Pest yet, but test framework should have some required features to be able to work efficiently with Infection, some of them are described here https://github.com/infection/infection/issues/45#issuecomment-332168173. I guess if the Pest is just a wrapper for PHPUnit and works with PHPUnit's config, it shouldn't be an issue.

Anyway, I'm happy to help here.

[theofidry]

@maks-rafalko I think it's also an opportunity to adapt the config locator.

I don't think it makes sense anymore to have it in the core with hard-coded path when its responsibility is to locate the test framework config path, which depends on the test framework itself.

One way would be to move the interface directly in https://github.com/infection/abstract-testframework-adapter or it could even be completely hidden in the adapter, leaving the responsibility to have a dedicated class for that or not to each individual adapters. A non trivial question though is how all of that should interact with the TestFrameworkConfigPathProvider (used for initializing Infection).

[nunomaduro]

Alright, I love infection - but at the moment I've no plans to work on this. Hopefully in the future someone will pick this.

[tomasnorre]

Missing a sad smiley. But you are the maintainer, it's your call.

[nunomaduro]

@tomasnorre Yeah - unfortunately my free time is very limited. And of course, someone else that needs infection in their test suite can submit a pull request that addresses this issue.

[tomasnorre]

I totally get that, I wasn't meant as a blame, was just hoping for it.

We just started using pest and I think that mutation tests are really valuable. Perhaps I get to the point where I will make a PR. But like you, I have limited free time too :)

Stay safe and take care.