Php-cs-fixer: Backwards-Incompatible changes since 2.11.2

[vbartusevicius]

In https://github.com/FriendsOfPHP/PHP-CS-Fixer/blob/v2.11.2/tests/Test/AbstractFixerTestCase.php#L72 you changed how fixers are instantiated when running tests.

In our company we extended php-cs-fixer by creating a lot of custom rules - https://github.com/paysera/lib-php-cs-fixer-config.
All Tests are written in same manner:

namespace Paysera\PhpCsFixerConfig\Tests\Fixer\PSR1;

use Paysera\PhpCsFixerConfig\Fixer\PSR1\ClassConstantUpperCaseFixer;
use PhpCsFixer\Test\AbstractFixerTestCase;

final class ClassConstantUpperCaseFixerTest extends AbstractFixerTestCase
{
    /**
     * @dataProvider provideFixCases
     */
    public function testFix($expected, $input = null)
    {
        $this->doTest($expected, $input);
    }

    public function provideFixCases()
    {
        return [/* test cases */];
    }

    public function createFixerFactory()
    {
        $fixerFactory = parent::createFixerFactory();
        $fixerFactory->registerCustomFixers([
            new ClassConstantUpperCaseFixer(),
        ]);
        return $fixerFactory;
    }

    public function getFixerName()
    {
        return 'Paysera/psr_1_class_constant_upper_case';
    }
}

Since 2.11.2 all tests fails since you changed how AbstractFixerTestCase::createFixer works - now it simply creates Fixer by replacing class name with regex. The regex only works for Fixer tests starting with PhpCsFixer namespace. In case of custom vendor namespace, the Fixer instance in tests becomes the test class itself:

var_dump(preg_replace(
    '/^(PhpCsFixer)\\\\Tests(\\\\.+)Test$/', 
    '$1$2', 
    'Paysera\\PhpCsFixerConfig\\Tests\\Fixer\\PSR1\\ClassConstantUpperCaseFixerTest'
)); 
// string(73) "Paysera\PhpCsFixerConfig\Tests\Fixer\PSR1\ClassConstantUpperCaseFixerTest"

phpunit output:

5) Paysera\PhpCsFixerConfig\Tests\Fixer\PSR1\ClassConstantUpperCaseFixerTest::testFix with data set #4 ('<?php class Sample {\n        ...     }', '<?php class Sample {\n        ...     }')
Error: Call to undefined method Paysera\PhpCsFixerConfig\Tests\Fixer\PSR1\ClassConstantUpperCaseFixerTest::supports()

/home/augustas/Projects/lib-php-cs-fixer-config/vendor/friendsofphp/php-cs-fixer/tests/Test/AbstractFixerTestCase.php:116
/home/augustas/Projects/lib-php-cs-fixer-config/tests/Fixer/PSR1/ClassConstantUpperCaseFixerTest.php:18

[kubawerlos]

Class AbstractFixerTestCase is marked as internal and "code marked with the @internal tags are excluded from our Backward Compatibility promise" (from here and AFAIK @keradus follows that), so it's all good.

Also, why don't you override createFixer instead of createFixerFactory? Would be much simpler.

[SpacePossum]

Hi @vbartusevicius ,

Great to hear you use fixer and got a lot of custom fixers, knowing about integrations and usage is always nice.
Sadly last fixer release doesn't work out very good for you. The AbstractFixerTestCase is marked as @internal, meaning it is not a part of the API on which we keep a BC promise. Building on top of this kind of code is risky as we might change behavior of it in any release.
I would advice to change you test-classes to extend a base class of your own, which might in the beginning even extend the AbstractFixerTestCase and from there start removing the dependency until you got what you need.
Bit of background, we tried to provide some base test-classes as part of the API so others could use those for their own. This proved to be difficult in terms of making releases and is therefore removed (where possible without BC breaks). I don't think we'll get back to supporting tests as part of the API soon.

edit: @kubawerlos already answered when I was typing, thanks :+1:

[vbartusevicius]

@SpacePossum - thank you for explanation.
Sadly this tells me that php-cs-fixer is basically limited to run only with built-in fixers as PhpCsFixer\AbstractFixer is also internal and is needed to create custom fixers. Of course wrapper/proxy class approach can be used, but still no guarantees it will not break with next patch release.

[SpacePossum]

We'll keep a BC promise on the various interfaces for the fixers, relying on those will be safe. Combining that with a base-class of your own is the best approach for 3rd party fixers and their integration.
The lack of base-classes for testing is a bit tedious at times, but is a compromise so we can evolve towards better testing of the fixers' own rules at any time.

Lemme know if can help out or if you've any questions about 3rd party rules, happy to help out :)