Phpunit: setUp and setUpBeforeClass run before filters/groups applied

[banks]

I'm not sure if this is intentional but I don't see the rationale if it is. I also couldn't find documentation or other issues to suggest this is a known/discussed topic.

Basically: I user the filter and/or groups options to limit the tests run by the test runner as documented.

I've realised though that since the code that calls setUp() and setUpBeforeClass() methods is _before_ the filtering is applied, (code link) every test in my suite will have setUp() run and every test case will have setUpBeforeClass(). This means any expensive setup work done is run for the whole suite even when the tests that require the work are not being run.

I'm aware that you can specify specific test case classes to run and thus avoid this but in a very large test suite, the flexibility of filter and groups is very useful. It comes with a huge cost of running potentially thousands of setup routines though just to run a single simple test.

If this is intentional behaviour, it would be great to understand the rationale behind it. If not, it appears a relatively simple fix and I am can fork and make the required changes.

Thanks for you time and effort on this PHPUnit!

[banks]

Am I the only person who thinks this is an issue? Certainly all other developers I work with agree that this appears to be totally unexpected and unnecessary behaviour?

I am totally willing to fix this myself and contribute the patch but I can't believe no one else could care about something that, in my experience, massively impacts on the usability of PHPUnit to construct and run large test suites.

Perhaps I'm missing something here. Please, anyone, share your thoughts on why this is written as it is and whether there is any reason not to simple move the setup calls to be after the filtering check.

[BRMatt]

I'd say this is a big issue. This bit me recently when a data provider relied on functionality that wasn't available. Even though the tests that used the provider had been excluded with groups the provider was still being invoked.

[dharkness]

I agree that this seems to be unintentional. I can think of no reason to run those methods if none of the tests in the test case will be run.

[naderman]

Just ran into this problem as well. Any news on this?

[tarjei]

Hi, the workaround is to add a static variable + a check in setup() :

if (self::$processManager === null) {
        self::$processManager = new ProcessManager();
        self::$processManager->startBackgroundJob(__DIR__ . "/../../../../../mongodb/run-mongo.php");
}

And then:

public static function tearDownAfterClass()
{
    if (self::$processManager !== null) {
        echo "\n\nKilling all pids!\n";
        self::$processManager->killAll();
    }
}

[banks]

@tarjei not sure how that helps?

The problem is not that the constructor runs multiple times. Given PHPunit's runner that is not surprising in some circumstances. The issue I'm complaining about is that the setup is run at all in the case where you are not actually running any tests from the class in question. In my case I don't see multiple setupBeforeClass runs for a single class, but I do see all of the setupBeforeClass methods for my entire testsuite run even though I used filter/groups to select only one specific test case.

[edorian]

Yes is this an issue related to #10.

With the current way of running the tests the filtering isn't applied before TestSuite.php executes the setup.

With a big test suite and a phpunit.xml what should work is using directory based filtering with phpunit tests/module/stuff but for example for the phpbb3 test suite that doesn't work out so there are cases where --filter is the only option.

The only idea i have to work around this is to lazy initialize your stuff in the first call to setUp but thats rather hackish.

I'm quite sure that this will be taken care of by #10 but if you need this sooner and the patch is rather small/manageable this might get into PHPUnit 3.7 (Not 3.6.x as it breaks bc).

The workaround by tarjei works as the code is in setUp and the self:: could just as well be self::$setupBeforeFirstTestCase

[banks]

The workaround by tarjei works as the code is in setUp and the self:: could just as well be self::$setupBeforeFirstTestCase

My apologies @tarjei, I misunderstood the intention.

@edorian, thanks for the comment yes #10 should fix this issue, I guess we'll have to wait and see.

[redgeoff]

Here is a post on SO that may help others who are still having problems with the filtering not preventing the fixtures from being run: PHPUnit filtering not working with setUpBeforeClass() and tearDownAfterClass()

I'm sorry if there is a better way that I have just missed in the docs.