Phpunit: Switch the order in which @before methods and setUp() are executed

[beberlei]

The methods that are tagged with @before are called _after_ the setUp() method.

That makes it impossible to use traits for some generic setup code and then use the already existing services from the generic setup to do a specialed setUp, example:

class MyTest extends PHPunit_Framework_TestCase
{
    use ContainerSetup;

    public function setUp()
    {
           $this->container->get('..'); // if ContainerSetup has a `@before` hook, its not called here yet.
    }
}

[TomasVotruba]

I would love this feature too.

:+1:

[sebastianbergmann]

Can you send a pull request, @beberlei? I think this can go into 4.5.

[Dr-Head]

👍 agreed

[giorgiosironi]

I want to open a PR for this. Some questions to avoid implementing the wrong thing:

1. what should be the behavior for @after? By simmetry it should be called after tearDown()?
2. what should be the behavior of @beforeClass and @afterClass? Will they follow the same pattern of being called respectively before and after setUpBeforeClass() and tearDownAfterClass()?

[chungth]

+1

[Thijmen]

+1

[deeky666]

+1

[KingMob]

+1 @giorgiosironi what's the status of PR 1722? I see it failed a CI check.

[giorgiosironi]

It failed the run on PHP 7 which was unstable at the time, no test failure on current versions of PHP.
My questions at https://github.com/sebastianbergmann/phpunit/issues/1616#issuecomment-105048397 stills stand to define some other behavior so that I can finish it.

[sebastianbergmann]

I don't know what the right behavior would be, @giorgiosironi. Mostly due to the fact that I don't use @before, etc.

[ghost]

+1

[sebastianbergmann]

This should do what you propose, @beberlei, right?

diff --git a/src/Util/Test.php b/src/Util/Test.php
index 1dcc921..28594de 100644
--- a/src/Util/Test.php
+++ b/src/Util/Test.php
@@ -806,19 +806,31 @@ public static function getHookMethods($className)

                 foreach ($class->getMethods() as $method) {
                     if (self::isBeforeClassMethod($method)) {
-                        self::$hookMethods[$className]['beforeClass'][] = $method->getName();
+                        array_unshift(
+                            self::$hookMethods[$className]['beforeClass'],
+                            $method->getName()
+                        );
                     }

                     if (self::isBeforeMethod($method)) {
-                        self::$hookMethods[$className]['before'][] = $method->getName();
+                        array_unshift(
+                            self::$hookMethods[$className]['before'],
+                            $method->getName()
+                        );
                     }

                     if (self::isAfterMethod($method)) {
-                        self::$hookMethods[$className]['after'][] = $method->getName();
+                        array_unshift(
+                            self::$hookMethods[$className]['after'],
+                            $method->getName()
+                        );
                     }

                     if (self::isAfterClassMethod($method)) {
-                        self::$hookMethods[$className]['afterClass'][] = $method->getName();
+                        array_unshift(
+                            self::$hookMethods[$className]['afterClass'],
+                            $method->getName()
+                        );
                     }
                 }
             } catch (ReflectionException $e) {

[sebastianbergmann]

As discussed with @beberlei, only the order for @before and @beforeClass must be changed.

[GrahamCampbell]

:-1:

[GrahamCampbell]

This breaks every single Laravel application.

[taylorotwell]

This is indeed a breaking change.

[sebastianbergmann]

I do not consider this a breaking change. The behavior was undefined before, now it is defined. You were relying on undocumented behavior, sorry.

[GrahamCampbell]

While the order of the before annotations was not defined, the fact they were after the setup method, was defined. This is breaking.

[marcioAlmada]

@GrahamCampbell could you explain how this "breaks every single Laravel application"?

I wonder if projects that use phpunit as a dependency, like codeception, would have issues too.

[GrahamCampbell]

Everyone will now get a fatal error because the setup method created an object that before methods then used.

[MadaraUchiha]

@GrahamCampbell That's not how breaking changes work. If you rely on undocumented, untested behavior, nothing is set in stone.

[GrahamCampbell]

I don't see how the actual details matter though. It's a breaking change, if you consider it's behaviour to have been defined.

[GrahamCampbell]

Just because something's undocumented, doesn't mean you should break it. There's plenty other stuff not documented that people would be unhappy about if it broke.

[GrahamCampbell]

That's an example of the mess this makes: https://travis-ci.org/StyleCI/StyleCI/jobs/94697689.

[taylorotwell]

Yeah, to me a breaking change is defined as something worked before and now it doesn't. That's "breaking" - as in you broke applications. Is that not a good definition of "breaking"?

[Ma27]

why not making the behavior configurable through the phpunit.xml?
We could release a phpunit 5.1.1 which fixes that and in phpunit 6 the old behavior can be dropped as breaking changes can be done in major releases according to semantic versioning.

[sebastianbergmann]

@Ma27 PHPUnit already has way too many configuration settings, so no on that.

[MadaraUchiha]

@taylorotwell No, if you rely, for example, on the existence of an internal global variable, and you use it for your purposes, and someone refactors that global variable away, it is not a breaking change.

Breaking changes are changes to the documented, tested, defined, public API. If you rely on behavior that only incidentally works from the way the code was written, but that behavior was not defined or tested, defining/testing/changing it does not constitute a breaking change.

Things don't need to be explicitly defined to be undefined, in order to be undefined.

[taylorotwell]

I'm going to look at this in a few minutes and see how hard it is for us to be agonstic about what order these things are run in.

[sebastianbergmann]

@taylorotwell The change has been reverted for PHPUnit 5.1 and was rescheduled for PHPUnit 6. If you want to look at something today ... why not look at https://github.com/laravel/framework/issues/10808 ;-)

[taylorotwell]

Just wanted to note that on the Laravel side we insulated ourselves from this change. Not sure if that effects you wanting to go ahead and put this in a 5.x series release since we were somewhat edge case in our usage.

[GrahamCampbell]

The change has been reverted for PHPUnit 5.1 and was rescheduled for PHPUnit 6.

:+1: for that, regardless of what laravel's doing.

[tmatsuo]

@sebastianbergmann
Do you still think it's not worthwhile to introduce a configuration?

I'm asking because currently PHPUnit 5.x has the order setUp -> @before (also setUpBeforeClass -> @beforeClass), and PHPUnit 6.x will change it, right? Then it's hard for us to build something reliable leveraging this feature that works on both. Are there any suggestions for that?

In particular, I'm trying to build an e2e test harness for a PaaS. It's very similar to the first comment from @beberlei but I'm actually trying to do the opposite. I want to run something before @beforeClass.

class MyTest extends PHPunit_Framework_TestCase
{
    use PaasDeploymentTestTrait; // It deploys an app in @beforeClass

    public static function setUpBeforeClass()
    {
           // I want to run this before the PaasDeploymentTestTrait's @beforeClass
           self::dumpConfigFromEnv(); 
    }
}

[mfn]

Came here because I upgraded phpunit6 in a Laravel 5.4 application and my own trait extensions broke because of this change.

Before this change, I was able to do this:

SomeTrait.php

trait SomeTrait {
    /**
     * @before
    */
    protected function someInitializationRequiresSetupBeingCalled() {
        $this->somethingInitializedInSetup->foo();
    }

TestCase.php

abstract class TestCase extends  BaseTestCase {
    protected function setUp() {
        $this->somethingInitializedInSetup = initialize_it();
    }
}

SomeTest.php

class SomeTest {
    use SomeTrait;
    public function testSomething() {
        // some test
    }
}

Previously, with these examples, it was possible to just by including SomeTrait you could run additional code after setUp.

The way Laravel augmented this is they have setUp which checks if a certain trait was loaded and calls its (trait specific) initializer method.

Of course it's possible to do it that way, but before I change my code I would rather find out if there is a better way?

Can I, by using @before or something similar, run code now between the setUp and the actual test method with phpunit6 so I don't have to be specific about it via my setUp?