Framework: Testing Issue - cached results in unit tests

[derrekbertrand]

• Laravel Version: 5.4.19 (5.3 also IIRC)
• PHP Version: 7.0, 7.1
• Database Driver & Version: N/A

Description:

When making multiple requests to the same route (I believe it is route specific), you sometimes get the same result as a previous HTTP request to that route. In my real environment, I'm attempting to test this flow in one test:

GET /api/v1/user/1 -> should return 404, we did not seed
GET /api/v1/user -> should return empty set
POST /api/v1/user -> should create user
GET /api/v1/user/1 -> should return new user; fails, returns old 404
GET /api/v1/user -> should return set of 1 users; fails, returns old empty set

Having failed inexplicably, I recreated the issue in a new repo (see below).

Steps To Reproduce:

Not sure what workflow to employ here... I have created a mostly blank app with two commits that reproduces the issue here. Run composer install and all that jazz. Run vendor/bin/phpunit. The lone test suite is under Features.

[sisve]

Controller instances are cached per route, as your tests show. Properties set in the controller constructor will be shared between all requests to that route during that test.

Can you show some real code where you get cached responses? Have you built some kind of per-controller-instance cache (like properties)?

The default setup()/teardown() will create a new application (and your routes) for every test, that's why the value/controller instance changes for every test.

[derrekbertrand]

Okay. Good to know this is by design! There is a demo here: https://github.com/derrekbertrand/l5testingtest

I guess I should keep my requests in separate tests, so that I don't accidentally bjoink myself. I can work with that now that I know what's up.

Is this mentioned in the docs? If not, I can submit a pull request to mention it.

EDIT:

Why are the controllers cached, other than speed? I would venture to say that most app developers would not want that behavior. I was hoping to recreate what happens in the wild.

[sisve]

Eh. Why am I not allowed to answer "speed"? You cannot just ask me for something and not allow me to state a reason...

This behavior has been with us for 10 months, since Laravel 5.3. I'm pretty sure that the anonymous mass "most app developers" doesn't write their code as you do.

Are you saying that the code is different "in the wild"? If so, can you show us a real controller that has this issue?

[crynobone]

I would venture to say that most app developers would not want that behavior.

In what scenario does app developer accessing multiple/same controller more than once in "most" application? Each web request boot a separated app instance with PHP.

[derrekbertrand]

What I'm getting at here is that $this->json(...), $this->post(...), and whathaveyou is not analogous to a request cycle. Application state is never preserved between requests in the real world, but it is between calls in a test. For example, say I wanted to test that deleting a resource twice sends a 200 OK and a 404 Not Found respectively - I think the below test is reasonable, but it would fail.

//in a test
$this->json('DELETE', '/api/user/7')
    ->assertResponseStatus(200);
$this->json('DELETE', '/api/user/7')
    ->assertResponseStatus(404); //this test would fail, you'd just get your 200 again because we are preserving state during a test

EDIT: Also, thanks to @crynobone for orchestra components. We've started using testbench internally, and it's been much better since. I no longer have to fake an app environment myself!

[crynobone]

In that case, you need to flush and recreate the application in between call (as if a real application) without going through seTup and tearDown

[derrekbertrand]

Sounds like a good idea! I can probably add a utility method like flushApp() to my TestCase.

I'm going to go ahead and close this, because my use case is clearly very niche and there are other ways to handle it.

[sisve]

@derrekbertrand Your example, calling a controller twice and getting 200 and 404 as a response ... is the way it currently works. I have to make up some odd code to do what you describe, because that is not how it works.

I asked for code to reproduce your issue. Can you share it with us?

[derrekbertrand]

@sisve If you make two HTTP calls to the same route in the real world, and you store some internal state in your controller (or other classes), it will not be there on the second call. PHP destroys all data between calls. HTTP applications are stateless by default, as you're probably aware.

In my example, I store the method and timestamp (which is supposed to be unique, I sleep(1)). When I make a second call during the same test, we do not get a fresh result. We get the old timestamp; the data is not cleared like in a real HTTP request cycle.

---

This is not a life or death thing, and has several solutions. I could structure my tests differently, or flush the app like crynobone suggested. I created this issue because I didn't know what the intended behavior was.

[therealchiko]

There are routes one might want to test for different users within a very specific, contained test. Without the ability to set a private variables, essentially doing away with using the constructor func, I don't think this helps. Seems each function has to define its own conditions privately, which defeats the purpose.

[itsmill3rtime]

To anyone having this issue still (i ran into it with Laravel 5.8), just change processIsolation="true" in your phpunit.xml

```
backupStaticAttributes="false"
bootstrap="vendor/autoload.php"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="true" <---------------- this property/value
stopOnFailure="false">

````

[wrabit]

@itsmill3rtime didn't work for me. It seems this isolates the test for each 'test', so it won't effect multiple requests inside one test method.