Framework: [5.4] TestResponse does not display a thrown exception

[jerguslejko]

• Laravel Version: 5.4.9
• PHP Version: 7.1.1
• Database Driver & Version: N/A

Description:

When testing an endpoint that returns an exception, Laravel does not show this exception and phpunit returns green for the test.

I believe that this is caused by Illuminate\Foundation\Testing\TestResponse::fromBaseResponse() which stores the exception on a TestResponse object but, by default, does not display it in the console.

I am not entirely sure if this is a bug. Although, when I test an endpoint that throws an exception, I'd like to know about it.

Steps To Reproduce:

1. $ laravel new app

2. Go to tests/Unit/ExampleTest.php and change testBasicTest() to:

   public function testBasicTest()
   {
       $this->get('/');
   }
   

3. Then, in routes/web.php change default route to following (notice misspelled dd() function):

   Route::get('/', function () {
       d(11);
   });
   

4. run $ phpunit - you should see the tests passing. There is no indication that the function is misspelled.

Thanks.

[themsaid]

Hello @JacobBennett :) I remember you had a solution for this, no?

[jerguslejko]

@themsaid I guess this https://github.com/laravel/framework/blob/5.4/src/Illuminate/Foundation/Testing/TestResponse.php#L36 could be changed to something like?

if (isset($response->exception)) {
    try {
        throw $response->exception;
    } catch (SomethingExpected $e) {
        $testResponse->exception = $response->exception;
    }
}

[JacobBennett]

@jerguslejko when calling $this->get('/') you aren't making any specific assertions against the response which is why you aren't getting any failing tests. If you instead did

$this->get('/')->assertResponseOk();

Then I believe your test will get an exception since the response will return a 500 instead of a 200.

@themsaid I think what you were referring to was this tweet which will re-throw your exceptions instead of having the framework handle them and return a valid response. This was warned against by Adam Wathan however so use with caution.

[jerguslejko]

@JacobBennett calling d(11); causes Symfony\Component\Debug\Exception\FatalThrowableError. I think this kind of exception should be thrown no matter what.

If I do $this->get('/')->assertStatus(200); the only thing I see is:

1) Tests\Feature\ExampleTest::testBasicTest
Expected status code 200 but received 500.
Failed asserting that false is true.

[JacobBennett]

It's really a philosophical thing I suppose. When you are calling get('/') I think the expectation is that you are doing this as an "acceptance" level test. You are making a mock request to the framework which is then passing it to the router > controller > view > etc... or in your case, just to the router. In the case that a user was visiting your site, would you rather them get a 500 error or would you rather them get a stack trace and an exception, or a white page? That is the level you are testing this at. The framework is handling this as if you are a client making a request to the server, and is responding as such. In this case, a 500 is exactly what you would be looking for. This is expected behavior.

Hopefully that helps. In the case that you run into such behavior and you need to figure out the ACTUAL exception that is being thrown, I would suggest following the tweet screen shot included above.

[jerguslejko]

Well, as much as I agree with you I still think that it would be much more useful to see actual error "by default".

While your solution helps, it requires some additional work + there are situations, as Adam Wathan mentions in the tweet, where you want the exception to be handled by the framework.

The "philosophical thing" is whether a test should pass if the response status was 500.

Any opinions @themsaid @taylorotwell ? thanks

[JacobBennett]

You are correct, the test should not pass if there is a 500. This is why the test should assertResponseOk(). This will make sure that the test fails when there is a 500 thrown.

Maybe you should test the controller method instead of a response getting routed to the controller method?

The potential problem with the proposed solution is that it would break any acceptance level tests that are looking for responses with status codes that depend on exceptions being thrown. In example, if you are checking to be sure that your validation is working in your controller, you might assertStatus('422'). If you were catching the exceptions and throwing them again, you would have to assert the exception being thrown rather than the status code that it would return when that exception occurs.

[jerguslejko]

This is an actual test from my testsuite:

/** @test */
public function it_returns_all_tasks()
{
    factory(Task::class)->create();

    $this->get('/tasks')
        ->assertJsonStructure(['id', 'title', 'description']);
}

If there is a bug, it results in Failed asserting that a row in the table [tasks] matches the attributes.... Not descriptive at all.

To the matter of the proposed solution: Sorry, that's mistake on my part. In the example above I meant to only catch exceptions which do not extend ErrorException. If I am right, these exceptions represents things like parse errors, undefined functions or Undefined offset.

I cannot think of a situation where someone would want to catch these.

[jerguslejko]

I guess this is now obsolete as https://github.com/laravel/framework/blob/master/src/Illuminate/Foundation/Testing/Concerns/InteractsWithExceptionHandling.php exists.

Thanks anyway!

[morcegon]

@jerguslejko how do I enable the exceptions?

[jerguslejko]

@morcegon call $this->withoutExceptionHandling() at the beginning of your test method. Or in the setUp method to disable exception handling for all your tests.

[cjreysayago]

Hi @jerguslejko . Trying to add $this->withoutExceptionHandling() method at the beginning of my test, throws a call to undefined method on Lumen 5.6. Do i have any other option to activate this feature?

[vaishnavmhetre]

Hi @jerguslejko . Trying to add $this->withoutExceptionHandling() method at the beginning of my test, throws a call to undefined method on Lumen 5.6. Do i have any other option to activate this feature?

I wrote manual helper in abstract TestCase class

public function disableExceptionHandling(){
     app()->instance(\App\Exceptions\Handler::class, new class extends \App\Exceptions\Handler{
         public function __construct () { }
         public function report ( Exception $exception ) {
             parent ::report( $exception ); // TODO: Change the autogenerated stub
         }

         public function render($request, Exception $exception)
         {
             throw $exception;
         }
     });
}

[BWoodfork]

Hi @jerguslejko . Trying to add $this->withoutExceptionHandling() method at the beginning of my test, throws a call to undefined method on Lumen 5.6. Do i have any other option to activate this feature?

I'm having the same issue.

[jerguslejko]

@BWoodfork hi, sorry I don't use lumen so can't help you out :/