Framework: phpunit: Unable to find JSON fragment

[Fuzzyma]

When I test my api I get the following error:

Unable to find JSON fragment ["company":{"name":"Johnston Group","token":"JVW","updated_at":"2015-11-24 16:47:04","created_at":"2015-11-24 16:47:04","id":1}] within [{"company":{"created_at":"2015-11-24 16:47:04","id":"1","name":"Johnston Group","token":"JVW","updated_at":"2015-11-24 16:47:04"},"company_id":"1","created_at":"2015-11-24 16:47:04","firstname":"","id":1,"lastname":"","pin":"1234","token":"JVWBkC","updated_at":"2015-11-24 16:47:04"}].

Here again the both json outputs aligned:

 ["company":{"name":"Johnston Group","token":"JVW","updated_at":"2015-11-24 16:47:04","created_at":"2015-11-24 16:47:04","id":1}]
[{"company":{"created_at":"2015-11-24 16:47:04","id":"1","name":"Johnston Group","token":"JVW","updated_at":"2015-11-24 16:47:04"},"company_id":"1","created_at":"2015-11-24 16:47:04","firstname":"","id":1,"lastname":"","pin":"1234","token":"JVWBkC","updated_at":"2015-11-24 16:47:04"}]

As you can see the fragment is indeed present (wrong order, though). So why this error appears?

My seeJSON command tests on this:

            ->seeJson([
               'company' => $company,
               'firstname' => '',
               'lastname' => '',
               'pin' => 1234,
               'role' => 1
            ]);

[jairovsky]

seeJson make strict comparisons, so 'pin' => 1234 (interpreted as integer) will fail because the JSON returned by your API is actually "pin":"1234" (interpreted as string).

Also, the JSON returned by your API lacks a role attribute.

[Fuzzyma]

I agree that i missed the role. However the error remains when I comment out pin and role.
Furtermore the error is pointing to the company-object which is obviously not found and not the pin or role.

So there actually is an issue here (or is there some other magic involved?)

Also there are some mistakes in the json representation as well:

 ["company":{"name":"Johnston Group","token":"JVW","updated_at":"2015-11-24 16:47:04","created_at":"2015-11-24 16:47:04","id":1}]
 ^--array literal but the content is what you would see in an object literal. Its no valid json

[Fuzzyma]

I dont know why this was closed prematurely since this is a real problem

[wojtekk]

I have the same issue like @jairovsky.

@GrahamCampbell Do you need more details? I don't understand why you closed this issue :(

[diegosurita]

I'm having the same problem as described by @Fuzzyma

[akoskm]

@jairovsky @GrahamCampbell @Fuzzyma this problem still exists. The assertion fails with

["data":{"attributes":{"username":"hello123"},"id":1,"type":"users"}]

within

[{"data":{"attributes":{"created_at":"2016-05-22 13:10:19","created_by":"sql",
location_id":1,"username":"hello123"},"id":1,"type":"users"}}].

my assertion looks like:

             ->assertJsonFragment([
                 'data' => [
                    'id' => 1,
                    'type' => 'users',
                    'attributes' => [
                        'username' => 'hello123'
                    ]
                 ]
             ]);

I see that the difference in [{"data" vs ["data", but I'm not sure how to format the parameter of assertJsonFragment so that it later becomes ["data".

EDIT
If I try to change the parameter to (object)['data' => ... I'm receiving the following exception:

ErrorException: Argument 1 passed to Illuminate\Foundation\Testing\TestResponse::assertJsonFragment() must be of the type
array, object given, called in /Users/akoskm/Projects/hello/tests/Feature/LoginTest.php on line 38 and defined

[ZiperRom1]

Same problem here.

Tests\API\FlowTest::testCreation
Unable to find JSON fragment ["response":{"description":"Aut adipisci aut quidem iste. Quaerat est placeat iste officiis molestiae. Molestiae in eum aut et. Dolorem nam qui eos voluptates architecto ex.","label":"esse optio quo","user_id":2}] within [{"message":"Success","response":{"created_at":"2017-08-25 19:10:01","description":"Aut adipisci aut quidem iste. Quaerat est placeat iste officiis molestiae. Molestiae in eum aut et. Dolorem nam qui eos voluptates architecto ex.","id":3,"label":"esse optio quo","updated_at":"2017-08-25 19:10:01","user_id":2},"success":true}].

<?php

namespace Tests\API;

use Laravel\Lumen\Testing\DatabaseTransactions;
use App\Models\User;
use Tests\TestCase;

class FlowTest extends TestCase
{
    use DatabaseTransactions;

    public function testCreation()
    {
        /**
         * @var User $user
         */
        $user        = factory('App\Models\User')->create();
        $label       = $this->faker->words(3, true);
        $description = $this->faker->text;

        $this->json(
            'POST',
            '/flow',
            [
                'userId'          => $user->id,
                'userAccessToken' => $user->access_token,
                'flow'            => [
                    'label'       => $label,
                    'description' => $description
                ]
            ]
        )->seeJson([
            'success'  => true,
            'message'  => 'Success',
            'response' => [
                'user_id'     => $user->id,
                'label'       => $label,
                'description' => $description
            ]
        ]);
    }
}

Because even with the good matching data, it seems that if more JSON fields are returned into a non root level JSON field, the method fails while it should not.

[ZiperRom1]

I "fixed" the issue by editing Lumen / Laravel MakesHttpRequests.php class.
I just added 4 lines of code in SeeJsonContains method.

Lines added

    if (is_array($value)) {
        $this->seeJsonContains($value, $negate);
        continue;
    }

New SeeJsonContains methods

/**
     * Assert that the response contains the given JSON.
     *
     * @param  array  $data
     * @param  bool  $negate
     * @return $this
     */
    protected function seeJsonContains(array $data, $negate = false)
    {
        $method = $negate ? 'assertFalse' : 'assertTrue';

        $actual = json_decode($this->response->getContent(), true);

        if (is_null($actual) || $actual === false) {
            return PHPUnit::fail('Invalid JSON was returned from the route. Perhaps an exception was thrown?');
        }

        $actual = json_encode(array_sort_recursive(
            (array) $actual
        ));

        foreach (array_sort_recursive($data) as $key => $value) {
            $expected = $this->formatToExpectedJson($key, $value);

            if (is_array($value)) {
                $this->seeJsonContains($value, $negate);
                continue;
            }

            PHPUnit::{$method}(
                Str::contains($actual, $expected),
                ($negate ? 'Found unexpected' : 'Unable to find')." JSON fragment [{$expected}] within [{$actual}]."
            );
        }

        return $this;
    }

phpunit result before changes

PHPUnit 6.2.4 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 95 ms, Memory: 16.00MB

There was 1 failure:

1) Tests\API\FlowTest::testCreation
Unable to find JSON fragment ["response":[]] within [{"message":"Success","response":{"created_at":"2017-08-25 19:26:28","description":"Reprehenderit voluptates veritatis et rerum nulla at. Nesciunt adipisci earum explicabo odit aut. Dolores nam est eaque eos. Ducimus rerum atque voluptas quo eius dolores.","id":7,"label":"minima expedita aut","updated_at":"2017-08-25 19:26:28","user_id":2},"success":true}].
Failed asserting that false is true.

/home/rom1/bbbot-api/vendor/laravel/lumen-framework/src/Testing/Concerns/MakesHttpRequests.php:288
/home/rom1/bbbot-api/vendor/laravel/lumen-framework/src/Testing/Concerns/MakesHttpRequests.php:213
/home/rom1/bbbot-api/tests/API/FlowTest.php:34

FAILURES!
Tests: 1, Assertions: 2, Failures: 1.

phpunit result after changes

PHPUnit 6.2.4 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 85 ms, Memory: 16.00MB

OK (1 test, 5 assertions)

I don't know if I should make a merge request as it is a bit "hacky"...

Actually it will look for the key / value in all the JSON at any depth so if you have a JSON with same key at different nested-level, the test may pass while the key / value cheked was not the correct one.

[johnlui]

@ZiperRom1 Thank you for your code!

I think it's better to do it in TestCase.php.

[geoffreyvanwyk]

A workaround for the bug with $response->assertJson($array) is $response->assertSee(json_encode($array)).

[tomfordweb]

I was also able to do this.

$response2->assertStatus(200)->assertJson([[
   'id' => $file->id
 ]]);

Seems to work considering if i purposely fail the test I get:

Failed asserting that an array has the subset Array &0 (
    0 => Array &1 (
        'id' => 4
    )
).

Since it is comparing json nesting arrays seems to be enough for an API route that returns a collection.

[Pablo1107]

A workaround for the bug with $response->assertJson($array) is $response->assertSee(json_encode($array)).

I was trying to assert that a JSON response contains an user in a 'user' field. I did this and now works. :)

// Given an auth user
$user = factory('App\User')->create();
$this->actingAs($user);

// When view hit /login status
$this->get('/loginStatus')
    ->assertStatus(200)
    ->assertJsonFragment([
        'isLogged' => true,
    ])  
    ->assertSee(json_encode($user->toArray()));
// Then server returns user is logged and user name

[microdesign]

I just come to this problem,
is there still no working example to seeJsonStructure alternative in Laravel 5.8 ?

[mdiazr2000]

Hi I have this json structure
'data' : [
"salesOrder": [
4690,
1597,
],
"repairOrder": [
0,
50,
]
]
If I need to find an specific value inside of subarrays I do this

$this->get('url...')
->assertJsonStructure([
'data' => [
'salesOrder','repairOrder'
]
])
->assertJsonFragment([
'0' => 1597])

and it will make ok assertions , if i not use any key text , if i put 0 in key value the string to compare in assertions only contain the value that is necesary to compare.
The string to compare wil be 1597] , 1597} 1597, and PHP unit could find that string in our Json

[mdiazr2000]

I just come to this problem,
is there still no working example to seeJsonStructure alternative in Laravel 5.8 ?

Hello @microdesign
The method alternative to seeJsonStructure in laravel 5.8 is assertJsonStructure

Example
In the next json file
{
'data' : [
total : {},
salesOrder : {}
]
}

->assertJsonStructure([
'data' => [
'total', 'salesOrder'
],
])

[kgkg]

+1
This is still issue, comparing array containing other array is impossible right now without some workarounds (assertSee is not working right now).
Is it going to fixed anytime in future?

[Fuzzyma]

Since this issue is already 4 years old, it seems like nobody cares about that even though it feels like this could be fixed fairly easily. @jairovsky @GrahamCampbell any news?

[kgkg]

@Fuzzyma: maybe try to create new issue, I think they may ignore it since it's closed.
This is a serious issue, because right now there is no convenient method to check API response if it's not just flat array.

[Fuzzyma]

@kgkg would be cool if you could handle that new issue. I didnt use Laravel in a long time. I just keep geeting updates about this issue and its actually quite amusing and sad at the same time :D

[ZiperRom1]

Same, I don't use Lumen anymore but it is quite funny that such an important point that testing API response (main purpose of this framework) is still not resolved several years after the bug came out.

[geoffreyvanwyk]

I wonder how many of these types of closed-but-unresolved issues exist.

[microdesign]

The problem is solvable and that is why maybe there is not a fix.

I just rewrite my tests to check the response in the new way:

->assertJsonStructure(['token']);

when the response is just

{"token": "XXXXXX"}

[kgkg]

@microdesign well, maybe it's solvable for trivial api responses like the one you presented.
But for anything more like real world example when you actually return more data it's quite the contrary.

@systemovich I don't know, but assertion for checking API answers (not just the structure) more complicated than flat array is like basic functionality for me. It makes me think that most of Lavavel/Lumen developers doesn't write tests at all or they just gave up on this buggy-thing and started to use some external libraries.

[nicolas-costa]

2020 and this is still a thing.