Core: Issue / Question about serializer @MaxDepth annotation

[gagaXD]

Hello everyone, I'm experimenting some strange behaviour with Apiplatform and the @MaxDepth annotation.

I have an FormField Entity :

namespace App\Entity\Library\Form;

use App\Entity\Library\FormComponent;
use Doctrine\ORM\Mapping as ORM;
use Doctrine\Common\Collections\Collection;
use App\Entity\Traits\TimestampableEntity;
use Symfony\Component\Validator\Constraints as Assert;
use ApiPlatform\Core\Annotation\ApiResource;

use Symfony\Component\Serializer\Annotation\MaxDepth;
use Symfony\Component\Serializer\Annotation\Groups;

/**
 * @ORM\Entity
 * @ApiResource(
 *      attributes={
 *          "denormalization_context"={"groups"={"in"}},
 *          "normalization_context"={"groups"={"out"}, "enable_max_depth"=true},
 *          "force_eager"=false
 *     }
 * )
 */
class FormField
{
    use TimestampableEntity;

    const FORMFIELD_TYPE_TEXT = 'text';
    const FORMFIELD_TYPE_EMAIL = 'email';
    const FORMFIELD_TYPE_TEL = 'tel';
    const FORMFIELD_TYPE_CHECKBOX = 'checkbox';
    const FORMFIELD_TYPE_RADIO = 'radio';
    const FORMFIELD_TYPE_SELECT = 'select';
    const FORMFIELD_TYPE_DATETIME = 'datetime';
    const FORMFIELD_TYPE_RATING = 'rating';

    /**
     * @var int
     * @ORM\Id
     * @ORM\GeneratedValue
     * @ORM\Column(type="integer")
     * @Groups({"out"})
     */
    private $id;

    /**
     * @var FormComponent
     *
     * @Assert\NotNull()
     * @ORM\ManyToOne(targetEntity="App\Entity\Library\FormComponent", inversedBy="fields")
     * @ORM\JoinColumn(onDelete="CASCADE")
     * @Groups({"in", "out"})
     * @MaxDepth(1)
    */
    private $form;

   // ... with getters and setters

The $form attribute is a relation to the FormComponent attribute :

namespace App\Entity\Library;

use ApiPlatform\Core\Annotation\ApiSubresource;
use Doctrine\ORM\Mapping as ORM;
use Doctrine\Common\Collections\Collection;
use App\Entity\Library\Component;
use App\Entity\Library\Form\FormData;
use App\Entity\Library\Form\FormField;
use ApiPlatform\Core\Annotation\ApiResource;
use Symfony\Component\Validator\Constraints as Assert;

use Symfony\Component\Serializer\Annotation\MaxDepth;
use Symfony\Component\Serializer\Annotation\Groups;

/**
 * @ORM\Entity
 * @ApiResource(
 *      attributes={
 *          "filters"={"admin.search_filter", "component.search_filter"},
 *          "denormalization_context"={"groups"={"in"}},
 *          "normalization_context"={"groups"={"out"}, "enable_max_depth"="true"},
 *          "force_eager"=false
 *     }
 * )
 */
class FormComponent extends Component
{

    /**
     * @var string
     *
     * @ORM\Column(type="text", nullable=true)
     * @Groups({"in","out"})
     */
    private $intro;

    /**
     * @var string
     *
     * @ORM\Column(type="text", nullable=true)
     * @Groups({"in","out"})
     */
    private $outro;

    /**
     * @var Collection
     *
     * @ORM\OneToMany(targetEntity="App\Entity\Library\Form\FormData", mappedBy="form", cascade={"persist", "remove"}, orphanRemoval=true)
     * @ORM\OrderBy({"createdAt" = "ASC"})
     * @Groups({"in", "out"})
     * @ApiSubresource()
     * @MaxDepth(1)
    */
    private $results;

    /**
     * @var Collection
     *
     * @ORM\OneToMany(targetEntity="App\Entity\Library\Form\FormField", mappedBy="form", cascade={"all"}, orphanRemoval=true)
     * @ORM\OrderBy({"position" = "ASC"})
     * @Groups({"in", "out"})
     * @MaxDepth(1)
    */
    private $fields;

   // ... with getters and setters

Expected output

When querying the form_fields endpoint (GET /form_fields/{id} ), expect a jsonld with only one level of depth :

{
    "@context": "/api/contexts/FormField",
    "@id": "/api/form_fields/24",
    "@type": "FormField",
    "id": 24,
    "form": {
        "@id": "/api/form_components/190",
        "@type": "FormComponent",
        "intro": "Unde iste inventore similique nemo.",
        "outro": "Itaque saepe est aliquam asperiores repudiandae.",
        "id": 190,
        "enabled": true,
        "name": "Mon form magic",
        "description": "Description de mon form",
        "createdAt": "2018-03-09T16:02:08+00:00",
        "updatedAt": "2018-03-09T16:02:08+00:00"
    },
    "position": 1,
    "type": "text",
    "label": "name",
    "description": "Give me your name",
    "choices": null,
    "dateFormat": null,
    "maxRange": null,
    "required": true,
    "maxLength": null,
    "createdAt": "2018-03-09T16:02:08+00:00",
    "updatedAt": "2018-03-09T16:02:08+00:00"
}

Current behavior

When I query the form_fields endpoint (GET /form_fields/{id} ), I get this result :

{
    "@context": "/api/contexts/FormField",
    "@id": "/api/form_fields/24",
    "@type": "FormField",
    "id": 24,
    "form": {
        "@id": "/api/form_components/190",
        "@type": "FormComponent",
        "intro": "Unde iste inventore similique nemo.",
        "outro": "Itaque saepe est aliquam asperiores repudiandae.",
        "results": [
            {
                "@id": "/api/form_datas/4",
                "@type": "FormData",
                "id": 4,
                "form": "/api/form_components/190",
                "device": {
                    "@id": "/api/devices/8",
                    "@type": "Device",
                    "type": "DEVICE_MOBILE",
                    "uuid": "XXXX-XX01",
                    "metaData": [],
                    "createdAt": "2018-03-09T16:01:58+00:00",
                    "updatedAt": "2018-03-09T16:01:58+00:00"
                },
                "application": {
                    "@id": "/api/applications/81",
                    "@type": "Application",
                    "id": 81,
                    "name": "ma première application",
                    "type": "APP_MOBILE",
                    "description": "Ceci est ma première application mobile",
                    "enabled": true,
                    "createdAt": "2018-03-09T16:02:08+00:00",
                    "updatedAt": "2018-03-09T16:02:08+00:00"
                },
                "datas": [
                    {
                        "@id": "/api/form_field_values/67",
                        "@type": "FormFieldValue",
                        "field": "/api/form_fields/24",
                        "value": "Toto",
                        "createdAt": "2018-03-09T16:02:08+00:00",
                        "updatedAt": "2018-03-09T16:02:08+00:00"
                    },
                    {
                        "@id": "/api/form_field_values/68",
                        "@type": "FormFieldValue",
                        "field": {
                            "@id": "/api/form_fields/25",
                            "@type": "FormField",
                            "id": 25,
                            "position": 2,
                            "type": "email",
                            "label": "email",
                            "description": "Give me your email",
                            "choices": null,
                            "dateFormat": null,
                            "maxRange": null,
                            "required": true,
                            "maxLength": null,
                            "createdAt": "2018-03-09T16:02:08+00:00",
                            "updatedAt": "2018-03-09T16:02:08+00:00"
                        },
                        "value": "[email protected]",
                        "createdAt": "2018-03-09T16:02:08+00:00",
                        "updatedAt": "2018-03-09T16:02:08+00:00"
                    },
                    {
                        "@id": "/api/form_field_values/69",
                        "@type": "FormFieldValue",
                        "field": {
                            "@id": "/api/form_fields/26",
                            "@type": "FormField",
                            "id": 26,
                            "position": 3,
                            "type": "rating",
                            "label": "rank",
                            "description": "Give me your rank",
                            "choices": null,
                            "dateFormat": null,
                            "maxRange": 5,
                            "required": true,
                            "maxLength": null,
                            "createdAt": "2018-03-09T16:02:08+00:00",
                            "updatedAt": "2018-03-09T16:02:08+00:00"
                        },
                        "value": "5",
                        "createdAt": "2018-03-09T16:02:08+00:00",
                        "updatedAt": "2018-03-09T16:02:08+00:00"
                    }
                ],
                "createdAt": "2018-03-09T16:02:08+00:00",
                "updatedAt": "2018-03-09T16:02:08+00:00"
            },
            {
                "@id": "/api/form_datas/5",
                "@type": "FormData",
                "id": 5,
                "form": "/api/form_components/190",
                "device": {
                    "@id": "/api/devices/8",
                    "@type": "Device",
                    "type": "DEVICE_MOBILE",
                    "uuid": "XXXX-XX01",
                    "metaData": [],
                    "createdAt": "2018-03-09T16:01:58+00:00",
                    "updatedAt": "2018-03-09T16:01:58+00:00"
                },
                "application": {
                    "@id": "/api/applications/81",
                    "@type": "Application",
                    "id": 81,
                    "name": "ma première application",
                    "type": "APP_MOBILE",
                    "description": "Ceci est ma première application mobile",
                    "enabled": true,
                    "createdAt": "2018-03-09T16:02:08+00:00",
                    "updatedAt": "2018-03-09T16:02:08+00:00"
                },
                "datas": [
                    {
                        "@id": "/api/form_field_values/70",
                        "@type": "FormFieldValue",
                        "field": "/api/form_fields/24",
                        "value": "Dede",
                        "createdAt": "2018-03-09T16:02:08+00:00",
                        "updatedAt": "2018-03-09T16:02:08+00:00"
                    },
                    {
                        "@id": "/api/form_field_values/71",
                        "@type": "FormFieldValue",
                        "field": {
                            "@id": "/api/form_fields/25",
                            "@type": "FormField",
                            "id": 25,
                            "position": 2,
                            "type": "email",
                            "label": "email",
                            "description": "Give me your email",
                            "choices": null,
                            "dateFormat": null,
                            "maxRange": null,
                            "required": true,
                            "maxLength": null,
                            "createdAt": "2018-03-09T16:02:08+00:00",
                            "updatedAt": "2018-03-09T16:02:08+00:00"
                        },
                        "value": "[email protected]",
                        "createdAt": "2018-03-09T16:02:08+00:00",
                        "updatedAt": "2018-03-09T16:02:08+00:00"
                    },
                    {
                        "@id": "/api/form_field_values/72",
                        "@type": "FormFieldValue",
                        "field": {
                            "@id": "/api/form_fields/26",
                            "@type": "FormField",
                            "id": 26,
                            "position": 3,
                            "type": "rating",
                            "label": "rank",
                            "description": "Give me your rank",
                            "choices": null,
                            "dateFormat": null,
                            "maxRange": 5,
                            "required": true,
                            "maxLength": null,
                            "createdAt": "2018-03-09T16:02:08+00:00",
                            "updatedAt": "2018-03-09T16:02:08+00:00"
                        },
                        "value": "2",
                        "createdAt": "2018-03-09T16:02:08+00:00",
                        "updatedAt": "2018-03-09T16:02:08+00:00"
                    }
                ],
                "createdAt": "2018-03-09T16:02:08+00:00",
                "updatedAt": "2018-03-09T16:02:08+00:00"
            }
        ],
        "fields": [
            "/api/form_fields/24",
            {
                "@id": "/api/form_fields/25",
                "@type": "FormField",
                "id": 25,
                "position": 2,
                "type": "email",
                "label": "email",
                "description": "Give me your email",
                "choices": null,
                "dateFormat": null,
                "maxRange": null,
                "required": true,
                "maxLength": null,
                "createdAt": "2018-03-09T16:02:08+00:00",
                "updatedAt": "2018-03-09T16:02:08+00:00"
            },
            {
                "@id": "/api/form_fields/26",
                "@type": "FormField",
                "id": 26,
                "position": 3,
                "type": "rating",
                "label": "rank",
                "description": "Give me your rank",
                "choices": null,
                "dateFormat": null,
                "maxRange": 5,
                "required": true,
                "maxLength": null,
                "createdAt": "2018-03-09T16:02:08+00:00",
                "updatedAt": "2018-03-09T16:02:08+00:00"
            }
        ],
        "id": 190,
        "enabled": true,
        "name": "Mon form magic",
        "description": "Description de mon form",
        "createdAt": "2018-03-09T16:02:08+00:00",
        "updatedAt": "2018-03-09T16:02:08+00:00"
    },
    "position": 1,
    "type": "text",
    "label": "name",
    "description": "Give me your name",
    "choices": null,
    "dateFormat": null,
    "maxRange": null,
    "required": true,
    "maxLength": null,
    "createdAt": "2018-03-09T16:02:08+00:00",
    "updatedAt": "2018-03-09T16:02:08+00:00"
}

As you can see, all my relations are serialized, even if I specified a maxDepth

Investigation

I looked into the Symfony\Component\Serializer\Normalizer\AbstractObjectNormalizer::normalize method, and understand that when calling the isMaxDepthReached, I try to dump the $key :

The keys are then store in the $context variable correctly.

=> If I use a Fetch_eager = false (lazy loading), You can see that doctrine use Proxy ClassName and these classNames are also used in the keys stored in the $context variable.

I also tried to dump/die here : (https://github.com/symfony/serializer/blob/8ab187e76ffec8f800aabdf051a2f414c7c66b5c/Normalizer/AbstractObjectNormalizer.php#L398) But it's never triggered.

I've try to dig further into the code and the issues, and, I came accross this issue #933 .

In the Symfony/Component/Serializer/Normalizer/AbstractObjectNormalizer::normalize method, I try to change the $class = get_class($object) by $class = \Doctrine\Common\Util\ClassUtils::getClass($object);. This correct the Proxy keys problem in the $context variable, but the issue about depth of the serialization remains the same.

After checking the Symfony/Component/Serializer/Normalizer/AbstractObjectNormalizer::isMaxDepthReached method's code, I have a question : Is the maxDepth annotation really doing what I am expecting, limiting the depth of the serialized entity, or, is it used only to limit the depth of "circular references" of the same entity in the serialization loop ?

Also, as you can see on the entities, I'm using the "out" group in all of my entities, is it correct, or should I use a different group by entity ( something like EntityName_out), to describe exactly what I want to be serialize in each relation?

Thanks in advance for your help !

[mantas-gudelevicius-hiccup]

+1 mee to having same issue.

[soyuka]

I think that you can't achieve what you want only with max depth, and your assumption of the "circular references of the same entity" looks correct.

To achieve the first result IMO you shouldn't serialize the "results" node. Could you use subresources and call multiple requests to achieve a clean and fast result?

[gagaXD]

Yes, If I use subresources + multiple API calls, I can get the expected answer. That's the workaround that I've implemented for my solution.

But, I remember in the old days, when I was using JMSserializer, the @MaxDepth annotation, I could achieve the expecting result.

Maybe I am misunderstanding the maxDepth annotation of the symfony serializer ?
And do you thinkon any way to achieve this on server side ?

[soyuka]

And do you thinkon any way to achieve this on server side ?

Build your own normalizer for that particular case maybe? You could compose one with our ItemNormalizer and add some additional conditions on when to get (or not) the relations.

Maybe I am misunderstanding the maxDepth annotation of the symfony serializer ?

I'm no expert but I think that it works per Type (ie object class here) and it only reads the @MaxDepth from the first property (@MaxDepth on the relation gets ignored).

[Nightbr]

Hey,

I think it could be a nice feature to the Symfony Serializer to work like JMS @MaxDepth annotation:

You can limit the depth of what will be serialized in a property with the @MaxDepth annotation.

What do you think on this? Should we open an issue on the Symfony Serializer repository?

[soyuka]

What do you think on this? Should we open an issue on the Symfony Serializer repository?

I'd need to investigate a bit more because I'm really not sure that this is an issue.

[Nightbr]

Ok no problem!

Note: If we look at the Symfony Serializer documentation, it seems it should work like we said:

The Serializer component is able to detect and limit the serialization depth. It is especially useful when serializing large trees.

But in fact, its only on the same Object it works like this. But this information is not explicitly said in the documentation. That's why I think perhaps it's an issue.

I think it's @dunglas who works on the Symfony Serializer (amazing work btw).

Thanks for your time btw!

[Nightbr]

Hey, it's been a long time but it seems like this is a solved issue. I retested on our API project and if you have "normalization_context"={"groups"={"out"}, "enable_max_depth"=true}, properly set and @MaxDepth(1) it will properly apply it. We can finally get ride of every force_eager=false and resolved max join bug properly :tada:

But we need to put "enable_max_depth"=true in every normalization_context or denormalization_context... It could be pretty nice to have a global config enable_max_depth=true to enable it in api_platform config. Whet do you think about this?

[soyuka]

Shouldn't this be available in symfony instead? I'll try to propose

serializer:
    context:
        enable_max_depth: true

[soyuka]

oh actually check out https://github.com/symfony/symfony/pull/28709

[leberknecht]

@Nightbr what version are you using? I'm still affected by this, using api-platform/core v2.3.6

[Nightbr]

we use the latest v2.3.6. Make sure you put "enable_max_depth"=true in every normalization_context or denormalization_context.

Other workaround which we use is to create special groups and use them in normalization_context or denormalization_context.

[leberknecht]

Hm, did that, but has no impact for me :/ Also i can confirm the proxy-class-key problem. As the isMaxDepthReached indeed only keeps track how often a specific class was serialized, i agree with @gagaXD that we might have misinterpreted what it is good for...or that the meaning is different from JMS maxdepth (where it says "dont resolve relations further than n steps in the graph"). With symfony/serializer v4.2.2 the objectClassResolver can set in the constructor, but AbstractItemNormalizer doesnt fill this parameter yet :/

[Nightbr]

Ah, yes, It works only for the same Entity... But this is a limitation of the Symfony Serializer I suppose.

See my old message https://github.com/api-platform/core/issues/1764#issuecomment-407710085

[leberknecht]

Btw: i fixed the proxy class key problem for me by using doctrine class resolve as proposed above without touching the vendor:
Overwrite api_platform.jsonld.normalizer.item service (if you are using json-ld of course, otherwise check what service you need to hook for your case). For ease of use:

    api_platform.jsonld.normalizer.item:
        class: App\Hook\ItemNormalizerService
        arguments:
            - "@api_platform.metadata.resource.metadata_factory"
            - "@api_platform.metadata.property.name_collection_factory"
            - "@api_platform.metadata.property.metadata_factory"
            - "@api_platform.iri_converter"
            - "@api_platform.resource_class_resolver"
            - "@api_platform.jsonld.context_builder"
            - "@api_platform.property_accessor"
            - "@serializer.mapping.class_metadata_factory"

For sf 4 users the bind might be required:

        bind:
            $resourceMetadataFactory: "@api_platform.metadata.resource.metadata_factory"
            $propertyNameCollectionFactory: "@api_platform.metadata.property.name_collection_factory"
            $propertyMetadataFactory: "@api_platform.metadata.property.metadata_factory"
            $iriConverter: "@api_platform.iri_converter"
            $resourceClassResolver: "@api_platform.resource_class_resolver"
            $contextBuilder: "@api_platform.jsonld.context_builder"
            $classMetadataFactory: "@serializer.mapping.class_metadata_factory"

Just copy from the vendor, leave as is, just change the namespace. Do the same with AbstractItemNormalizer, ensure that your ItemNormalizerService is dervied from App\HooksAbstractItemNormalizer

In AbstractItemNormalizer constructor, change:

parent::__construct($classMetadataFactory, $nameConverter, null, null, null, $defaultContext);

to

parent::__construct(
            $classMetadataFactory,
            $nameConverter,
            null,
            null,
            function ($object) {
                return \Doctrine\Common\Util\ClassUtils::getClass($object);
            },
            $defaultContext);

[soyuka]

Closing this, feel free to ping me if you still have any questions.