Core: v2 metadata - YAML resource config

[teohhanhui]

This is a discussion on PR #375

It seems like the current code only supports using the Resource annotation?

[dunglas]

Yes. YAML and XML support is planned. I prefer XML but the better is to support both. As always, all help is very welcome.

[3kynox]

:+1:

[soyuka]

How are you picturing this?

1. CompilerPass
2. Read tagged services api_platform.resource
3. Add classes to the ResourceCollectionMetadata
4. Generate ResourceItemMetadata (operations, filters, normalization contexts etc)
5. Should a resource be a service? By that I mean that we can add method calls on definitions. Or, do we simply parse configurations and handle them like we handle annotations?
6. normalization contexts are defined by operations now right? So are filters apparently?

Maybe we could start by setting up the full configuration schema?

Proposal:

    resource.product:
        class:  'Product\ProductBundle\Entity\Product'
        parent: '?'
        itemOperations:
            - methods: ['POST']
              denormalization: ['some', 'context', 'group']
            - methods: ['GET'] # Following is applied to every 'GET itemOperation' 
              dataprovider: 'instance of ItemDataProviderInterface'
            - methods: ['GET']
              route_name: 'some_route_name' # Following will be active on the current Operation identified by the route_name
              route: '/my/new/route' 
              controller: '@some.service'
              dataprovider: 'instance of ItemDataProviderInterface' 
        collectionOperations:
            - methods: ['GET', 'POST'] # Apply filters and data provider to every collectionOperation methods
              filters: ['@some.filter'] 
              dataprovider: 'instance of CollectionDataProviderInterface'
        tags: [ { name: 'api_platform.resource'} ]

Now a base resource would be defined as:

    resource.person:
        class: 'Party\PartyBundle\Entity\Person'

IMO it'd be nice that a configuration defined Operation just extends the base ones. For example, here the collectionOperations will still have the POST method, even if not declared. I don't know if there's any interest in adding a way of not having standard REST method, I don't see any. To override an Operation it'd require a configuration on the Operation identifier, here the route_name (maybe it's not the right identifier?).

From what I've seen, there's a lot of reusable code in Annotation parsing.

[teohhanhui]

@soyuka You're getting the wrong idea.

In this case the yml files are for configuration, like the Symfony Serializer and Validator configuration files. They are not Symfony services.

[soyuka]

I know yes but in v1 the configuration declares a kind of service (hence
the parent attribute).
Le mer. 24 févr. 2016 à 13:02, Teoh Han Hui [email protected] a
écrit :

@soyuka https://github.com/soyuka You're getting the wrong idea.

In this case the yml files are for configuration, like the Symfony
Serializer and Validator configuration files. They are not Symfony services.

—
Reply to this email directly or view it on GitHub
https://github.com/dunglas/DunglasApiBundle/issues/421#issuecomment-188222370
.

[dunglas]

@soyuka there are 2 distinct things:

• Adding a YAML (and XML) loader for the new metadata system. It doesn't need any configuration or service, it should be independent of Symfony (except maybe some component like the YAML one). You can take a look at how it's done for the Symfony Serializer as an example: https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Serializer/Mapping/Loader/YamlFileLoader.php
• Creating a compatibility layer with the v1. IMO it should be a special metadata loader using the Symfony DIC to retrieve resources and filters service definition from the v1 and convert them into instance of *Metadata classes of the v2

[soyuka]

Thanks @dunglas it's clearer.

Still, what about filters, normalization groups etc. which are defined by operations, no more by resources. Should they still fit the v1 configuration schema? Agreed that they should be a compatibility layer in any case.

[dunglas]

Everything was service in the v1, and now everything is value objects (*Metadata). It should be possible to have a BC layer for most of them but it's a hard work IMO.

[soyuka]

+1, that said the configuration in v2 can be better than the one in v1 (for example defining normalization groups by operation).

[dunglas]

Yes it was one of the goals of this large refactoring. The new system is a solid foundation, with to dependency to the Symfony framework (can be used with Laravel or Drupal as well).

[soyuka]

Indeed, we're on the same page. So, what would the new configuration schema look like (if we skip the BC layer)? From what I understood this is a configuration parsing, this means that no symfony services will be injected.

If I transform my previous try it'd be something like:

# \\MyBundle\Resources\config\resources.yml (or xml)
resources:
    product:
        shortName: 'someproductshortname'
        description: 'Product resource'
        class:  'Product\ProductBundle\Entity\Product'
        itemOperations:
            - methods: ['GET']
              filters: ['product.my_awesome_filter'] # this is the filter id 
            - methods: ['GET']
              route_name: 'some_route_name'
              route: '/my/new/route' # Controller can be set in the routing configuration for a given path with symfony 
        collectionOperations:
            - methods: ['GET', 'POST'] # Apply filters and data provider to every collectionOperation 
              denormalizationContext: {groups: ['some', 'context', 'groups']}
methods
        iri: 'someirischema'

Where filters have an id which does the link (instead of having a symfony service).

I haven't found (yet) where dataProvider (now split into CollectionDataProvider and ItemDataProvider) defines the resources that are supported. Anyway this is linked to Doctrine so I left it aside.

What about Annotations on top of Configuration? Should they merge?

[dunglas]

I would suggest:

resources:
    product:
        shortName: 'someproductshortname'
        description: 'Product resource'
        class:  'Product\ProductBundle\Entity\Product'
        itemOperations:
            my_op_name: {} # an arbitrary associative array that can contain anything
            my_other_op_name: {} # same
        collectionOperations:
            my_collection_op: {} # same again...
        iri: 'someirischema'
        attributes: {} # Another random associate array
        properties: # Here is the mapping for Metadata\Property\ItemMetadata
            price:
                description: 'The price of the product'
                #type: I would let this away for now, as the PropertyInfo loader already do the job
                readable: true
                writeable: true
                readableLink: true
                writeableLink: true
                required: true
                iri: 'http://example.com'
                identifier: false
                attributes: {} # An associate array, still without constraints
            # Mapping for other properties here

[soyuka]

Interesting, could you please explain what properties metadata is used for? I saw that the normalizer uses some of them to check if they are writeable/readable. If I'm not mistaken, this was handled in the v1 through the PropertyInfo component (within symfony), and still is in v2 on the Symfony bridge.

What's the readableLink/writeableLink use case?

Thanks!

[teohhanhui]

@dunglas I'm trying to figure out how to borrow the Loader pattern from the Symfony Serializer, and fit it into the chain of decorator factories. Do we just forgo the Loader pattern altogether and create another factory, e.g. YamlResourceMetadataFactory?

[teohhanhui]

And I'd definitely love to support this kind of functionality:

https://github.com/symfony/framework-bundle/blob/e65c60c9d8f8288a93ab3a6fdfa8165bbc2b2af0/DependencyInjection/FrameworkExtension.php#L941-L978

[soyuka]

+1 about using the symfony Finder to do automatic configuration parsing like this (maybe only in the Bridge?).
This should support XML too. Maybe that there will be common paths between XML and YML factories that would have to merge in some kind of loader?

[soyuka]

I was going through the json schema documentation and reading it remind me of the resource configuration proposal two posts above.

For example the json schema hypermedia overview:

{
    "title": "Written Article",
    "type": "object",
    "properties": {
        "id": {
            "title": "Article Identifier",
            "type": "number"
        },
        "title": {
            "title": "Article Title",
            "type": "string"
        },
        "authorId": {
            "type": "integer"
        },
        "imgData": {
            "title": "Article Illustration (small)",
            "type": "string",
            "media": {
                "binaryEncoding": "base64",
                "type": "image/png"
            }
        }
    },
    "required" : ["id", "title", "authorId"],
    "links": [
        {
            "rel": "full",
            "href": "{id}"
        },
        {
            "rel": "author",
            "href": "/user?id={authorId}"
        }
    ]
}

Just a thought, but declaring properties in a resource configuration feels too much. Also, json schema's are kind of declaring everything we need here, and are truly portable (you can do a lot based on json schema's).

[teohhanhui]

@soyuka I believe JSON Schema is out of scope and undesirable for this project. It's been built around JSON-LD and Hydra (though there's support for XML serialization). The domain model should be RDF (JSON-LD is a serialization format for RDF), though it doesn't seem to be enforced especially since there's no support provided here for building / maintenance of RDF schema. A good starting point would be schema.org

[soyuka]

@teohhanhui agreed that it's out of scope here. I just want to understand the use of

        properties: # Here is the mapping for Metadata\Property\ItemMetadata
            price:
                description: 'The price of the product'
                #type: I would let this away for now, as the PropertyInfo loader already do the job
                readable: true
                writeable: true
                readableLink: true
                writeableLink: true
                required: true
                iri: 'http://example.com'
                identifier: false
                attributes: {} # An associate array, still without constraints
            # Mapping for other properties here

in the configuration. I don't see the need, nor the use case.

Still, a json schema can represent a json-ld resource and be helpful for validating input data.

[teohhanhui]

Uhh yes, previously all those things were inferred by other mapping loaders (now metadata factories) - Doctrine ORM, Symfony Serializer, Symfony Validator, PHPDoc etc.

Has that changed? (Doesn't seem so to me...) Are they necessary to be configured manually now, or is it optional?

[dunglas]

One of the goal of the v2 is to be a fully independent library. The goal here is to be able to configure everything, just like you can do with annotations. But of course, if PropertyInfo and - for instance - Doctrine ORM are installed, values are guessed (exactly like in v1) if they are omitted.

IMO, YAML and XML should give the same flexibility than annotations (use the defined configuration value if explicitly defined, guess it if not defined).

We can add a dependency to the Symfony Finder Component (I want to avoid a dependency to FrameworkBundle but some standalone components are OK). However, the PHP built-in RegexIterator should be enough to discover some directories: https://github.com/dunglas/DunglasApiBundle/blob/master/src/Metadata/Resource/Factory/CollectionMetadataAnnotationFactory.php#L56-L63

I removed the chain loader from the v1 to use decorators. IMO it fits more to this case than a chain because you can define some config in YAML and some others using annotations.

[soyuka]

@dunglas What should be done to close this, I think I've read something about properties mapping?

[teohhanhui]

@soyuka My 2 cents: https://github.com/api-platform/core/pull/490#issuecomment-218667018

[dunglas]

@soyuka supporting property mapping (IIRC, only resources are supported for now)
@teohhanhui the current implementation is consistent with how Symfony components work (e.g. http://symfony.com/doc/current/book/validation.html#properties), I would keep it as is. However 👍 for renaming the file.

[soyuka]

@teohhanhui It could map *api_resource(s?).{xml|yml} to make defining one configuration file per resource possible. If we all agree it could be a nice feature it can easily be implemented.

Alright I'll try to find some time to add this! I assume I'm scaling on the symfony properties mapping for the xml/yaml structure?

[dunglas]

Looks good to me!

[teohhanhui]

@dunglas With Symfony Serializer and Validator we can use one file per class (under the src/AppBundle/Resources/config/serialization and src/AppBundle/Resources/config/validation respectively). The functionality is provided by the FrameworkBundle.

Link again: https://github.com/symfony/framework-bundle/blob/5ffc8af4619f8847ce35e1df3b6fd83e7eee7ac0/DependencyInjection/FrameworkExtension.php#L823-L832

[soyuka]

@teohhanhui So, the final structure would accept:

src/AppBundle/Resources/config/api_resources.{yml|xml}
# or
src/AppBundle/Resources/config/api_resources/*.{yml|xml}

[dunglas]

@teohhanhui I forgot about that, 👍 to be should be consistent with this convention!

[soyuka]

Ok I'm setting up this convention asap and then we'll see about adding properties (a bit more work to do on that part \o/).

[dunglas]

Thank you for working on that!

[mvrhov]

Can we have a file per resource please. Symfony not supporting this is pain.

[soyuka]

Yes we discussed it and it'll be done soon.
Le lun. 6 juin 2016 à 11:38, Miha Vrhovnik [email protected] a
écrit :

Can we have a file per resource please. Symfony not supporting this is
pain.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/api-platform/core/issues/421#issuecomment-223912311,
or mute the thread
https://github.com/notifications/unsubscribe/ABQr874E3QHjHFzUg6pBpm2rOqKlYSlAks5qI-qlgaJpZM4HfdGN
.

[Simperfit]

@soyuka @dunglas this can be closed ?

[teohhanhui]

@Simperfit There are still some practical issues if we try to actually use YAML config.

1. The YAML key should be used as fallback for class for a nicer syntax.
2. Mixed format config (merging) should be possible. As far as I can tell, it works for Symfony Validator.
3. @ApiProperty is not included yet.

[soyuka]

Mixed format config (merging) should be possible. As far as I can tell, it works for Symfony Validator.

Currently if a parent metadata exists it's not overridden (did coverage recently here). Can you detail what merging you'd see?

[teohhanhui]

One should be able to specify part of the configuration in annotations, and part of it in YAML, for the same resource. This is especially important for property metadata, which currently is not supported by the YAML / XML configuration yet.

[soyuka]

The behaviour you describe is working just fine atm.

[soyuka]

Closing in favor of https://github.com/api-platform/core/issues/739

[jorgeluisacostaalonso]

Hello, i'am new on apiplatform and i'm searching how define api_platform.resource in the yml files but i didn't find the doc about that!

For example: this definiton is not in the online doc, i would want to know about:
resource.product:
class: 'Product\ProductBundle\Entity\Product'
parent: '?'
itemOperations:
- methods: ['POST']
denormalization: ['some', 'context', 'group']
- methods: ['GET'] # Following is applied to every 'GET itemOperation'
dataprovider: 'instance of ItemDataProviderInterface'
- methods: ['GET']
route_name: 'some_route_name' # Following will be active on the current Operation identified by the route_name
route: '/my/new/route'
controller: '@some.service'
dataprovider: 'instance of ItemDataProviderInterface'
collectionOperations:
- methods: ['GET', 'POST'] # Apply filters and data provider to every collectionOperation methods
filters: ['@some.filter']
dataprovider: 'instance of CollectionDataProviderInterface'
tags: [ { name: 'api_platform.resource'} ]

please can anyone help me?

have a nice day