Symfony: Extract API from Implementation

[webmozart]

Hi everyone,

We had this discussion before, but at a point when we didn't have composer yet. I would like to bring this topic up again now.

Problem (updated December 5, 2012)

I just had a longer discussion with @fago, who is currently integrating the Validator component into Drupal. For Drupal it is necessary to replace the translation mechanism used in the Validator component by a custom implementation. Even when implementing Symfony\Component\Translation\TranslatorInterface in their code, this still pulls in all of the Symfony Translation implementation (and its potential dependencies).

The same is true for any Symfony component/bundle/bridge. When someone wants to use A which relies on B's interfaces, and B relies on C and D, he needs to pull in A, B, C and D _and_ provide a custom implementation for B (that might depend on E and F). Lots of installed dependencies: A, B, C, D, Bcustom, E, F

      .·´ `·.
     B     Bcustom
   .´`.     .´`.
  C    D   E    F

Proposal (updated January 7, 2013)

I would like to suggest and discuss a backwards compatible extraction of the Symfony API. In a nutshell, make it possible to use A which relies on B's interfaces but replace B with a custom implementation (i.e. only install A, Bcustom, E, F).

      |
   Bcustom
    .´`.
   E    F

Implementation

api/
    Symfony/
        Component/
            Translation/
                TranslatorInterface.php
    composer.json [symfony/api]
src/
    Symfony/
        Component/
            Translation/
                Translator.php
                composer.json [symfony/translation, requires symfony/api, provides symfony/translation-implementation]
            Validator/
                composer.json [symfony/validator, requires symfony/api, suggests symfony/translation-implementation]

Example: Replacing the translator

Drupal/
    Translation/
        composer.json [drupal/translation, requires symfony/api, provides symfony/translation-implementation]

The package MUST NOT add symfony/translation to its "replace" section. Multiple implementations of the same interface are NOT mutually exclusive and CAN be used at the same time.

Example: Using the Validator with its simplistic default translator

{
    "require": {
        "symfony/validator": "2.2.*"
    }
}

Example: Using the Validator with the Symfony Translation component

{
    "require": {
        "symfony/validator": "2.2.*",
        "symfony/translation": "2.2.*"
    }
}

Example: Using the Validator with a custom implementation

{
    "require": {
        "symfony/validator": "2.2.*",
        "drupal/translation": "..."
    }
}

The API would comprise a _closed set_ of interfaces and classes of the components, that is:

• the top-level interface(s)
• the classes and interfaces that the top-level interface(s) refer to in type hints and @return tags
• the classes and interfaces that _these_ classes and interfaces refer to
• etc.

Because of composer, this change would be fully BC.

Benefits (updated December 5, 2012)

1. A Symfony component A can rely on interfaces of another component B without forcing the user to use B.
2. Other projects can depend on Symfony's interfaces with a lower barrier to entry.
3. The API clearly documents the stable part of the components (it is ideally guaranteed not to change)

4. The separation helps us to decouple our components more. For example, if a component A instantiates a class of another component B and thus requires "symfony/b", we can improve it to make use of DI so that it only depends on "symfony/api" instead.

   Drawbacks (updated January 8, 2013)

5. One more composer package (symfony/api) has to be downloaded when using a Symfony component.

6. Core developers have to maintain interfaces in a separate directory.

Please let me know what you think.

[marijn]

:+1:

[iambrosi]

+1

[havvg]

For anybody not being allowed to use composer this will (worst case) double the amount of dependencies.
I don't see an actual benefit from it. What's wrong with having an implementation within the component? I mean it's about shipping some KB source code, or did I miss the point?

[fago]

Having the interface separated would be really useful. In fact, we even think about extracting it ourselves for Drupal - see http://drupal.org/node/1852106.

@havvg: Having a bunch of unused source code lying around which is actually duplicating existing functionality isn't very nice. The duplicated code is potentially confusing for newcomers who try to wrap their heads around the source code as it isn't immediatelly clear that the implementation is not used.

[tPl0ch]

:+1: for separation of API

[DavidBadura]

+1

[adrienbrault]

@fago What about symfony developers who'd have to look into 2 differents directories ?

:-1: because IMHO saving some KB doesn't justify complexifying the directory structure.

[webmozart]

@adrienbrault The more complex directory structure is mainly relevant for us core developers, and then only in part, because the interface certainly will change much less frequently than the implementation.

[willdurand]

-1 for the same reasons @havvg mentioned.

[lolautruche]

This is indeed very interesting, but then we're not really talking about API, but SPI (_Service Provider Interface_). We already have this approach in eZ Publish 5 (though not separated in Composer), so I'm obviously +1 :smiley: .

[webmozart]

I added an example above on how this would make it possible to replace Symfony packages by custom implementations.

[webmozart]

Updated the description to be less vague (I hope).

[lsmith77]

its certainly interesting .. i am hoping that we will have a general trend where PSR will provide those interface packages rather than Symfony2.

[trompette]

It shouldn't be done by changing the structure of the source code.
Either the interfaces are standard (as @lsmith77 stated) or there is a way to programmatically "replace" an interface.

[Seldaek]

I am kind of -1, it will duplicate the amount of symfony packages you have installed for not so much gain. If it's done in a generalized manner for all components I'm definitely -1. If targetted to a few specific components maybe, but I am not sure where we would draw the line.

[magnusnordlander]

I'm +1, but wouldn't a possible compromise be to store the interfaces both in a separate API package, and in the Symfony component package, and mark the component as a replacement for the API package?

It would make it a bit more of a hassle when the interfaces are changed (but still possible to automate), but it's not THAT different from the component subtree splits on GitHub.

[webmozart]

@Seldaek Could the people who vote -1 please also propose an alternative? The use case to replace a core component apparently exists, so we need to find a solution.

[DavidBadura]

@Seldaek Maybe a package called "symfony-api", that includes all api interfaces?

[Seldaek]

Let's be clear that replacing a core component is already possible. I read @fago's point and I can appreciate that, but I don't agree that this is the best solution.

Maybe having one Symfony\Interface package with all the main interfaces that really glue stuff together would be ok. I'd guess we only need 5-10 interfaces in there, and the components needing them could require it. That way we don't have an explosion of packages/complexity, but we still provide some amount of separation for those that really need it. Most likely the drupal guys would anyway need most of the other interfaces that are put in there, so combining them is not a huge problem.

Edit: Eh well, @DavidBadura beat me to it :)

[fspillner]

I'm kind of -1 and don't like the Symfony\Interface package, too.

An alternative: Maybe this cannot work, but just an idea: Let's pimp Composer to be able to download only certain packages / directories like this for example (I hope, @Seldaek doesn't get headache, after he has read this suggestion ;-)):

{
    "require": {
        "symfony/translation": {
            "version": "2.1.4", 
            "directories": ["Interface/"]
        }
    }
}

All interfaces of Translation component are moved into Interface directory.

[schmittjoh]

Do a few classes more or less really matter that much?

On Sun, Dec 2, 2012 at 10:37 PM, Fabian Spillner
[email protected]:

I'm kind of -1 and don't like the Symfony\Interface package, too.

An alternative: Maybe this cannot work, but just an idea: Let's pimp
Composer to be able to download only certain packages / directories like
this for example (I hope, @Seldaek https://github.com/Seldaek doesn't
get headache, after he has read this suggestion ;-)):

{
"require": {
"symfony/translation": {
"version": "2.1.4",
"directories": ["Interface/"]
}
}}

—
Reply to this email directly or view it on GitHubhttps://github.com/symfony/symfony/issues/6129#issuecomment-10935630.

[dlsniper]

I'm also -1 for this. Besides skipping a few files what's the point?

If it's not extensible for Drupal or any other project it means it's not extensible for many other projects so maybe we need to have a look on how to change that rather that change the whole structure of the framework.

Splitting the structure now would confuse the new developers even more and to be honest it would be really annoying to have the files split like that :)

[lennerd]

-1

Agree on @dlsniper.

[seiffert]

Why do you think that this would confuse? How often do you change an API method and touch both files - interface and implementation? And as a beginner, you don't dive into the core components on your first day...

Separating interfaces and implementation is a good idea IMO. I would even say that interfaces should always be designed from the client's point of view and therefore be placed outside of the implementation's structure.

[schmittjoh]

There might be some value in extracting the top-most interfaces of a
hierarchy (like the TranslatorInterface), but any interface that is only
required by a certain implementation of another, higher-level interface
(like for a example a TranslationLoaderInterface) does not make sense to
extract imo.

On Mon, Dec 3, 2012 at 1:13 PM, seiffert [email protected] wrote:

Why do you think that this would confuse? How often do you change an API
method and touch both files - interface and implementation? And as a
beginner, you don't dive into the core components on your first day...

Separating interfaces and implementation is a good idea IMO. I would even
say that interfaces should always be designed from the client's point of
view and therefore be placed outside of the implementation's structure.

—
Reply to this email directly or view it on GitHubhttps://github.com/symfony/symfony/issues/6129#issuecomment-10950561.

[webmozart]

@schmittjoh Your last statement is a bit unclear. It would be necessary to extract a _closed set_ of interfaces and classes, i.e., the topmost interface, all interfaces and classes that this interface refers to (via type hint or @return annotation), all interfaces and classes that these interfaces and classes refer to etc.

Of course, if an interface is not referenced in any way from the topmost interface, it's not necessary to include it in the API.

A single "symfony/api" package would be fine for me (but using the same namespaces that we have now).

[dlsniper]

So if I understand correctly, basically we should have:
namspace Symfony\Component\Yaml;

interface YamlInterface {}

but in the file symfony/api/Symfony/Component/Yaml/YamlInterface.php
and used in symfony/src/Symfony/Component/Yaml/Yaml.php for the framework

and then a vendor could write into his lib something like:

namespace Symfony\Component\Yaml
Yaml implementsYamlInterface
under vendor/X/src/Yaml.php in order to replace the functionality but maintain the interface?

Sorry, I really don't understand the use case and I want to learn more about this :) (if the example is not easy to understand I'll write a gist later on)

Thanks!

[schmittjoh]

@bschussek, that would make sense to me.

So, we should probably start with identifying these top-most interfaces, and once we have them, we can extract them and everything that they or their dependencies refer to via type hint, or return annotation. I'd also create different components for each api that we extract from a component.

[AdrianSchneider]

+1 for a single repository with only top-level interfaces.

I'd love to see the PHP community standaridze these into a more neutral group (php-fig?), but this is still a great start.

[webmozart]

@dlsniper Yes, except for the vendor implementation, which would obviously go into his own namespace, e.g.:

Vendor\Yaml\Yaml implements Symfony\Component\YamlInterface

I realize now that using "replace" or "provide" in composer is probably not that a good idea. For example, if I created a package "my/event-dispatcher" with the following composer.json:

{
    "name": "my/event-dispatcher",
    "require": {
        "symfony/api": "..."
    },
    "replace": {
        "symfony/event-dispatcher": "..."
    }
}

Then any package that requires "symfony/event-dispatcher" but is loaded together with "my/event-dispatcher" would fail if it actually instantiated a new Symfony\Component\EventDispatcher\EventDispatcher object. The conclusion is:

1. Packages that depend only on API interfaces or classes can require "symfony/api". These can be combined with any implementation.
2. Packages that instantiate concrete implementation classes need to require "symfony/concrete-component". In such a case, the component cannot be replaced by custom implementations (custom implementations can be used in parallel though).

Examples:

1. Validator uses the TranslatorInterface. So Validator would require the "symfony/api" package in its composer.json.
2. Form instantiates EventDispatcher instances. Its composer.json needs to require "symfony/event-dispatcher", the implementation cannot be replaced.

[webmozart]

I'd also create different components for each api that we extract from a component.

@schmittjoh Yes. I would completely mirror the current directory structure, only in a root directory api/ instead of src/.

[webmozart]

I updated the proposal above with the currently discussed results.

[javiereguiluz]

@bschussek in addition to Benefits, don't you think that it would be necessary to add a section about Drawbacks? This would make it easier to fairly evaluate your proposal.

[webmozart]

@javiereguiluz You're absolutely right. I added that now.

[webmozart]

I added a working PR for the Translation component as a PoC.

[stof]

@bschussek There is another big drawback with this proposal (see my comment on the related PR).

[adrienbrault]

I must have missed something, but woudln't it be better to find a solution at the dependency management level (composer) ?

[bamarni]

But why interfaces should be removed from symfony/symfony? imo better would be to have an automatic mechanism wich keeps them up-to-date in the symfony/api package, just like it's done for standalone components. And for the current use-case one would only need to create his own translation component depending on symfony/api and replacing symfony/translation?

[stof]

@bamarni Another package implementing the interface should not tell composer it replaces symfony/translation as it would forbid installing both packages at the same time (whereas there is no valid reason to forbid it)

And the proposal is not to remove the interfaces from the full symfony package but to move them inside it so that they can end in a different subtree split

[stof]

@adrienbrault if symfony does not distribute a package with the interface without the implementation, I don't see how Composer could magically give you such package.

[bamarni]

@stof : ok, I had missunderstood the proposal a bit and didn't see @bschussek explanation about replace drawback.

I don't really know how the hook on symfony/symfony works but would it be possible to have some other rules for an extraction than a subtree? Eg. tagging all the wanted interfaces with a given annotation / phpdoc and the script would grab them? This way nothing as to be moved.

[webmozart]

@bamarni Why the hassle? Basically the only consequence of moving them is that we core developers have to look into another directory for the interfaces. IMO this shouldn't be a problem really.

[stof]

@bamarni If we want to have all the hard work done by Packagist, we need to provide the individual packages as git repos. Otherwise, we would not have them for dev version and would have to build a custom composer repo by hand (which is what ZF does but causes many headaches when they mess their packages.json file or when someone forgot to add their repository).
And having a git repo without the proper history (having commit "updated the code from symfony/symfony at ...." all the time for instance) does not seem a good idea

[stof]

thus, you would still have the interface in each component subtree split, which would cause even more issues

[webmozart]

I updated the proposal again and added more examples and explanation.

[makasim]

-1

[bamarni]

@bschussek : I don't think it's a functional issue (just having them in a separate directory, personally I don't care) but it's more theorical.

This api package is not a "natural" package one would do but a given "view" of the symfony/symfony package to address a given problematic, while I'd understand such effort if we were talking about allowing a custom implementation of a fig accepted interface (if any?), this is really about "symfony making his own standards".

For eg. I don't think symfony/symfony#5968 would have been opened if an api folder existed. I'm not saying it shouldn't be done I'm kind of -0 on this but this is clearly not an obvious choice without any consequences.

[KrzysztofZalasa]

Composer can use git _sparse checkout_ feature - with it we can clone partial repository into project's vendors. Package author can describe API files in composer.json file so composer can build vendors based on it. So, we must not move files into separate folder / repository, we just can list it into composer.json

{
    "name": "famous/component",
    "api": {
        "Acme\AcmeInterface.php",
        "Acme\AcmeOtherInterface.php"
    }
}

{
    "name": "custom/compontent",
    "require-api": {
        "famous/component":  "1.0.*"
    }
}

[Seldaek]

Just to be clear, what @KrzysztofZalasa describes is a suggestion, this is not the case right now. It's also unlikely to happen IMO.

[dlsniper]

@bschussek I've given more thought on this one and I think that having the interfaces split into packages would indeed be a good thing but we'll need a list of which interfaces are separated and which aren't.

Also the interfaces should be shipped as separate packages, just like components are in order to promote the decoupling, I think, you want to achieve.

So, in the end, I'm +1 for this but given the time left to do this, we really need a decision fast on this one.

[webmozart]

@dlsniper Separating the API by component would duplicate the amount of Composer packages, as @Seldaek mentioned, and this way increase the download time and vendor directory complexity.

Nevertheless, we are waiting for @fabpot's blessing here.

[webmozart]

I updated the description with @stof's ideas from #6189. Symfony components should all feature a provide section:

{
    "name": "symfony/translation",
    "require": {
        "symfony/api": "self.version"
    },
    "provide": {
        "symfony/translation-implementation": "self.version"
    }
}

Other Symfony components should then require an implementation instead of our specific component.

{
    "require": {
        "symfony/translation-implementation": "self.version"
    }
}

This way, fetching a component will still fetch its mandatory dependencies by default like it did before, but making it possible to replace any component with a different implementation.

Userland projects will continue to just require symfony/translation.

[fabpot]

That might be something we should discuss for Symfony 3.

[dongilbert]

@ircmaxell's PHP Protocol's RFC could actually simplify this a LOT. Too bad it was pulled. It might be possible to get it revived if there's enough interest. https://wiki.php.net/rfc/protocol_type_hinting

[cordoval]

:-1: I think maybe this problem happens because of coupling between components. Also if someone is using just one package then why do i need all Symfony interfaces? is loading extra files I don't need. So that cons the point from @fago coming from Drupal. Some problem similar happens with the DI and Config component and how Drupal is not putting the last one under vendors folder https://github.com/symfony/symfony/pull/10920#issuecomment-44375340. Drupal is already hell heavy so I don't see the point of loading some extra classes. :-1:

[harikt]

Hi @webmozart ,

This is a great proposal and I love to see this. I have seen many of the core people disagree here, but you should consider the advantage of this than saying hard to look that code. API's once finalized will not change like implementations.

Thank you.

[linaori]

I would like to see a generic container interface, event dispatcher interface, response/request perhaps.. But that's not up to Symfony. I think this should be done via PSR and can then be implemented _backwards compatible_

[unkind]

As I've mentioned in #12544 and #12115, if you want to use some generic service that Symfony doesn't have yet (e.g. service which gets source code of closure), you are in difficult position as you can't add interface in any of existing components. It looks like separate package with API solves my problem.

[harikt]

Hi @unkind ,

I feel the one you have discussed in #12115 is different .

[unkind]

@harikt well, I was talking about last comment in #12115. ClosureDumperInterface requires by both DependencyInjection and Routing components and there is no reason to put it in one of them.

Current structure forces to couple interface with one of the components.

Yet another example: Symfony has Symfony\Component\DependencyInjection\LazyProxy\Instantiator\InstantiatorInterface. This is a very specific interface for the DependencyInjection component. You are not able to use @Ocramius's ProxyManager somewhere else without creating yet another too specific interface with moving part of component-related logic to the Bridge.

So it looks like a good idea for me. :+1:

[unkind]

More examples:

1. Symfony doesn't have Cache component, but some generic API would be useful in order to avoid explicit dependency on Doctrine Cache. Doctrine is just implementation detail.
2. Lock component request (#9357) was rejected because "there are already solutions to this problem". But later LockHandler was merged in 2.6 as part of Filesystem component. That's a little bit weird for me as you could reject it with the same reason. File-based lock handler is a nice feature, but it's an implementation detail as well. It doesn't work for multiple servers, for instance, so it could be a reason of scaling problem in the future. Lock handler interface is a good trade-off in this case, IMHO.

[ostrolucky]

I think decoupling/modularity is one of the goals of Symfony 4.0? I think this is perfect opportunity to reopen discussion about this. I welcome idea for something like symfony/contracts, it would help php ecosystem to adopt established interfaces instead of writing their own. Ping @fabpot!

[xabbuh]

I am not sure if this is something that Symfony should do. For reusable interfaces across projects there already is the PHP FIG. This group is doing a good job in creating standards for components that need a common interface and we were always fast in implementing those interfaces.

[ostrolucky]

You are probably right, this is ideal job for php-fig, even though I'm not confident in their activity level.

Anyway I investigated current situation and picked interfaces which might seem useful for general purpose. I also found some which might seem they should belong to the list but are IMO too coupled to implementation:

~Symfony\Component\EventDispatcher\EventDispatcherInterface~
\Symfony\Component\Lock\LockInterface
\Symfony\Component\PropertyAccess\PropertyAccessorInterface
\Symfony\Component\Serializer\SerializerInterface
\Symfony\Component\Serializer\Encoder\DecoderInterface
\Symfony\Component\Serializer\Encoder\EncoderInterface
\Symfony\Component\Serializer\Normalizer\NormalizerInterface
\Symfony\Component\Serializer\Normalizer\DenormalizerInterface
\Symfony\Component\Serializer\NameConverter\NameConverterInterface
\Symfony\Component\Templating\EngineInterface
~Symfony\Component\Translation\TranslatorInterface~
~\Symfony\Component\Validator\Validator\ValidatorInterface~

As you see in the end there isn't many of them. It's mostly interfaces of Serializer, because they are lean enough.

Also I checked coupling of Validator to Translator and it seems it can be decoupled?

Anyway I would like to put the end to this old RFC and see official decision.

[xabbuh]

👎 for me to make this change.

@symfony/deciders What is your opinion on this topic?

[javiereguiluz]

If I'm right, this is what Laravel does in its "Contracts" package, which contains all the interfaces used in their framework: https://github.com/laravel/framework/tree/5.4/src/Illuminate/Contracts

[unkind]

@ostrolucky if one doesn't use Symfony at all, I don't see reason to use Symfony contracts. Symfony interfaces don't magically make things better. Moreover, they are designed for Symfony needs and they probably won't satisfy _your_ needs later, so you restrict yourself for no reason. Why people are so obsessed with frameworks so they want to couple their projects with them as much as they can? What's the problem to make your own contracts?

https://speakerdeck.com/jakzal/decoupling-from-the-framework — probably, you may find it interesting (by the way, @jakzal is one of the Symfony maintainers).

[ostrolucky]

Why people are so obsessed with frameworks so they want to couple their projects with them as much as they can? What's the problem to make your own contracts?

This makes me believe you don't understand point of this proposal at all? It's not coupling to a framework but opposite. Currently when you depend on Symfony interface you need to pull all components that depend on implementation. Point of not writing own interface is to allow people of your project to swap implementation with some other (that is already made) without writing adapter. For example Doctrine could adopt many interfaces from symfony serializer and you would be able to use implementations across these projects as you seem fit.

[nicolas-grekas]

I think there is value in having shared interfaces, as the FIG does.
In Symfony's case, some interfaces could be worth moving outside of their component (eg ServiceSubscriberInterface from DI)
BUT the FIG would be a better place to publish them.
I think right now, Symfony didn't embrace the mission of providing generic interfaces on their own.
Until that happens (if it does at all), I'm :-1: also because that'd mean added maintenance.

Instead, I strongly suggest people to contribute to the FIG to move useful stuff there.

[javiereguiluz]

I'm closing this as "won't fix" because at least two core members voted against it. Thank you all for the discussion!

[unkind]

It's not coupling to a framework but opposite
Currently when you depend on Symfony interface you need to pull all components that depend on implementation.

We don't understand each other, because we use this term differently. I mean coupling between classes/interfaces, you talk about, in fact, composer packages. So when you start to use Symfony interfaces directly in your project you make coupling with Symfony. Symfony is a tool with its own restrictions, models, etc.

Let's say you depend on EventDispatcherInterface in your project. You decided to make your events immutable and exclude setName, stopPropogation from the Event? You can't do it anymore. Because it's not your contracts. So what's next? You come here and ask to change Symfony contracts, because it doesn't satisfy your needs now: #21827. So, you first couple with Symfony as much as possible and then you come to "fix" Symfony, because it doesn't meet your internal requirements.

[nicolas-grekas]

Continued in https://github.com/symfony/symfony/pull/27093