Composer: Consider adding basic inheritance support for composer.json

[schmittjoh]

Opening at Jordi's request as discussed on FOSRestBundle.

This would come in handy for testing a bundle against multiple different vendors without duplicating the entire file.

// composer.json
{
    require: { 
        "symfony/symfony": "2.*", 
        // ... more dependencies
    }
}

// tests/composer-2.0.json
{
    inherit: "../composer.json",
    require: { "symfony/symfony": "2.0.*" }
}

// tests/composer-2.1.json
{
    inherit: "../composer.json",
    require: { "symfony/symfony": "2.1.*" }
}

Edit: Solution

Use the Composer Merge Plugin from Wikimedia https://github.com/wikimedia/composer-merge-plugin

[naderman]

Hmm, I'm sort of torn on this feature. How regularly does one need this? And doesn't one typically just create a second file for temporary testing which one doesn't maintain?

[Seldaek]

The idea of that tests thing is to be able to run the CI against multiple dependencies configurations, which is definitely nice. But yeah, as I said on the other ticket, I'm not too keen on doing this because it introduces complexity in terms of merging rules. How do we add or override or remove etc..

The copies in tests/ really don't need to be full copies, they only need the require (and autoload I guess) since it's only about loading different dependencies, they do not need the complete package info with version, name, etc.

[schmittjoh]

It might also be useful in other scenarios like for the subtree splits of the components in the Symfony2 repository. Regarding the merging, one option might be to use the Symfony2 Config component which would also have the nice side-effect of adding documentation for the available config options.

[naderman]

Can you explain how this is useful for subtree splits?

[schmittjoh]

You could have the common information in a file that you inherit from in all subtree splits.

[Seldaek]

How can you inherit from it if it's not present in the subtree splits?

[naderman]

But how do you get that common information when you only have an individual subtree?

[schmittjoh]

I think using git filter-branch this should not be necessary i.e. you would have the base file in all splits, but I haven't tried it.

[lsmith77]

i am in a similar situation trying to test doctrine/phpcr-odm against different backends. what i really just want is the ability to do:

php composer.phar install jackalope/jackalope-jackrabbit
php composer.phar install

[gggeek]

I am not sure if this is exactly the same usecase (or if the need I have is already solved shomehow):
with EZP, both version 4 and 5, we distribute a composer.json which resides at the root of the "user project" (or web app). This is not ideal because the end user should probably have his own composer.json there - if he just hacks the one we deliver, his changes will be lost on upgrades.
The cleanest solution I can think of is having one composer.json including another one, a sort of composer-user.json.
Does that make sense?
Notes:

• having the main composer.json delivered as part of the CMS makes it easy for it to use all the goodies coming with being the root package
• we can not just depend in the cms composer.json on a "user-project" package in a standard way, as there is no fixed name for it

[cmfcmf]

I'm having a similar use-case like @gggeek, where the user might want to add custom requirements to the composer file which shouldn't be overridden on upgrade.

[sergeylukin]

@gggeek +1 here. I have multiple apps that are related and use shared files. Currently each app has it's own composer.json but they are mostly identical with minor differences. I'd like to have one composer.json in the shared space and one per each app so that composer could read from both in every project.

[unn]

+1

[ahukkanen]

+1

[dredav]

+1

[rskvazh]

+1

[patrickallaert]

+1 Same here, and as @gggeek, this is precisely on an eZ Publish (EZP) 5 setup that I am currently facing that issue.

[FlashPanther]

+1

[vaxvhbe]

+1

[andrerom]

I would have to agree with @gggeek here, however it is probably not the same use case as this issue.
And ideally @gggeek 's use case should be a dist file in upstream repo, and an optional user file with _always_ extends the dist.

[nadar]

+1

[maximilianredt]

+1

[bgallagher]

:+1:

[RobinRadic]

+1 i would use this a lot during development

[legoktm]

For MediaWiki we have developed a composer-merge-plugin to add this feature.

The core MediaWiki repo has a composer.json file which includes a composer.local.json file if it exists. Since some MediaWiki extensions also have dependencies installed via composer, a sysadmin would source those composer.json files in their composer.local.json file, and running "composer update" would fetch everything.

[marcegarba]

+1

Project inheritance (similar to what Maven does) would be wonderful.

[gpupo]

+1

[Zoddo]

+1

[onlab]

+1

[maechler]

+1 Facing a similar use case as @gggeek

[cansik]

+1

[johnitvn]

+1

[secondtruth]

This is extremely useful for the use case that @gggeek described. Contrary to @schmitt joh's suggestion, for this use case a include(s) field should be preferred (see here). The parameter structure of the composer-merge-plugin looks very good. This project seems to be a good starting point.

Another project that, I think, would like to use this feature is Roundcube Webmail. They use Composer for their plugin system. And I prefer this solution for my own framework-like project, too.

Basic usage:

{
    /* ... main dependencies ... */

    "includes": [
        "composer-local.json"
    ]
}

or something like this:

{
    /* ... main dependencies ... */

    "includes": {
        "local": "composer-local.json", /* local = no warning, if file not found */
        "test": "composer-test.json" /* to use: invoke `composer --test-mode` ? */
    }
}

[aequasi]

+1

[haydenk]

+1

This could be a lot more useful than the local directories feature. You could include subprojects similar to how you can in gradle.

[etiennemarais]

+1

[dashdanw]

+1 this solves a very particular problem we're having at my company right now using composer with multiple projects.

[matt-schwartz]

+1 This would make it a lot simpler for app installations to have custom additions, e.g. for installing plugins. In Symfony this would make adding multiple 3rd party bundles easier.

[ocasta]

+1 This would make it much simpler to maintain a core docker wordpress image and then add application specific plugins in a dependant container/image

[lpalgarvio]

this would be a great addition for drupal when using profiles and features

[mikebronner]

+1 @secondtruth I would love this feature for usage with package development as submodules, but then pull in their composer.json to the main project. This would make CI so much easier. (If there's a work-around for this already, please let me know.)

[dashdanw]

@mikebronner we have been dealing with similar issues at our job, myself and @matt-schwartz would love to hear if you ever figure out a reliable solution.

[bd808]

@mikebronner composer-merge-plugin should work pretty well for the use case of a main project that wants to aggregate the composer.json files from a number of child packages that are included via git submodules or even embedded in a larger git repo.

Something this simple might do all the work needed:

"require" : {
  "wikimedia/composer-merge-plugin": "dev-master"
},
"extra": {
  "merge-plugin": {
    "include": [ "*/composer.json" ]
  }
}

This would look for a composer.json file in every top level subdirectory of the project and include the contents of all the files found (e.g. foo/composer.json, bar/composer.json) in the internal Composer configuration at runtime.

composer-merge-plugin was originally only concerned with processing require and require-dev configuration but has been expanded to support many other configuration sections.

If we can keep this kind of complex use case support out of Composer proper but make it easily available for advanced use cases through the plugin mechanism that seems like a net win for Composer itself in terms of both code complexity and extensiblity. That being said I'd be more than happy to see a core implementation proposed and merged that made composer-merge-plugin obsolete.

[RyanThompson]

@Seldaek if this could be included in the plugin API as well, that would be epic.

My only use case is Pyro but in this particular use case, addons are composer packages (at least in spec, not necessarily on Packagist or anything). So being able to run an update from the project root and let it be aware of the dependencies of each addon would be pretty amazing.

Being that things are moving more and more towards packages and decoupling, I think will be more and more of an important feature.

Ryan

[curry684]

Please stop posting +1 all over and use the thumbs up on the main PR post: https://github.com/blog/2119-add-reactions-to-pull-requests-issues-and-comments

[keithy]

if you use multiple composer.json files with different vendor-dir you end up with two vendor directories.
lib.json with vendor-dir /lib
composer.json with vendor-dir /vendor

Then have the lib one include "files":[ "vendor/autoload.php" ]
and have your code include "lib/autoload.php".

generate the /lib directory like so:
env COMPOSER=lib.json composer install

This allows your code to be managed by lib.json
And the user gets their own composer.json with require and require-dev dependencies.

K.

[Seldaek]

I just don't see us working on this anytime soon, or ever, so I'm going to close as won't-fix, with the understanding that the composer-merge-plugin is a good solution for those who absolutely need this.