Drupal-project: Add support for installing libraries via asset-packagist

[fago]

Let's support adding libraries via asset-packagist.

If the setup would be done by drupal-project, adding libraries would be simple. We are using this approach for adding module-needed libraries via composer for a while at drunomics now and it works great. It's been adopted by thunder 2.x as well, see https://github.com/BurdaMagazinOrg/thunder-project/pull/5/files.

Once this is done, you can add libraries via a simple command:

composer require bower-asset/dropzone=~4.2

More explanation: https://asset-packagist.org/site/about

[rodrigoaguilera]

I have been using it also and I think is great.
I copy the comment from here
https://github.com/drupal-composer/drupal-project/pull/215

I found a way that can make the installation of external libraries from npmjs.org or bower into the folder /web/libraries/ easier

Recently this composer repo was created
https://asset-packagist.org/
but there still the problem that this libraries do not set a "type" for "composer/installers" to use so is difficult to set a path like /web/libraries/$name

There is this plugin that add a default type to all packages
https://github.com/oomphinc/composer-installers-extender

So we modify this project to add the assets repo, the plugin and write something like the following in the composer.json

"installer-types": ["npm-asset", "bower-asset"],
"installer-paths": {
  "web/libraries/{$name}": [
    "type:drupal-library",
    "vendor:npm-asset",
    "vendor:bower-asset"    
  ],
}

So we can do things like
composer require "npm-asset/jquery.easing":"~1.0"

and it will place the library into /web/libraries/jquery.easing

There is another explanation here
https://asset-packagist.org/site/about

No need to duplicate js libraries in packagist for a drupal "version".
In projects with a lot of external libraries the repositories section can get quite messy and https://github.com/fxpio/composer-asset-plugin is slow and complicated.
Also overcomes the limitations referenced in this PR about changing the library version.

Is it interesting to add this to the drupal composer project?

[arh1]

Great -- thanks.

To restate the steps above more explicitly:

1) Add the Asset Packagist repository to composer.json :

    "repositories": [
...
        {
            "type": "composer",
            "url": "https://asset-packagist.org"
        }
    ],

2) Add the Composer Installers Extender plugin:

composer require oomphinc/composer-installers-extender

3) Add installer types and paths to composer.json :

    "extra": {
...
        "installer-types": ["npm-asset", "bower-asset"],
        "installer-paths": {
...
            "your/installation/path/{$name}": [..., "type:npm-asset", "type:bower-asset"],
...
        }
...
    }

4) Add your npm or Bower packages from Asset Packagist, e.g.:

composer require npm-asset/select2

With the installer path above, select2 should now be installed to your/installation/path/select2

[rodrigoaguilera]

Exactly.

Now I extended this pattern to all my drupal 8 projects and it is working great.

I feel is time to do a PR with the changes explained here plus a little explanation in the README based on
https://github.com/drupal-composer/drupal-project/pull/215

[ao2]

If we assume that polluting the root composer.json is OK, then a solution without a central repository could also be adding a "package" repository snippet directly to the root composer.json file to download the external library, defining the Drupal assets in hook_library_info_build.

However, IMHO, if a module depends on some external asset there should be a way to specify that dependency in a way local to the module (e.g. confined to the module composer.json or .info.yml) without leaking this info into the root composer.json, which would look like a layering violation to me (the main project knowing too much about the dependencies of its dependencies). Composer does not make this easy tho, because repositories can only be specified in the root composer.json file, see https://getcomposer.org/doc/faqs/why-can%27t-composer-load-repositories-recursively.md

The libraries module and the Libraries registry project try to solve that without the need to change the root composer.json file.

[rodrigoaguilera]

You have really good points but building a libraries registry sounds like setting up another niche repository of libraries useful for Drupal forgetting all that composer can bring you with semver and easy management.

However, IMHO, if a module depends on some external asset there should be a way to specify that dependency in a way local to the module (e.g. confined to the module composer.json or .info.yml)

I completely agree but there is no solution that allows you to manage the asset dependencies of your module from composer.

The libraries module and the Libraries registry project try to solve that without the need to change the root composer.json file.

I think is easier to submit the asset to packagist and define the dependency on the composer.json of your local module.

So from my point of view polluting the root composer.json is a trade-off for having an easier way of specifying asset dependencies for composer drupal projects and getting the flexibility of downloading them not only on "web/libraries"

[ao2]

You have really good points but building a libraries registry sounds like setting up another niche repository of libraries useful for Drupal forgetting all that composer can bring you with semver and easy management.

I am not a huge fan of the Libraries Registry either, however one point about it is that the format used for library definitions describes the assets from a Drupal point of view, freeing the module developer from having to make the assets accessible to the Drupal rendering system via hook_library_info_build() or MODULE.libraries.yml, this solves a Drupal problem and reduces code duplication.

However, IMHO, if a module depends on some external asset there should be a way to specify that dependency in a way local to the module (e.g. confined to the module composer.json or .info.yml)

I completely agree but there is no solution that allows you to manage the asset dependencies of your module from composer.

Yeah, composer not supporting recursive repositories is a major pain point.

I thought about a local solution for the libraries module in https://www.drupal.org/node/2887738 I'll try to come up with a proof-of-concept patch about it.

The libraries module and the Libraries registry project try to solve that without the need to change the root composer.json file.

I think is easier to submit the asset to packagist and define the dependency on the composer.json of your local module.

So from my point of view polluting the root composer.json is a trade-off for having an easier way of specifying asset dependencies for composer drupal projects and getting the flexibility of downloading them not only on "web/libraries"

If this was going to be the “official” way, adding support for package types from asset-packagist.org directly to composer/installers might be worth a try, this would make it possible to avoid composer-installers-extender and minimize the pollution of the main composer.json file.

I would still miss a solution without a centralized asset repository tho.

Anyway, an opinion from Drupal core developers could be useful on these matters.

[hawkeyetwolf]

I posted these comments on the PR but they should live here:

---

Adding component into the installer-types and library path definition would also add support for all the packages in the components org on Packagist proper, like components/chosen (for Chosen module), components/jquery_ns_autogrow (for Autogrow Textarea), and components/Sortable (for Paragraphs v1.2).

---

The asset-packagist approach doesn't work for contrib modules right now because they can't safely require asset-packagist dependencies. Per the composer docs:

Repositories are only available to the root package and the repositories defined in your dependencies will not be loaded. Read the FAQ entry if you want to learn why.

And this causes two problems when a contrib module requires an asset-packagist dependency:

1. Breaks testbot, since she doesn't have the asset-packagist repository definition in her root project (right now).
2. Prevents requiring the module via Composer by any site which doesn't have the asset-packagist repository defined. We can put big bold words on the d.o project page and in the README of each project, but it still seems problematic.

What do you think? Is there any way around this?

[fago]

What do you think? Is there any way around this?

Yeah, standardization & documentation. If we could standardize on using asset-packagist (or an alternative on d.o. infrastructure) this would be a non-issue.

[digitaldonkey]

This would be amazing. It would release the pressure of creating a update of a Drupal module every time a library packed changed, which currently makes using js assets practically unmaintainable.

[mxr576]

Related PR is here: https://github.com/drupal-composer/drupal-project/pull/286

[rodrigoaguilera]

PR rebased and tests passing.

I just noticed more distributions are adding the asset-packist repo so is really becoming a standard already.
https://www.drupal.org/project/varbase/issues/2974100

One solution that hasn't been mentioned is that maybe the npm-asset and bower-asset vendors can be integrated somehow into the Drupal packagist repo
https://packages.drupal.org/8
after all is just free software and is published at
https://packagist.org/packages/hiqdev/asset-packagist
And since this libraries will be for only for Drupal they can have the drupal-library type already to simplify the root composer.json.

This will involve the Drupal association to do a significant amount of work but I feel it will be pretty ideal :)

[mxr576]

https://github.com/fxpio/composer-asset-plugin/pull/330 if this PR gets accepted this could provide a better way not just for us, but for other PHP frameworks to depend on JS libraries (assets) without depending on the source of the library.

This is also related to https://github.com/hiqdev/asset-packagist/issues/100.

[francoispluchino]

@mxr576 Thank you for your comment at the beginning of your PR. Indeed, Asset Packagist internally uses the Composer Asset Plugin to work. Also, I answer you directly in this issue because my answer will be much wider than the scope of your PR.

To answer your question, yes I know Drupal, even though I have never developed for this excellent content platform, and I am delighted to see that other projects use my plugin, even through other tools like Asset Packagist, to achieve exactly what motivated me to create this plugin in 2014, ie manage asset dependencies of PHP libraries (public and private).

As indicated in the Readme file of CAP, I do not add new features personally anymore, but I still continue to merge the work of the contributors and maintain the versions. So if you need the PR fxpio/composer-asset-plugin#330, and that it does not break the backward compatibility, I do not see why it would not be merged.

The reason for this decision is simply because I made a new version incompatible with CAP, which is so a new project with another name, and which is much more efficient and more in line with the current situation of the Front-end development. But before you expose my work, I want to clarify that I appreciate very much the work done by the team of @hiqdev for their server, even if consequently, my plugin is much less used :-).

Since I see that you plan to use my plugin via Asset Packagist to manage the asset dependencies of Drupal modules and themes, I would like to bring to your attention the more recent work I have done to remove all the constraints of CAP and Asset Packagist, while taking into account the new uses of Javascript libraries. To understand my motivations, you must look at this comment and the issue fxpio/composer-asset-plugin#93.

As you probably are, Bower was deprecated since October 2017, and so he recommend to migrate to the NPM registry using NPM or Yarn. But given that we have to use more and more javascript tool such as Webpack, Gulp, Grunt, Babel, TypeScript, Scss / Sass, Less, etc ... to use Javascript dependencies, we must therefore use more and more Nodejs with our PHP projects, even if we manage the versioning of assets via CAP or Asset Packagist. It should also be known that unlike Bower, NPM and Yarn have not a flat dependency tree, but it is possible to install multiple versions of the same dependencies, which Composer does not. Of course, I tried to hack the mechanism, but this one is limited and we always find incompatibilities or bug, ditto for the conversion of the ranges of version.

From this finding, and from the poor performance of the Composer's Solver SAT with the VCS Repositories, while noting that NPM and Yarn no longer have limitations on missing important features, I so realized the Composer plugin Foxy. This one does the reverse work of CAP, and so it leaves asset management to NPM or Yarn, while providing the same basic functionality as CAP / Asset Packagist (the one you are looking for), but giving you the ability to use all the tools and workflow available for front-end development.

I will not rewrite all of its benefits and features here, but in a few words, Foxy is a fast, reliable, and secure NPM/Yarn bridge for Composer. So, if you are interested by this project and would like to know more about it, I recommend reading the Readme and the FAQ, mainly:

• Foxy Readme
• Why this plugin?
• What is the difference between Foxy and Fxp Composer Asset Plugin?
• How does the plugin work?