I have been using fxp and until now I had only trouble with it. It feels like it's flawed by design and makes managing the bower/npm assets a real PITA. The idea is extremely awesome, but this plugin is just near to useless.
I have been patching some very big issues it has with npm/bower/git, but issues keep popping and it just feels that the entire plugin was badly designed and that those issues won't be fixed ever or will require major changes.
Is it time to switch to something else? Or maybe invest some time working on that plugin?
the plugin was mainly used to allow requiring js and PHP dependencies together, we will need to find a better solution for 2.1. The plugin works fine for simple cases but if you have a complex bower/npm setup in your application you should consider using it natively. It was mainly ment to make it easier for unexperienced users.
I guess I'll do as you say and switch to npm/bower instead of using composer for js/css deps.
I wanted to share how I'll be switching awaay from fxp. After hitting a wall with NPM's useless prefix
and .npmrc
, I decided that running commands from composer.json
isn't a bad idea at all:
"scripts": {
"post-install-cmd": [
"npm install",
"bower install",
"rsync -aX node_modules/ vendor/npm/",
"rsync -aX bower_components/ vendor/bower/",
"rm -rf node_modules bower_components"
],
"post-update-cmd": [
"php composer.phar run-script post-install-cmd"
]
},
you should remove it completely. it's useless piece of garbage.
jquery and other projects are available via packagist now. if not, you can make it on your own.
In my view, the best way is you can simply use distributions like ajax.googleapis.com for demonstration purposes and leave choice for end user
also, this plugin requires you make it global or install before yii2 installation via composer, otherwise you are stuck with 2.0.xbeta with very strange errors!
by now, i'm forced to get useless jquery files etc etc. and composer doesnt have option to ignore dependencies or i don't know it. i use something like this so far to suppress useless stuff
"replace": {
"bower-asset/jquery" : "*"
},
regards
You might want to evaluate https://github.com/koala-framework/composer-extra-assets, we wrote it for our cms and it does the job for us.
@nsams Does it work? Or it's the same shit as FXP? And by "work" I mean does it handle:
1
, 5 || 6 || 7
, 3.x
, etc...)A quick test is trying to install jsonpath
, which has the 3rd and the 2nd point, and "npmconf" which contains the 1st point.
Please keep your words calm, fxp plugin is not shit, it serves well for its purpose, which is to make it easy to simply require bower/npm packages with composer, it is not meant to replace more complex use cases of bower/npm. It is mainly useful to get a quick start where you do not have to setup a set of different tools to make a basic app work. When you want all the features of npm/bower, you should use them directly. Coupling this extension so tight to yii2 was a mistake which we have to fix in 2.1.
@yurii-github you're wrong about packages availability from packagist. There are some packages but the number is nothing like the number of packages available from bower. Also, most of these libraries maintained in bower only so you have to do extra work of maintaining your own composer mirror which isn't really good.
Anyway, as @cebe said, we'll, most probably, use bower/npm directly in 2.1 or even won't stick to any method of getting client packages leaving it to end user.
@alexandernst: yes that all works:
{
"require": {
"koala-framework/composer-extra-assets": "*"
},
"extra": {
"require-npm": {
"jsonpath": "0.1.7",
"npmconf": "2.1.2"
}
}
}
...simply because native npm is used.
Then I'd vote to switch from FXP to Koala
Both things sound reasonable, imho.
imho, we should use different package managers for different languages. I didn't saw any "plugins" to install php packages with npm or bower... Or we should ask @seldaek to handle any package from github, not only those who have composer.json inside, like go language does.
@trntv Koala (from what I understand) is using NPM and Bower. The benefit from using a plugin is that you can have all your libs in a single file, instead of fighting with 3 different files.
About the other thing... That won't happen. Composer devs refuse to even modify the SAR logic so that composer can handle upper/lowercase names. Making George R. R. Martin finish the last 2 books of "Song of Ice and Fire" before the end of 2015 would be easier than making composer devs even think about what you suggest.
we can always fork composer project or create own package manager - this is opensource world.
just dreaming :-) But, imho, we still can use different package managers for different languages, this is ugly, but faster and more reliable than fxp plugin.... and someone using php framework to create application rather than CMS should be familiar npm or bower etc.
One of the main thing we should not forget about assets: we should give developers the way to submit assets with their composer packages.
Currently FXP gives us ability to define bower or npm assets in composer.json not only for root project, but also for dependencies. We should keep this ability. I mean not to include them via composer.json, but may be some other way.
One of the main thing we should not forget about assets: we should give developers the way to submit assets with their composer packages.
Currently FXP gives us ability to define bower or npm assets in composer.json not only for root project, but also for dependencies. We should keep this ability. I mean not to include them via composer.json, but may be some other way.
:+1:
One of the main thing we should not forget about assets: we should give developers the way to submit assets with their composer packages.
Currently FXP gives us ability to define bower or npm assets in composer.json not only for root project, but also for dependencies. We should keep this ability. I mean not to include them via composer.json, but may be some other way.
I agree. For Yii extensions, there needs to be a way to define a bower or npm asset as a dependency.
There's also the issue that two extensions can require two different, incompatible versions of a bower asset, and since koala-framework/composer-extra-assets uses the post-install hook, I don't think it can solve this.
The FXP plugin converts the bower package to a composer package, so it can influence dependency resolution, and composer can either figure out how to satisfy all the dependencies (e.g. install an older version of one extension that depends on a compatible asset), or abort the whole installation. I think koala will just fail to install the bower packages: https://github.com/koala-framework/composer-extra-assets/pull/8#issuecomment-92891128
Nope, it will install both versions.
Are you sure? It merges the versions as "somepackage": "2.0.* 3.0.*", which is an intersection: https://github.com/npm/node-semver#ranges
Comparators can be joined by whitespace to form a comparator set, which is satisfied by the intersection of all of the comparators it includes.
If it installs both, then where does it install them? They can't both be in vendor/bower-asset/somepackage, but both Yii extensions will expect to find them there.
Are we talking about bower or npm?
bower. koala installs npm packages into the package folder, so that shouldn't be a problem.
However, the problem then becomes that different versions of the same npm package are installed multiple times.
I don't like it either, I also vote to remove it from 2.1, and just use plain old bower.json.
Plain old bower.json isn't a solution for Yii extensions. What fxp/composer-asset-plugin aims to do is great: it allows a Yii extension to depend on a bower or npm asset, which gets installed globally, so in a large project you don't end up with three copies of a jquery plugin in three different places in three different yii extensions, which are then all loaded together and conflict with each other.
You also have to consider that if you remove the asset plugin, you'll break all the Yii extensions that depend on a bower or npm asset.
Breaking extensions isn't a huge issue since 2.1 would be incompatible in terms of API (at least in some cases).
That doesn't mean you have to break compatibility.
I don't like all the problems this plugin causes either, but I don't see a better solution to the problem that it fixes. Just throwing it out isn't a solution.
Plain old bower.json isn't a solution for Yii extensions. What fxp/composer-asset-plugin aims to do is great: it allows a Yii extension to depend on a bower or npm asset, which gets installed globally, so in a large project you don't end up with three copies of a jquery plugin in three different places in three different yii extensions, which are then all loaded together and conflict with each other.
It's not great, it's annoying. Right now you need install bower packages even if you don't use it. It's always should be a app decision which frontend dependencies should use, not extension.
I think this can be solved by composer post-install command which will search bower.json files in dependencies, simple merge it and save to project bower.json or display some instructions how to deal with it.
It's always should be a app decision which frontend dependencies should use, not extension.
Not if the extension needs a frontend package to work.
This frontend package still can have conflicts with another package. Or you can use fork or alternative (like bootstrap vs bootstrap-sass). So extensions should have a way to define frontend dependencies, but it is the end user should decide whether to use them.
You can use repositories
in your root composer.json if you really have to override a package.
In any case, that's not the typical use case. Normally, a Yii extension will have something like "bower-asset/foobar": "2.*"
, and if you want to use a specific version in your app, you just specify a stricter version constraint in your app's composer.json. If you want to use "bower-asset/foobar": "3.*"
in your app, then you can't have both that and the extension that needs version 2.0, but that's working as intended, because the extension is not compatible with version 3.*.
On a related note, I think it would be a good idea to split the frontend parts of Yii into an optional package. That of course means you'll lose a lot of functionality, e.g. activeform's client side validation. But if you're writing a backend-only api, that's fine.
I think this can be solved by composer post-install command which will search bower.json
files in dependencies, simple merge it and save to project bower.json or display some
instructions how to deal with it.
that is exactly what https://github.com/koala-framework/composer-extra-assets does.
@nsams have you tried it?
Anyone knows how composer installs bower-asset packages without fxp/composer-asset-plugin
installed?
$ cd ~/projects/yii2
$ export COMPOSER_HOME=~/.new_composer
$ rm -r vendor
$ composer install --prefer-dist
.......
Installing bower-asset/jquery (2.1.4)
Downloading: 100%
- Installing bower-asset/jquery.inputmask (3.1.63)
Downloading: 100%
- Installing bower-asset/punycode (v1.3.2)
Downloading: 100%
- Installing bower-asset/yii2-pjax (dev-master 3f20897)
Downloading: 100%
........
If you have a lock file and you use --prefer-dist, it installs them into vendor/bower-asset
, since it can download the zip file referenced in the lock file.
Otherwise it doesn't work, since it can't find the package in packagist.
@samdark I'm the author of composer-extra-assets and we use it in production
:) OK. Have you ever had problems with Composer breaking compatibility?
@nsams I have a question: does composer-extra-assets install bower, npm and then execute those commands?
@samdark: I don't use any internals of composer except the Plugin API so the plugin is pretty robust. One issue I had though: https://github.com/koala-framework/composer-extra-assets/commit/928695baebee8b0a2cc5ad233f4824a1155365fe
@Faryshta: yes, but if they are installed globally those will be used
@nkovacs, thiank you
@nsams Does it allow install only PHP dependencies or update only bower/npm packages? Does it override bower.json and package.json on every composer install/update? How it deals with bower packages conflicts?
Does it allow install only PHP dependencies or update only bower/npm packages?
No - as it is a plugin that executes on post-install and post-update
Does it override bower.json and package.json on every composer install/update?
Yes.
How it deals with bower packages conflicts?
In case of an conflict you get an error. Php packages and npm packages are installed, only bower are missing then.
@nsams Do you have plans to improve this? I think so user should have a option to decide if he wants install npm or bower dependencies. Maybe some prompt?
It should also be possible to merge local bower.json with dependencies bower.json files.
@nsams The approach is interesting and we're going to explore it furhter. Not sure if we'll go with using the plugin, writing our own (mainly to have better control over it) or doing something else.
Do you have plans to improve this? I think so user should have a option to decide if he wants install npm or bower dependencies. Maybe some prompt?
I can't think of a reason why a user doesn't want to install all dependencies including npm+bower.
It should also be possible to merge local bower.json with dependencies bower.json files.
You specify dependencies in composer.json and those will be merged. We don't create a bower.json at all. Any specific reason why you need to do that?
The approach is interesting and we're going to explore it furhter. Not sure if we'll go with using the plugin, writing our own (mainly to have better control over it) or doing something else.
Nice to hear that, feel free to fork it and/or create pull requests
I can't think of a reason why a user doesn't want to install all dependencies including npm+bower.
I can have all scripts and styles already build and served by CDN, so I don't need npm and bower packages on production server.
You specify dependencies in composer.json and those will be merged. We don't create a bower.json at all. Any specific reason why you need to do that?
I just think so it is better to keep it as close as possible to native solutions - many people are familiar with bower and they will search frontend dependencies in bower.json. There are already tools that are based on bower and bower.json file, like wiredep. At some point a non-standard solution will begin to limit the possible tools that we can use.
I just want to say that Bower really shouldn't be used as it's completely unreliable. More information here https://gofore.com/ohjelmistokehitys/stop-using-bower/
I think if you use fxp/composer-asset-plugin, it processes bower subdependencies with composer as well, since it just converts all the packages into composer packages. That means composer will simply refuse to install this set of packages, because it's incompatible. That is a good thing, although I think you can override it with aliases.
In the example given in the blog post, angular-bootstrap is not compatible with 1.3.0, but angular-leaflet requires angular 1.3.x. This is a conflict. Bower allows you to override the conflict, and then pray that either angular-bootstrap works with 1.3.x, or angular-leaflet works with 1.2.x. It's possible that angular-leaflet works with angular 1.2.x, but the developers didn't bother to test it, so they just restricted it to 1.3.x.
The real problem with bower is that it's not universally accepted in the frontend js world, and often packages are registered by someone who doesn't even own the repository. Some frontend package developers refuse to use bower and don't want to include dist files in their repos, so even if there's a bower package, it's not of much use for yii, because you have to build the package with whatever tools they use, you don't have a dist file you can load in a browser.
Npm as an alternative is better, but it's mainly used for server-side js and stuff that uses browserify or webpack to build the frontend js. So again you're often left without a dist file you can just load in a browser.
I can't think of a reason why a user doesn't want to install all dependencies including npm+bower.
I can have all scripts and styles already build and served by CDN, so I don't need npm and bower packages on production server.
We don't run composer on production server, we build locally and copy the result during deployment. And you probably need npm dependencies during runtime.
Thing is that you can't add a custom parameter to composer install/upgrade
a la --skip-bower
.
You specify dependencies in composer.json and those will be merged. We don't create a bower.json at all. Any specific reason why you need to do that?
I just think so it is better to keep it as close as possible to native solutions - many people are familiar with bower and they will search frontend dependencies in bower.json. There are already tools that are based on bower and bower.json file, like wiredep. At some point a non-standard solution will begin to limit the possible tools that we can use.
Due to the merging that has to be done we can only support a subset of bower.json. Additionally bower.json in the root package will change on composer update
when dependencies have changed bower dependencies.
Regrarding npm vs. bower: a flat structure is a must for frontend packages. You don't want multiple versions of the same library loaded.
But npm will probably soon be a able to do that: http://blog.npmjs.org/post/101775448305/npm-and-front-end-packaging
the less 3rd parties the better your app is.
how is it hard to copy one js file? Or check it's off website once a month? Or even make own checker for released file by hash? Definitely too hard, so we need another buggy mess to do it, and copy not one file, but whole project with building tools. Why life make easier, it must be hell. Great.
If framework is based on composer/gist, then it must be only composer, otherwise we get redundancy and increase error fields.
bower will die rather sooner than later. it's not a company.
npm may die later as it is company and nodejs is keeping it alive, but nodejs market is less then 0.2%. So, it's rhetoric question.
in the end, i just use composer replace to not get yii2 dependency nonsense.
unsubbing, it makes me disappointed.
@yurii-github that's basically the same thing as composer. If you'll check forums, once in a month there's a rant about it i.e. "why all the composer mess if we can download zips and extract ourselves".
bower will die rather sooner than later. it's not a company.
Composer isn't a company as well. Yii is not a company. Well, PHP isn't.
i'm one of them then. lol
too many "I" but anyway..
i'm not a fun of composer too, any company that blindly rely on it is non-profit or suicide company, it's like adding unknown 3rd party repo to your linux for some needed software instead of building it by yourself. Faster? probably. Results? finger crossed.
Framework is based on it and i have no real choice, of course, i cannot decline advantage of it during unstable phase of development.
in the end, i'd rather use simple git repositories by my self, less error prone.
about Yii,
so, Yii is a company, small/personal, but still.
PHP definitely has backed company. it is ZEND
composer on other hand is just "Open Source Developer, John STEVENSON" certificate aka https://www.linkedin.com/in/jr0cket when installing on windows.
74 bugs open https://github.com/composer/composer/labels/Buge
etc
i'm moving from point "if you can get less dependencies you should do that"
same comes for codeception that is built on top of phpunit etc etc.. hopefully i can skip it's usage.
somehow it didn't unsub me.. did it again. lol.. uh, i clicked unsub before posting that comment
Don't feed the troll, please.
@cebe created a repository cebe/assetfree-yii2 that makes possible to remove FXP plugin. The trick is in the provide
option of composer.json
that makes Composer think, that packages are already provided. Installation will not fail without FXP, but you will need to manage bower/npm packages manually if you need them.
You mean running npm install
and/or bower install
manually?
via composer
2016-04-07 0:26 GMT+03:00 Alexander Nestorov [email protected]:
You mean running npm install and/or bower install manually?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/yiisoft/yii2/issues/8688#issuecomment-206577512
@alexandernst yepp, exactly.
What about running both commands from the composer.json
? Maybe in the post-install-cmd
?
@alexandernst you can do that but then you could just use the asset plugin as it is much better integrated with composer than a post-install-cmd
...
Vote to remove bower-asset\xxxx
from composer.json
require but still keep fxp plugin :D. All frontend required should be refactoring into another extension.
In my country, we need more than a year just to campaign what composer is. When we held a workshop or seminar, we need a half of session just to introducing composer
. If yii2 also introduce another package manager, i dont know what time we need.
We are working on switching from FXP. Most probably we will leave bower-asset\xxxx
dependencies in composer.json
but they will work without FXP or Node.js installation
P.S.: Oh wow :open_mouth: Does Indonesia really work on modern projects without composer?
@silverfire: no.. indonesian yiier still user composer.. but @mdmunir only say that introduce composer for newbie, it take much time :)
@mdmunir @hscstudio Why do you think the coding abilities of Indonesian newbies should influence in any way this framework?
@mdmunir fxp plugin is proven to be completely buggy and yii devs already said they will change it.
@alexandernst it works well in most cases and we are not going to kick it completely. The problem right now is that we do not offer a chance to skip it if you dont want or need it.
@alexandernst
I am not against with removing fxp, but i am against with introducing new package manager. I am ok with all tecnology that you introduce, but how about new comer? Not only in my country, but all of place.
Do you active in your local forum? If new comer isnot your problem, then i cannot speck more.
For start, removing asset dependency from composer is good enough. Actually, fxp is out of yii. You can install or uninstall is up to you.
@cebe No, it doesn't work well in most cases. It doesn't work even regularly well in most cases. I proved that in the first post of this issues and you agreed with me right after. Quite some people backed me after that. Keeping FXP at this point sounds, after all the problem it has caused (remember all the times yii couldn't be installed because of random bugs in FXP that were patched a week later?), a very, very, very bad idea.
@mdmunir As @cebe already said, FXP can't be removed, yii is pretty much tied to it. And as I said, FXP is extremely bad at handling but the most basic configurations. And no, newbies are not my problem, neither are yours. A newbie is just a person that is starting from 0 with something. Starting from 0 with FXP or starting from 0 with any other similar tool should be the same.
Fxp is work ok for me in most/all of case.
@alexandernst
Where your problem get from? From yii, from extension or from your root composer?
@mdmunir Take your time and read the entire issue conversation. I don't have any problems with fxp as I entirely bypassed it the third time it gave me headaches.
@SilverFire
We are working on switching from FXP. Most probably we will leave bower-assetxxxx dependencies in composer.json but they will work without FXP or Node.js installation
Any details how this gonna work?
remember all the times yii couldn't be installed because of random bugs in FXP that were patched a week later?
even though most of what you say is true, most of these issues come from composer chainging its api and breaking things. Remember that composer has been alpha status until a few weeks ago.
specific combinations of fxp and composer work, other do not, thats mostly about composers api stability.
Any details how this gonna work?
not yet. What will be possible is to have yii install without the plugin if you want. We do not have a concept for that yet, also not for how the framework itself may be split up or not.
To anyone who is interested in improving speed of the fxp/composer-asset-plugin
, please give this a read and feedback, if possible.
https://github.com/francoispluchino/composer-asset-plugin/issues/93#issuecomment-222528971
The problem is not the speed (well, actually that too), but the problems that fxp is causing. The way it parses tags and versions is completely rubbish and even a single package can break it (try giving fxp "1.0.0-beta.5”).
El 30 may 2016, a las 23:03, Tobias Munk [email protected] escribió:
To anyone who is interested in improving speed of the fxp/composer-asset-plugin, please give this a read and feedback, if possible.
francoispluchino/composer-asset-plugin#93 (comment) https://github.com/francoispluchino/composer-asset-plugin/issues/93#issuecomment-222528971
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/yiisoft/yii2/issues/8688#issuecomment-222554213, or mute the thread https://github.com/notifications/unsubscribe/AAFef0b8REFgRVIxSKsW_7lqZZraN33pks5qG1C9gaJpZM4E68WB.
try giving fxp "1.0.0-beta.5”
Of which package? What happens and what do you expect?
There's a pretty comprehensive documentation about version conversion, btw.
How is the name of the package relevant in any way? The question is that this is a valid NPM version tag, but is being parsed as invalid by FXP. What I’d expect is FXP to properly download the requested package, but what happens is that FXP quits unexpectedly.
El 30 may 2016, a las 23:59, Tobias Munk [email protected] escribió:
try giving fxp "1.0.0-beta.5”
Of which package? What happens and what do you expect?
There's a pretty comprehensive documentation about version conversion https://github.com/francoispluchino/composer-asset-plugin/blob/bae7a15712e9339a2439aeca3b7c5ecf65b087d1/Resources/doc/schema.md#verison-conversion, btw.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/yiisoft/yii2/issues/8688#issuecomment-222559291, or mute the thread https://github.com/notifications/unsubscribe/AAFef8fgTLx7fqOemHw1sViskvLRD8BUks5qG12xgaJpZM4E68WB.
The question is that this is a valid NPM version tag, but is being parsed as invalid by FXP.
The problem is that its and invalid version tag for composer. Have you open an issue or pr about it on FXP repo?
How is the name of the package relevant in any way?
I'd like to test it. This does not look like an unsolvable problem to me. Related issue https://github.com/francoispluchino/composer-asset-plugin/issues/211
I desisted after personally patching composer multiple times
I know this can be easily fixed, but this is just a simple case. I couldn't list all the (similar) bugs I have found so far. At this point, keep patching FXP instead of just replacing it seams like a cat&mouse game.
I couldn't list all the bugs I have found so far - _in our bower & npm projects_.
I think this kind of discussion leads nowhere.
The things fxp/composer-asset-plugin
is doing is just wonderful - but these are just my 2 cents. Case closed for me.
@schmunk42 you are the second person I know who actually likes Fxp, the first one being me. Now that composer api is stable, i don't expect to see lots of issues with fxp breaking suddenly.
And the discussion has devolved into 'fxp is bad' instead of alternatives. I think so far only @nsams has proposed something that is actually usable.
The fact that so many people complain shows a valid case for decoupling FXP.
When it works, it does what it is supposed to do and it does it great.
On the contrary:
There comes a time when you have to make a decision to put an end to this and make yourself less dependant on external forces, uselessly consuming hours and hours.
Last month I chose to disconnect FXP and to use NPM to retrieve the correct assets. I couldnt be happier since. The deploys are extremely fast again, composer has regained some of its functionality and no more errors or debugging. Finally back in control again.
- it does do it slow (and that is an understatement)
I am also annoyed by that, that's why I addressed that in my post yesterday.
If we can make fxp
use git ls-remote
, I'd almost bet it's faster than native bower.
- it breaks some of composers CLI functionality
Which ones? _I didn't read this thread line by line._ But I am very curios.
- it fails often. Mostly because of cache issues or misfits with composer itself, it give you headaches and frustration. Personally, Ive lost tens of hours over the years because of debugging issues because of the Composer + FXP.
That's true - I'd say I've lost even more :D ... decoupling in general is nice.
But I strongly believe this is just left pocket, right pocket. A npm & bower solution simply has other short-comings when it comes to managing assets.
Just give it a try :) Remove fxp plugin and add these lines to composer.json:
"repositories": [
{
"type": "composer",
"url": "https://asset-packagist.org"
}
]
It doesn't slow down composer and doesn't create other problems.
I am also annoyed by that, that's why I addressed that in my post yesterday.
I want my deploys to be a fast as possible, especially for having fast feedback about potential errors. FXP is a bottleneck on many aspects. It forces me to get some coffee in the mean time (and then to come back noticing unnecessary errors). NPM on the contrary, enables parallel execution. I also dont need dependency comparison like with bower.
So finally, after 2 years, I am finally back to a deploy pipeline within <30 seconds (including assets compiling, parallel testing & deploying, etc.)
Which ones? I didn't read this thread line by line. But I am very curios.
Like this one:
https://github.com/francoispluchino/composer-asset-plugin/issues/94#issuecomment-73676397
That's true - I'd say I've lost even more
I dont think we want to know the real number of hours ;)
I also noticed that the "just-one-package" updates are kinda broken, but never had the time to investigate. I think it would also be improved by no-api
. Looks like I need to sit down an implement this 😓
So finally, after 2 years, I am finally back to a deploy pipeline within <30 seconds (including assets compiling, parallel testing & deploying, etc.)
Would love to take a peek at the setup... :)
You can see the issue francoispluchino/composer-asset-plugin#226.
I think we should switch to using https://asset-packagist.org/.
asset-packagist
is available in case of emergency we can get another server ready in hours.Does usage of asset-packagist
require any adjusments at composer.json
comparing with the asset-plugin
?
if we use it, we should provide it under a domain controled by yii core team to provide a reliable service.
Does usage of asset-packagist require any adjusments at composer.json comparing with the asset-plugin?
it requires adding a repository entry, which would be part of the application template, so no extra command for installing the global plugin will be needed.
it requires adding a repository entry
Does it affect only the root (project) composer config, or each package should provide it?
In particular: can we compose composer.json
at yii2-bootstrap
in the way end-up project may use either asset-packagist
or asset-plugin
?
if we use it, we should provide it under a domain controled by yii core team to provide a reliable service.
I am unsure. Such service can be used by any PHP project, not only powered by Yii.
If there are people started it and ready to maintain it - we should rely on them.
Still it may make sence to have an own back-up server, just in case, until the current asset-packagist
become popular and wide-spred enough.
Does it affect only the root (project) composer config, or each package should provide it?
In particular: can we compose composer.json at yii2-bootstrap in the way end-up project may use either asset-packagist or asset-plugin?
I think so. repositories
is root-only, so you have to add it to your root project, not yii2-bootstrap. Or you can install composer-asset-plugin and use that.
I don't know what happens if you have both though. My guess is that composer will use both and you'll have duplicates of packages (it'll make composer update slower), so you'll have to uninstall composer-asset-plugin.
I am unsure. Such service can be used by any PHP project, not only powered by Yii.
If there are people started it and readu to maintain it - we should rely on them.
+1. It's better to have one well-maintained service backed by multiple frameworks than a yii-specific fork.
The server use always the fxp/composer-asset-plugin
. So, it would be interesting to incorporate a proxy server in the plugin, which increases the performance and compatibility is kept. I'm not against this idea if you are interested. An implementation start has been made (see fxpio/composer-asset-plugin#215).
+1. It's better to have one well-maintained service backed by multiple frameworks than a yii-specific fork.
Sure, I am not saying that it should be limited to yii, my point was that we should have some control over the service so we can ensure it stays up.
@francoispluchino that's fine but there's still a big con is this case. People tend to forgot to install fxp plugin or install it locally despite it being documented in installation guide. Moreover, after trying with incorrectly installed fxp, vendor dir becomes messed up and proper installation doesn't change much. It still errors.
This proxy-repository solves the issue above.
@cebe currently the server is managed by @SilverFire colleague. I guess we may try asking for emergency access credentials.
In particular: can we compose composer.json at yii2-bootstrap in the way end-up project may use either asset-packagist or asset-plugin?
@klimov-paul yes, the composer.json of bootstrap extension just requires the package. If it is provided by a repository that is configured in root composer.json or by the asset plugin is up to the user. Actually you currently can just use the asset repository instead of the plugin already, the question here is basically what will be the recommended way by yii, i.e. the default used by basic and advanced template.
If this matter affects only the project root composer config, then the final choice can be left to the developer. He can choose, which way he want to get asset packages.
This choice should be documented.
Still, by default our porject templates should use asset-packagist
as it does not require any extra installations and thus unlikely to make troubles even for the newcomers.
Note: which way is better in real life is matter to be found out in the future. Thus both posibilities should be documented.
Agree. Documenting all the possibilities is a good idea.
@cebe I've asked @hiqsol. He said that @SilverFire already has access to the server and more people could be added.
Hmm... seems it's actually not breaking backwards compatibility at all. How about doing it in 2.0 instead of 2.1?
we can add the docs for 2.0.x too but make the switch of the default in 2.1. This way the asset-packagist gets more testing before becoming the deault.
seems it's actually not breaking backwards compatibility at all.
except asset-packagist
download package into vendor/bower-asset
instead of vendor/bower
.
OOT.
IMO, it seems to be ok if we add ability to yii2-composer
to add root autoload psr-4 into alias path.
So we can solve this via
"autoload": {
"psr-4": { "bower\\": "vendor/bower-asset" }
}
vendor/bower-asset instead of vendor/bower
this is one of the worst decision made when implementing the asset bundles, yii uses another place than the default of the asset plugin. We should try to get rid of this in 2.1.
I think we should switch to using https://asset-packagist.org/.
Is it scalable? How asset-packagist handles GitHub rate limits or private repositories?
No. It's not managed.
@rob006 GitHub removed usage limits a while ago. Private repositories aren't supported.
@samdark the GitHub API has rate limits: https://developer.github.com/v3/#rate-limiting, and composer-asset-plugin uses the API by default. I don't know if and how this affects asset-packagist.
@hiqsol any comments on rate limit?
Those limits just don't concern users of asset-packagist cause fxp is run
under the hood on the server independently from requests made by user's
composer.
And as soon as I understand fxp switches to cloning repository when it
meets the github limits.
12 Авг 2016 г. 15:41 пользователь "Alexander Makarov" <
[email protected]> написал:
@hiqsol https://github.com/hiqsol any comments on rate limit?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yiisoft/yii2/issues/8688#issuecomment-239435617, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ALRdTXHeQrvZvHnUhQ1bBnuj8Ny0dsL1ks5qfGoCgaJpZM4E68WB
.
I tried out asset-packagist. If you have composer-asset-plugin installed locally, it seems composer prefers the packages from asset-packagist (I'm not sure this is guaranteed), but composer-asset-plugin still runs and has to download all the info from github, slowing down composer update significantly.
So we need an option to disable composer-asset-plugin in the root project's composer.json.
If a package is not in asset-packagist, composer will fall back to the package provided by composer-asset-plugin.
Composer-asset-plugin also does some post-processing on install (removing ignored files mainly), I don't think the lack of this should be an issue. It seems both asset-packagist and composer-asset-plugin put the same zipball into composer.lock.
So we need an option to disable composer-asset-plugin in the root project's composer.json.
If composer-asset-plugin
is a only plugin installed, you can use --no-plugins
composer option:
composer update --no-plugins
I know, but that's not ideal. I'm not the only one working on the project, and even I'll forget to use --no-plugins
. I'd like to make sure the plugin is disabled in composer.json.
--no-plugins
will skip the yii2-composer
plugin also. So you have to choose one option, you can not use asset-packagist and the asset plugin at the same time.
That's bad, since the plugin has to be installed globally - so when you're switching between projects, you have to keep installing and uninstalling it constantly.
@nkovacs Why you uninstall/install for each project ??? It's not necessary.
See my comment above:
https://github.com/yiisoft/yii2/issues/8688#issuecomment-239797923
If a project is using asset-packagist, c-a-p is not needed and it slows down composer update significantly (because it still needs to fetch the info from github and generate composer packages for bower packages).
If a project is not using asset-packagist, I obviously need c-a-p.
I'm also wary of having both enabled. I've seen weird behavior from composer before when I had duplicate packages caused by a satis used as a cache. It also caused massive memory and cpu time increases (probably exponential) in the dependency resolution process.
IMO, this project should not be used, why:
I'm not against the idea, but in the state, this solution is not viable for mass production.
About the negatives points of the Composer plugin, they can be resolved quickly, such, we could activate the plugin only when a parameter is present in the Composer root or a specific dependency is present. And regarding the speed of the plugin, a server caching is being implemented with the flexibility to choose a compatible server.
And yes, I am the developer of this plugin, and I also developed a server solution before to develop a local solution with the VCS Repositories (see why this plugin).
Personally, I think it makes more wiser to focus the efforts of all the developers involved in the same project, instead of being each in his corner trying to solve the same problem. Especially as I remain open to suggestions and improvements. Those who have ears should hear! :-)
@francoispluchino I think the most critical downsides of the plugin are:
@samdark
For 1. We can add a dependency in each project to detecting the presence or not of the global plugin and throw an exception with a clear message if the plugin is not installed (it could replace the option of the proposal 'require-global' in Composer). I would have to test if it is possible to use this same dependency to enable composer-asset-plugin.
With the asset-packagist, we must add manually the repository in each Composer root package (with the command line or edit the json with a text editor), so the constrainst is identical. But with my proposal, beginners will immediately understand the problem.
For 2. Read my previous comment (server proxy)
the server has the limit with 5000 request/hour for Github API, and this limit does not be increase without token for each user, and it's not possible without a plugin of Composer
I can not tell in which way asset-packagist
is implented, but in general such problem is easy to bypass.
First of all, server may cache packages with stable version internally and return them from its own cache inistead of call bower for them.
But most obvious walk around is creating some dummy GitHub account named 'asset-packagist', fetch the access token for it and use at server.
With the asset-packagist, we must add manually the repository in each Composer root package (with the command line or edit the json with a text editor)
Add it into the templates and the end user doesn't have to care. It also doesn't globally pollute your composer setup. There were compatibility issues between the plugin and composer that completely broke composer unless you ran it with --no-plugins (it's slightly better now that composer is stable).
Related issue about global installation: https://github.com/composer/composer/issues/3607
@francoispluchino
the server uses internally the fxp/composer-asset-plugin and we do not know which version is used (difficult debugging)
sorry, didn't get the possible problem, could you give an example
all problems of the plugin are in this project (ex. case sensitivity of Bower, bugs of conversion)
actually I do lowercasing of package names - it seems to work
I don't know about conversion bugs, could you give me an example
private repositories does not work
packagist.org doesn't support private repositories too :) there are other solutions for that
the server has the limit with 5000 request/hour for Github API, and this limit does not be increase without token for each user, and it's not possible without a plugin of Composer
answered earlier in this same thread - this limit doesn't bother user.
and a github token is actually used on the server as @klimov-paul suggested
if the server is down, another server cannot be used, and all developers are blocked
what is the SLA of this service?
this is the real actual problem, but it could be solved by providing enough redundancy
there is a time lag between updates packages and those of the server
yes, that's why there's web interface to fetch packages when you need it
it's not an ideal and not a final solution though
custom directory of the assets installation does not work
it's ok for yii because proper aliases for bower and npm can be used
when more flexibility needed other non-global plugin can be used, eg. oomphinc/composer-installers-extender
disable or replace the deleting of the ignore files for Bower and NPM does not work
override of the main files for Bower does not work
you cannot disable the search of assets in Composer
override the config of vcs repository works correctly?
Usage with multiple versions of the same dependency works correctly?
Git/SVN/etc sources can be used?
sorry, didn't understand the problem
I'm not against the idea, but in the state, this solution is not viable for mass production.
I'm not against the plugin too :)
Actually the plugin does much more then asset-packagist.
But I think asset-packagist does enough to be used for Yii projects.
And it solves the main problems mentioned by @samdark that bothered me personally very much: speed and global installation (not the cryptic error messages but the problems it gives with CI services integration).
@samdark
- Solver performance. This one can be solved which is great.
Is solver performance really a problem? I just ran an update
with a large number of asset packages, see https://gist.github.com/schmunk42/fcce54e458f612cfde3804a83a2655c8 - and dependency resolution just takes about 2 seconds.
[260.0MB/107.91s] Resolving dependencies through SAT
[261.5MB/110.19s] Dependency resolution completed in 2.277 seconds
[261.6MB/110.27s] Analyzed 4021 packages to resolve dependencies
[261.6MB/110.27s] Analyzed 85785 rules to resolve dependencies
[263.3MB/110.30s] Resolving dependencies through SAT
[263.3MB/110.30s] Dependency resolution completed in 0.001 seconds
PS/btw: Is anyone using c-a-p without GitHub API requests, but with git-repos?
Bower has more 80k packages and NPM, even more. So even if you can make only one request for each package, your proposal is not viable in the long term. in addition, using multiple accounts to work around the limitations, you violate the terms of use of Github, and the staff of Github can block your IP at any time. Of course, you can always work around this problem (proxy, vpn, etc). But do a bypass, of a bypass, of another bypass... It's not a good idea, IMO.
@hiqsol
Regarding your different points:
sorry, didn't get the possible problem, could you give an example
See the list of fixed bugs in the last version, mainly caused by changes in Composer. So the end user will see no exception, but simply that his package is not found, or at best, that the repository returns a 500 error. This is a big blur, and only people who maintain the server can correct. In addition, you must wait until the correction is made in composer-asset-plugin, to be integrated into the server, and deploy. And in the meantime, everyone does not know what happens outside that nothing works.
actually I do lowercasing of package names - it seems to work
I don't know about conversion bugs, could you give me an example
See fxpio/composer-asset-plugin#70
packagist.org doesn't support private repositories too :) there are other solutions for that
Packagist does not support the private repository but Composer support the private repository via the VCS Repository. The plugin use the native support of vcs repository. So, without plugin, private repository is not supported for assets.
answered earlier in this same thread - this limit doesn't bother user.
and a github token is actually used on the server as @klimov-paul suggested
Or not. See the top of this comment for @klimov-paul.
this is the real actual problem, but it could be solved by providing enough redundancy
Yes for the technic, and a contract for your end users. especially for business, but is case it's the trust a thecenterprise to a new provider. it's not a really problem, but it is good to indicate.
yes, that's why there's web interface to fetch packages when you need it
it's not an ideal and not a final solution though
It's not that this is not ideal, it's just that it's not scalable, alas.
it's ok for yii because proper aliases for bower and npm can be used
when more flexibility needed other non-global plugin can be used, eg. oomphinc/composer-installers-extender
You don't response to the problem.
sorry, didn't understand the problem
It's all features of Composer that you can use with the plugin, but not with the assrt-packagist.
I'm not against the plugin too :)
Ouf! I was afraid! :-)
Actually the plugin does much more then asset-packagist.
But I think asset-packagist does enough to be used for Yii projects.
And it solves the main problems mentioned by @samdark that bothered me personally very much: speed and global installation (not the cryptic error messages but the problems it gives with CI services integration).
This solution solve 2 problems, yes and no, but add news problems, soved by the plugin. For performance, we can have the best the 2 solutions, and regarding the global installation, yes it's a problem, but the lead developers of Composer wan't choice a solution yet.
@francoispluchino
You don't response to the problem.
That's not a problem. That's a feature of the plugin and is not going to be implemented due to reasons I've mentioned. Similar situation with other plugin's features.
The real potential problem is the availability of the service and it can be solved.
Long time no see :)
@francoispluchino thank you for contribution in our discussion and thank you for the work you're doing for all the PHP community with composer-asset-plugin
. I really appreciate it.
Turning back to to the question at hand, it's important to say that this issue is one of the most disputable in the Yii2 issues tracker. It means at least that the problem bothers a lot and should be solved somehow. Currently we have two major reasons to switch from composer-asset-plugin to something different:
composer update
on all projects under the user because of previous pointFor now asset-packagist.org is the only solution, that solves these problems. @francoispluchino have posted really interesting and important thought, so let me share my opinion on some of them.
- if the server is down, another server cannot be used, and all developers are blocked
- what is the SLA of this service?
What if bower.io, packagist.org or gitbug.com is down? :) Unlikely, asset-packagist
is a new point of failure, but as @hiqsol said, this is solvable using balancing server(s) and multiple backends.
So the end user will see no exception, but simply that his package is not found, or at best, that the repository returns a 500 error.
This is a real problem. Currently asset-packagist has pre-loaded TOP1000 packages from Bower and NPM. When you run composer update
and see that the package was not found, you should visit asset-packagist.org and fetch it manually by searching it.
At the same time, this is the reason why this method works fast. You fetch packages.json
and you know a list of packages that repository can provide for the installation. That's why we have no way to say to asset-packagist.org that we need a brand-new bower-asset/super-js-magic
so service could fetch it for us.
- there is a time lag between updates packages and those of the server
And as a result of a previous point, the lag is infinite, unless you press Fetch
button on the package details page of asset-packagist.org. Or unless we fetch it with cron.
So these are the most annoying problems for now. What about others? Let's remember 80/20 rule and say, that 80% of users need 20% of features. Asset-packagist surely implements these 20% of composer-asset-plugin features. If you want to use private sorces as well as other features @francoispluchino have listed - feel free to delete asset-packagist from your project's composer.json
and use composer-asset-plugin
since it is fully backward-compatible.
But if you're lucky and you are in those 80% who uses only basic thing, you get speed and no global plugins. Easy.
Thoughts?
As I already said here
there is no absolute solution for this matter.
The only way to find out, which solution is the best is long-time testing.
At the present state it sounds too risky to change default flow from composer-asset-plugin
to asset-packagist
. It is hard to predict at which stage it may cause problem.
We should start from documenting possibility of usage asset-packagist
as an alternative of composer-asset-plugin
, while application templates should remain intact.
Once these docs are spead enough and enough people will start to use asset-packagist
, we will be able to see, if it is safe to switch to it by default.
Only time can show...
@SilverFire I have no new thoughts, everything has been said, I think. just in response to your question, Github.com has a SLA, and that Bower and Composer can work without their registries (Bower.io and Packagist.org). more specifically, you can implement your own server Packagist or its lite version: Satis.
Otherwise, I am totally agree with you on the rule 80/20%, and I see only one problem currently : the installation in global mode for the plugin (and this is not the lack of insistence and proposal on the Composer project that blocks this problem, but for them, this kind of feature is not urgent because not used much - 80/20).
Concerning speed, I have already mentioned, we can implement (work in progress) a proxy/cache server to obtain the speed of a server Packagist/Satis, without the constraints of asset-packagist and while keeping all the features of the plugin. We could imagine in this case that the asset-packagist becomes the 'main/official' proxy server. IMO, it's the best compromise.
In conclusion, only the installation of plugin in global mode is problematic. Now, if this single issue is really too embarrassing and negative points of asset-packagist is bearable for you, indeed switch, I will not be shocked, quite the contrary, I still consider to choose the best compromise. if my work has been useful some time to your project, it's already cool!
@klimov-paul Indeed, It's more careful.
OK. As it was noted, switching immediately isn't safe. So:
@SilverFire
- Necessity to install it globally
- Lower performance of composer update on all projects under the user because of previous point
I agree that installing the plugin globally, or better missing a global install, is an issue which comes up very often.
But, as said before, it is absolutely not the reason for slow performance. The c-a-p is slow, when it uses GitHub API requests (I already linked to a wiki article describing the details several weeks ago) for gathering information about asset packages. Can we agree on that?
Some benchmarks of composer update.
First, all the bower packages that are required:
"bower-asset/chai": "~1.10"
in rootyiisoft/yii2-bootstrap 2.0.6 requires bower-asset/bootstrap 3.3.* | 3.2.* | 3.1.*
yiisoft/yii2 2.0.9 requires bower-asset/jquery 2.2.*@stable | 2.1.*@stable | 1.11.*@stable | 1.12.*@stable
yiisoft/yii2 2.0.9 requires bower-asset/jquery.inputmask ~3.2.2
"bower-asset/mocha": "~2.1"
in rootyiisoft/yii2 2.0.9 requires bower-asset/punycode 1.3.*
"bower-asset/sinon-chai": "~2.6"
in rootyiisoft/yii2-gii 2.0.5 requires bower-asset/typeahead.js 0.10.* | ~0.11.0
yiisoft/yii2 2.0.9 requires bower-asset/yii2-pjax ~2.0.1
Except for chai, mocha and sinon-chai, these are all just Yii's basic requirements.
asset-pattern-skip-version
is set to "(-build|-patch)"
.
Composer is 1.1.3
fxp/composer-asset-plugin is v1.2.1.
With plugin, using github api:
composer update -vvv 13,50s user 4,37s system 23% cpu 1:14,91 total
(without vendor directory, but with composer cache)composer update -vvv 11,30s user 1,06s system 22% cpu 54,273 total
(second run with vendor directory)composer update -vvv 11,21s user 1,00s system 23% cpu 52,430 total
(third run)composer update -vvv 24,56s user 4,12s system 6% cpu 7:05,64 total
(clean run with no vendor directory and no composer cache)With plugin, using git:
composer update -vvv 35,24s user 10,66s system 52% cpu 1:26,73 total
(first run without vendor directory, and probably uncached since I haven't used the github-no-api option before)composer update -vvv 12,48s user 1,54s system 39% cpu 35,704 total
(second run with vendor directory and cached repositories)composer update -vvv 12,08s user 1,79s system 44% cpu 30,961 total
(third run)composer update -vvv 41,03s user 10,28s system 22% cpu 3:53,05 total
(clean run with no vendor directory and no composer cache)Without plugin, using asset-packagist:
composer update -vvv 10,14s user 3,64s system 82% cpu 16,795 total
(first run without vendor directory, but with composer cache)composer update -vvv 10,18s user 1,10s system 94% cpu 11,994 total
(second run with vendor directory)composer update -vvv 10,22s user 0,98s system 94% cpu 11,821 total
(third run)composer update -vvv 16,11s user 3,19s system 11% cpu 2:52,83 total
(clean run with no vendor directory and no composer cache)So c-a-p is still 2.5-3x slower when using cached git repos, and even slower when adding a new package without a cached git repo.
First of all, thank you for your work and inputs. I am not against a specific solution for no reason, but I also want to give you some feedback about my findings.
So yes, AP (asset-packagist) is faster than c-a-p, but in which intervals are updates run on the server? Because this is the main reason IMHO, _why_ it is faster.
Just to note that here, having a WebHook or Service on GitHub, like Packagist does, won't help, I highly doubt that (npm/bower) developers would add another service to their repos.
How would you handle updates with 1.000s or 10.000s of packages?
When using c-a-p _all asset repos_ are updated and checked, this is what takes "so long". If there would be an option to skip that, I think we won't notice a difference.
I gave asset-packagist a try, I mainly ran composer update --no-plugins
, just to disable c-a-p.
I tested with this composer.json file.
What's rather problematic to me is, that I need to manually add packages via the web interface, if AP can't find it.
The user gets an error - which is pretty similar to a missing global plugin installation btw - I don't know if this could be solved by automatically querying for a package if it is requested by an installation or update process.
[111.8MB/2.63s] Reading /Users/tobias/.composer/cache/repo/https---asset-packagist.org/provider-bower-asset$angular.json from cache
[596.4MB/8.93s] Resolving dependencies through SAT
[596.5MB/8.93s] Dependency resolution completed in 0.001 seconds
[596.6MB/8.93s] Your requirements could not be resolved to an installable set of packages.
[596.6MB/8.93s]
Problem 1
- cinghie/yii2-cookie-consent v1.4.3 requires bower-asset/cookieconsent2 * -> no matching package found.
- cinghie/yii2-cookie-consent v1.4.2 requires bower-asset/cookieconsent2 * -> no matching package found.
- cinghie/yii2-cookie-consent v1.4.1 requires bower-asset/cookieconsent2 * -> no matching package found.
- cinghie/yii2-cookie-consent v1.4.3 requires bower-asset/cookieconsent2 * -> no matching package found.
- Installation request for cinghie/yii2-cookie-consent ^1.4 -> satisfiable by cinghie/yii2-cookie-consent[v1.4.1, v1.4.2, v1.4.3].
Potential causes:
- A typo in the package name
- The package is not available in a stable-enough version according to your minimum-stability setting
see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.
Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.
[176.8MB/9.50s] Memory usage: 176.85MB (peak: 596.87MB), time: 9.5s
As said above ... you need to know, that you have to go to https://asset-packagist.org, add the package there, and re-run the update, so I added bower-asset/cookieconsent2
.
[111.8MB/1.66s] Reading /Users/tobias/.composer/cache/repo/https---asset-packagist.org/provider-bower-asset$angular.json from cache
[596.6MB/7.89s] Resolving dependencies through SAT
[596.6MB/7.89s] Dependency resolution completed in 0.001 seconds
[596.8MB/7.89s] Your requirements could not be resolved to an installable set of packages.
[596.8MB/7.89s]
Problem 1
- romka-chev/yii2-swiper v2.0.2 requires bower-asset/swiper >=v3.0.4 -> no matching package found.
- romka-chev/yii2-swiper v2.0.1 requires bower-asset/swiper >=v3.0.4 -> no matching package found.
- romka-chev/yii2-swiper v2.0.0 requires bower-asset/swiper >=v3.0.4 -> no matching package found.
- romka-chev/yii2-swiper v2.0.2 requires bower-asset/swiper >=v3.0.4 -> no matching package found.
- Installation request for romka-chev/yii2-swiper ^2.0 -> satisfiable by romka-chev/yii2-swiper[v2.0.0, v2.0.1, v2.0.2].
Potential causes:
- A typo in the package name
- The package is not available in a stable-enough version according to your minimum-stability setting
see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.
Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.
[176.9MB/8.49s] Memory usage: 176.87MB (peak: 597.01MB), time: 8.49s
Added bower-asset/swiper
[111.9MB/2.54s] Reading /Users/tobias/.composer/cache/repo/https---asset-packagist.org/provider-bower-asset$angular.json from cache
[596.8MB/8.84s] Resolving dependencies through SAT
[596.9MB/8.84s] Dependency resolution completed in 0.001 seconds
[597.0MB/8.84s] Your requirements could not be resolved to an installable set of packages.
[597.0MB/8.84s]
Problem 1
- dmstr/yii2-cms-metapackage 6.0.0-beta1 requires dmstr/yii2-prototype-module ^0.4.0 -> satisfiable by dmstr/yii2-prototype-module[0.4.0, 0.4.0-beta1].
- dmstr/yii2-cms-metapackage 6.0.0-alpha1 requires dmstr/yii2-prototype-module ^0.4.0-beta1 -> satisfiable by dmstr/yii2-prototype-module[0.4.0, 0.4.0-beta1].
- dmstr/yii2-cms-metapackage 6.0.0-beta1 requires dmstr/yii2-prototype-module ^0.4.0 -> satisfiable by dmstr/yii2-prototype-module[0.4.0, 0.4.0-beta1].
- dmstr/yii2-prototype-module 0.4.0 requires beowulfenator/yii2-json-editor ^1.1.2 -> satisfiable by beowulfenator/yii2-json-editor[1.1.3, 1.1.2].
- dmstr/yii2-prototype-module 0.4.0-beta1 requires beowulfenator/yii2-json-editor ^1.1.2 -> satisfiable by beowulfenator/yii2-json-editor[1.1.3, 1.1.2].
- dmstr/yii2-prototype-module 0.4.0 requires beowulfenator/yii2-json-editor ^1.1.2 -> satisfiable by beowulfenator/yii2-json-editor[1.1.3, 1.1.2].
- beowulfenator/yii2-json-editor 1.1.3 requires bower-asset/json-editor 0.* -> no matching package found.
- beowulfenator/yii2-json-editor 1.1.2 requires bower-asset/json-editor 0.* -> no matching package found.
- beowulfenator/yii2-json-editor 1.1.3 requires bower-asset/json-editor 0.* -> no matching package found.
- Installation request for dmstr/yii2-cms-metapackage ^6.0.0-alpha1 -> satisfiable by dmstr/yii2-cms-metapackage[6.0.0-alpha1, 6.0.0-beta1].
Potential causes:
- A typo in the package name
- The package is not available in a stable-enough version according to your minimum-stability setting
see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.
Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.
[177.0MB/9.43s] Memory usage: 176.97MB (peak: 597.28MB), time: 9.43s
A bit harder to find in the error message :) ... added bower-asset/json-editor
.
You also need to wait about 30 seconds until the package gets listed, this may be problematic with more users (GitHub API limit)
From here on I was able to install/update my project.
On to the benchmarks...
I was using only c-a-p only with no-api
and always with filled caches.
vendor
beforehand)composer update -vvv --profile
[663.3MB/65.09s] Resolving dependencies through SAT
[665.3MB/72.65s] Dependency resolution completed in 7.567 seconds
[665.4MB/72.75s] Analyzed 8387 packages to resolve dependencies
[665.4MB/72.75s] Analyzed 320268 rules to resolve dependencies
...
[187.3MB/121.33s] Reading ./composer.lock
[187.9MB/121.34s] Generating autoload files
[188.5MB/121.56s] Memory usage: 188.5MB (peak: 668.05MB), time: 121.56s
[663.4MB/70.31s] Resolving dependencies through SAT
[665.4MB/77.83s] Dependency resolution completed in 7.527 seconds
[665.5MB/77.93s] Analyzed 8387 packages to resolve dependencies
[665.5MB/77.93s] Analyzed 320268 rules to resolve dependencies
...
[187.4MB/124.09s] Reading ./composer.lock
[187.9MB/124.10s] Generating autoload files
[188.5MB/124.33s] Memory usage: 188.5MB (peak: 668.17MB), time: 124.33s
vendor
)[611.7MB/62.39s] Resolving dependencies through SAT
[613.5MB/63.92s] Dependency resolution completed in 1.523 seconds
[613.6MB/63.94s] Analyzed 7746 packages to resolve dependencies
[613.6MB/63.94s] Analyzed 286557 rules to resolve dependencies
[613.6MB/63.94s] Nothing to install or update
[615.1MB/63.96s] Resolving dependencies through SAT
[615.2MB/63.96s] Dependency resolution completed in 0.001 seconds
[183.2MB/64.60s] Reading /Users/tobias/hrzg/github.com/schmunk42/p5-kitchen-sink/vendor/composer/installed.json
[184.2MB/64.61s] Executing command (/Users/tobias/hrzg/github.com/schmunk42/p5-kitchen-sink/vendor/funson86/yii2-blog): git log -n1 --pretty=%ct '657458955e9b6bbbb7e4e584752d67089b675a49'
[184.5MB/64.63s] Reading ./composer.lock
[185.1MB/64.64s] Generating autoload files
[185.9MB/64.86s] Memory usage: 185.88MB (peak: 615.53MB), time: 64.86s
vendor
beforehand)composer update -vvv --profile --no-plugins
[594.2MB/9.79s] Resolving dependencies through SAT
[595.9MB/16.18s] Dependency resolution completed in 6.393 seconds
[596.0MB/16.26s] Analyzed 7250 packages to resolve dependencies
[596.0MB/16.26s] Analyzed 279352 rules to resolve dependencies
...
[178.4MB/65.83s] Reading ./composer.lock
[178.9MB/65.84s] Generating autoload files
[179.4MB/66.06s] Memory usage: 179.43MB (peak: 598.13MB), time: 66.06s
[594.5MB/10.72s] Resolving dependencies through SAT
[596.2MB/17.58s] Dependency resolution completed in 6.865 seconds
[596.3MB/17.66s] Analyzed 7250 packages to resolve dependencies
[596.3MB/17.66s] Analyzed 279352 rules to resolve dependencies
...
[178.5MB/67.37s] Reading ./composer.lock
[179.0MB/67.38s] Generating autoload files
[179.5MB/67.62s] Memory usage: 179.47MB (peak: 598.31MB), time: 67.62s
vendor
)[598.8MB/8.92s] Resolving dependencies through SAT
[600.6MB/10.54s] Dependency resolution completed in 1.618 seconds
[600.6MB/10.56s] Analyzed 7378 packages to resolve dependencies
[600.6MB/10.56s] Analyzed 282084 rules to resolve dependencies
[600.6MB/10.56s] Nothing to install or update
[602.2MB/10.59s] Resolving dependencies through SAT
[602.2MB/10.59s] Dependency resolution completed in 0.002 seconds
[177.2MB/11.22s] Reading /Users/tobias/hrzg/github.com/schmunk42/p5-kitchen-sink/vendor/composer/installed.json
[178.2MB/11.23s] Executing command (/Users/tobias/hrzg/github.com/schmunk42/p5-kitchen-sink/vendor/funson86/yii2-blog): git log -n1 --pretty=%ct '657458955e9b6bbbb7e4e584752d67089b675a49'
[178.6MB/11.25s] Reading ./composer.lock
[179.1MB/11.26s] Generating autoload files
[179.8MB/11.48s] Memory usage: 179.8MB (peak: 602.53MB), time: 11.48s
I don't know if this could be solved by automatically querying for a package if it is requested by an installation or update process.
Unfortunately, no, since composer requests a list of all packages available, and then checks if the package is in that list.
@schmunk42 good idea, I have to see how I can skip the analysis , but it should be achievable. Maybe use an environment variable FXP_SKIP_ASSET=true
. Please create a new issue in the plugin repo for this.
Please create a new issue in the plugin repo for this.
there is already one: https://github.com/fxpio/composer-asset-plugin/issues/249
@cebe disabling the plugin is problematic, because the behavior will be identic that if the plugin is not installed, so Composer will throw an excpetion. On the other hand, skip the analytics of new versions on update will keep the asset packages but without requests. It's not the same behavior. So, a new issue must be created for this feature.
@francoispluchino Why would it throw an exception? Because it can't find bower-asset/jquery
for example? But if you have asset-packagist, it will find it there.
Yes it is the case if you add the asset-packagist repository. I just indicate the difference between disable the plugin and skip the assets.
It may be useful to return to the beginning with this issue.
Original problem, which asset-plugin
solves is lack of bower
client written on PHP.
The problem is the native bower
client requires too much dependencies to be installed at server, dependencies which usually do not need for a web server running PHP application.
It would be really nice to have some bower.phar
as a standalone bower client written on PHP, which will be able to process bower.json
files as a native bower client.
The best solution will be allowing to use native bower.json
inside the project instead of mixing it into composer.json
.
Each particular extension, which uses assets from bower/npm
should have bower.json
at its root and specify assets here instead of composer.json
.
Consider following algorithm:
1) Run composer install
on the project, no plugin, no asset-packagegist
, loading all dependencies
2) After composer install traverse over vendor
directory in search of bower.json
files by pattern vendor/*/*/bower.json
3) collect all bower requirements and compose them into the single bower.json
at the project root.
4) Now bower.json
amy be processed by native bower client or in other way, as well as it may be ignored at all.
This algorithm can be packed into yii2-composer.
Bower requirements are just the .json
files, thus they can be easily read and processed.
Although there could be a problem with the asset versions conflict between different extensions, I suppose it could be handled somehow.
The only crucial thing is to have a standalone PHP written bower client - bower.phar
.
I suppose this should not be difficult for @francoispluchino to implement after his work on the asset-plugin
.
P.S. of course, this is a solution for 2.1
version.
Bower requirements are just the .json files, thus they can be easily read and processed.
Although there could be a problem with the asset versions conflict between different extensions, I suppose it could be handled somehow.
Sounds not that easy to me, that if there's a conflict found in bower.json
files, "someone" has to decide which PHP packages might have to be changed to solve these conflicts. That's what the SAT solver of composer does and c-a-p is doing a great job by using this tool.
IMHO, PHP and asset-packages belong together, you need both for your application, despite the fact there's no common package manager.
There's bower PHP port: https://bowerphp.org/
@klimov-paul
Consider the following scenario:
yii2-ext-a
:1.*
and yii2-ext-b
:1.*
.yii2-ext-a
version 1.0 depends on awesomejs
:1.*
.yii2-ext-b
version 1.0 depends on awesomejs
:1.*
.yii2-ext-b
version 1.1 depends on awesomejs
:2.*
.1) Run composer install on the project, no plugin, no asset-packagegist, loading all dependencies
yii2-ext-a
:1.0
and yii2-ext-b
:1.1
are installed.
2) After composer install traverse over vendor directory in search of bower.json files by pattern vendor/_/_/bower.json
3) collect all bower requirements and compose them into the single bower.json at the project root.
4) Now bower.json amy be processed by native bower client or in other way, as well as it may be ignored at all.
The dependencies are merged, and bower is asked to install awesomejs
:1.*
and awesomejs
:2.*
. This is impossible.
Although there could be a problem with the asset versions conflict between different extensions, I suppose it could be handled somehow.
The only way to handle this is to inject the bower dependencies into composer's resolution algorithm so that it can figure out the solution: install yii2-ext-a
:1.0
and yii2-ext-b
:1.0
.
The only way to handle this is to inject the bower dependencies into composer's resolution algorithm so that it can figure out the solution: install yii2-ext-a:1.0 and yii2-ext-b:1.0.
Can not we just use Composer API inside yii2-composer
for this task?
That's why composer-asset-plugin has to be installed globally. If it isn't installed, the dependency resolution will fail (bower-asset/jquery won't be found, for example), so the plugin won't be installed either into the project.
PHP and asset-packages belong together, you need both for your application, despite the fact there's no common package manager.
It is a pity that composer core team does share your opinion. Our original hope was the composer will adopt the asset-plugin functionality into the core, and we will not have all these troubles. However, they have not considering those packages are not bound together.
The dependencies are merged, and bower is asked to install awesomejs:1.* and awesomejs:2.*. This is impossible.
Since composer somehow merges versions and its API is already exposed for yii2-composer
perhaps we can do the same thing.
Personally, I have never encountered the situation assets from different extensions have a comflict or even the situation, where same bower package is requested in 2 different Yii extensions used together at the same project. It is of course possible in theory, but this is not the main our concern.
Besides I propose to generate the output file, which then can be reviewed and reedited by developer, thus he can always make some adjustments into it.
One more thing to think about: I wonder how does the Ruby solve this puzzle?
Since composer somehow merges versions and its API is already exposed for yii2-composer perhaps we can do the same thing.
It does this by generating a huge dependency graph and then finding a solution that satisfies all dependencies. For that to work, you have to inject all the bower dependencies (and their dependencies etc.) into the graph before the dependency resolution. To do that, you have to install a plugin globally.
It also needs to parse all the versions of bower packages and convert all of them (and their dependencies and so on) to composer packages, which is why it's slow.
Once you've already decided to install yii2-ext-b
:1.1
, you can't go back and change that decision because you now know that this causes a bower conflict.
If it was possible to do this in a better way, composer-asset-plugin would be doing that.
Personally, I have never encountered the situation assets from different extensions have a comflict or even the situation, where same bower package is requested in 2 different Yii extensions used together at the same project.
What about jquery? What if one extension requests 2.*
, and the other requests 3.*
?
Besides I propose to generate the output file, which then can be reviewed and reedited by developer, thus he can always make some adjustments into it.
This isn't a good idea. The developer probably has no idea how the extensions use the asset.
One more thing to think about: I wonder how does the Ruby solve this puzzle?
Looks like there's a very similar solution (convert bower packages to gems): https://rails-assets.org/#/
But that's not the only option.
Anyway, maybe it would be better to use npm and something like browserify or webpack. That way you could use two different versions of an asset, the only drawback would be that you'd potentially have multiple versions of an npm package sent to the client.
@nkovacs Thanks for your comment, I wanted to write the same things (mostly).
@klimov-paul See https://gist.github.com/schmunk42/d5d192a5e727f4262507cffe84a650a4 for an example of a bit larger project ... first command is "composer show outdated", second is "composer why is jquery installed" - That's just an example of how complex things can become.
It does this by generating a huge dependency graph and then finding a solution that satisfies all dependencies. For that to work, you have to inject all the bower dependencies (and their dependencies etc.) into the graph before the dependency resolution.
No, no, no. You are mixing 2 different things here. With the approach I propose we do NOT need to process the depenency graph - it will be done by Bower client via its native functionality.
For example: consider we have a project which have following require section in composer.json
:
"require": {
"yiisoft/yii2": "~2.0.5",
"yiisoft/yii2-bootstrap": "~2.0.0",
},
After installation it will load 2 libraries in the vendor
each of which have its own bower.json
.
1)vendor\yiisoft\yii2\bower.json
contains:
"require": {
"jquery": "2.2.*@stable | 2.1.*@stable | 1.11.*@stable | 1.12.*@stable",
},
2)vendor\yiisoft\yii2-bootstrap\bower.json
contains:
"require": {
"bootstrap": "3.3.* | 3.2.* | 3.1.*"
},
Although bootstrap
Bower package may also specify its own dependency on jquery
- we have no need to process it. We just merge 2 bower.json
files into a single one, which will look like following:
"require": {
"jquery": "2.2.*@stable | 2.1.*@stable | 1.11.*@stable | 1.12.*@stable",
"bootstrap": "3.3.* | 3.2.* | 3.1.*"
},
If there is a conflict in jquery
package it will be found and resolved by Bower client once it processes the file we have created.
The only trouble will be if we add some another package in our project composer.json
, which will have either jquery
or bootstrap
inside its bower.json
:
"require": {
"jquery": ">2.2.0",
},
Only then we have to merge >2.2.0
with 2.2.*@stable | 2.1.*@stable | 1.11.*@stable | 1.12.*@stable
somehow.
We do NOT load a Bower packages in this flow and thus we do NOT care about thier dependencies - this will be done by Bower client and the end of the process.
The only trouble will be if we add some another package in our project composer.json, which will have either jquery or bootstrap inside its bower.json:
But what if that other package requires jquery
3.*
?
You can't merge that with "2.2.@stable | 2.1.@stable | 1.11.@stable | 1.12.@stable"
.
Yes, that is the problem, which should be resolved: we should check if version constraits can be merged and then - merge them, if they can not - we should trigger an error.
@klimov-paul It is an approach that I had considered, see fxpio/composer-asset-plugin#93, but this merge is complex and you break the compatibility.
It is an approach that I had considered
Perhaps, It is, at least, similar.
you break the compatibility
I am aware of that. That is why it is only an option for Yii 2.1.x but not fot 2.0.x.
It's not just about breaking compatibility. You're also losing functionality.
The only loose in functionality is that conflict in bower packages version will no longer break the composer installation. While the in oppisite - you can use the features Bower native client provides like processing via Grunt.
I just try to find the solution for the question posted at this issue titile: 'Switch from fxp assets plugin'.
Unless you can convince the Composer core team to build Bower/NPM client inside their product by default we do not have much options.
This question has already been treated by the Composer core team, and the answer is: No. So no chance for this solution.
I do know that.
Personally, I do not think myself to be wiser then Composer core team on this matter. I suppose if they consider bower/npm can not be processed with the composer packages at the same time - it must be some valid reason for this.
That is why I think that instead of struggeling against this restiction, we should find a walk around and process composer and bower dependencies as separated ones.
@klimov-paul The reason is quite justified: they would not add a compatibility layer for another dependency management tools, because, in early of Composer, the system of versions management was very different from SemVer 2.0 in the syntax of dependencies (this is less the case now), and the Composer packages are case insensitive. Also , they consider that this feature can be managed in a plugin, and it's the case with my plugin. You can add the fact that some NPM or Bower packages may require a "compilation" to build a dist. You'll have more details after the section 'Proposal' of fxpio/composer-asset-plugin#93.
In my approach to the v2.0 and therefore using Nodejs, I do not know if the approach is the one you want, but I want to use all the possibilities of NPM, Bower and Nodejs, using native bower.json and package.json files (always more details in this issue, and especially why the other approaches are not appropriate in my view, in this comment).
@klimov-paul
We do NOT load a Bower packages in this flow and thus we do NOT care about thier dependencies - this will be done by Bower client and the end of the process.
IMHO, that's a huge step in the wrong direction.
There's not only bower, but also npm. I recently had an issue with a package which dropped bower support. With c-a-p I was able to switch to npm.
I would even say that c-a-p's dependency resolution with composer is superior to native bower and npm.
but ... Coming back to https://github.com/yiisoft/yii2/issues/8688#issuecomment-246127281
It may be useful to return to the beginning with this issue.
Let's do that, I'd even vote for closing this issue and start with a fresh one.
Because, honestly, I don't even see anymore what's problematic right now. Issues of the original post are resolved, performance has been vastly improved, ... Yes, there's still the global installation - but compared to all other solutions proposed in this thread this "problem" looks minimal to me.
I think c-a-p and Yii 2.0 are still having the best approach for asset handling. It's the best concept I've seen so far, across all frameworks and languages - and yes it's hard to make it work 100%, but there's no free lunch :)
@klimov-paul
You could have 2 packages:
A
that have two branches: 2.0
that require twitter bootstrap 3.* and 2.1
that require twitter bootstrap 4.*B
that works only with twitter bootstrap 3.*If in composer.json
you have dependencies like:
"A": "2.*",
"B": "1.*"
composer will install [email protected]
and [email protected]
which are incompatible. You can avoid this if you consider assets dependencies on composer update. This functionality is lost if you separate assets and composer dependencies.
Documentation issue created #13064
I am closing this issue, as complete.
Most helpful comment
https://asset-packagist.org/