ember-cli is slow with > 100,000 files in bower_components
Slowest Trees | Total
----------------------------------------------+---------------------
SourcemapConcat | 2559ms
SourcemapConcat | 2486ms
SourcemapConcat | 2277ms
SourcemapConcat | 2166ms
SassCompiler | 1953ms
Funnel: App JS Files | 814ms
Slowest Trees (cumulative) | Total (avg)
----------------------------------------------+---------------------
SourcemapConcat (8) | 9517ms (1189 ms)
SassCompiler (1) | 1953ms
Funnel: App JS Files (1) | 814ms
package.json:
"devDependencies": {
"broccoli-asset-rev": "^2.0.2",
"ember-cli": "1.13.1",
"ember-cli-app-version": "0.4.0",
"ember-cli-babel": "^5.0.0",
"ember-cli-content-security-policy": "0.4.0",
"ember-cli-dependency-checker": "^1.0.0",
"ember-cli-htmlbars": "0.7.9",
"ember-cli-htmlbars-inline-precompile": "^0.1.1",
"ember-cli-ic-ajax": "0.2.1",
"ember-cli-inject-live-reload": "^1.3.0",
"ember-cli-qunit": "0.3.15",
"ember-cli-release": "0.2.3",
"ember-cli-sass": "3.3.1",
"ember-cli-uglify": "^1.0.1",
"ember-data": "1.13.5",
"ember-disable-proxy-controllers": "^1.0.0",
"ember-document-title": "1.13.0",
"ember-export-application-global": "^1.0.2"
}
ember-cli-build.js:
/* global require, module */
var EmberApp = require('ember-cli/lib/broccoli/ember-app');
module.exports = function(defaults) {
var app = new EmberApp(defaults, {
sourcemaps: {
enabled: false,
sourceMap: false
}
});
//import material design lite
app.import('bower_components/material-design-lite-src/dist/material.js');
app.import('bower_components/material-design-icons/iconfont/MaterialIcons-Regular.woff2',{
destDir: 'assets/fonts'
});
app.import('bower_components/material-design-icons/iconfont/MaterialIcons-Regular.woff',{
destDir: 'assets/fonts'
});
app.import('bower_components/material-design-icons/iconfont/MaterialIcons-Regular.ttf',{
destDir: 'assets/fonts'
});
app.import('bower_components/material-design-icons/iconfont/MaterialIcons-Regular.eot',{
destDir: 'assets/fonts'
});
return app.toTree();
};
and, my hrad disk is SSD.
help me,please. thanks very much.
0xDing
All 48 comments
- is this initial build or incremental
- what are the times in 10.10 ?
- how many files in app/*
Can you provide a sample repo that demonstrates this issue, as this will make deducing the problem easier.
stefanpenner
on 15 Jul 2015
@stefanpenner
is incremental.
app/* have 20 files.
this is source code: https://github.com/poyueh/slow-ember-cli-project
thanks very much.
And, in OS X 10.10:
Build successful - 12866ms.
Slowest Trees | Total
----------------------------------------------+---------------------
SourcemapConcat | 2606ms
SourcemapConcat | 2437ms
SourcemapConcat | 2257ms
SourcemapConcat | 2217ms
SassCompiler | 2084ms
Funnel: App JS Files | 951ms
Slowest Trees (cumulative) | Total (avg)
----------------------------------------------+---------------------
SourcemapConcat (8) | 9550ms (1193 ms)
SassCompiler (1) | 2084ms
Funnel: App JS Files (1) | 951ms
Something seems strange. I'm building 100kloc app in way less time then that
stefanpenner
on 16 Jul 2015
holy crap the MaterialDesign stuff is > 100MB and there are 85,590 files contained within...
stefanpenner
on 16 Jul 2015
o.O yikes
runspired
on 16 Jul 2015
We still shouldn't be pulling everything in, so something is likely sideways.
Also, i have no idea why ember-cli-sass is being configured from config/environment.js this file is strictly for runtime configuration
stefanpenner
on 16 Jul 2015
The fact the sourceMapConcat is showing up in the slowTrees, and removing the icons fixes the perf problem, would seem to indicate that sourceMapConcat is likely accidentally trying to concat the icons...
I need some sleep will look more in the AM.
stefanpenner
on 16 Jul 2015
@stefanpenner
yep. this problem about app.import webfonts.
when I use google fonts.it's work. :
file deleted styles/app.scss___jb_bak___
file deleted styles/app.scss___jb_old___
file changed styles/app.scss
Build successful - 934ms.
Slowest Trees | Total
----------------------------------------------+---------------------
TreeMerger (vendor) | 167ms
SourcemapConcat | 132ms
SassCompiler | 59ms
SourcemapConcat | 54ms
SourcemapConcat | 53ms
Funnel: App JS Files | 51ms
SourcemapConcat | 49ms
Slowest Trees (cumulative) | Total (avg)
----------------------------------------------+---------------------
SourcemapConcat (8) | 325ms (40 ms)
TreeMerger (vendor) (1) | 167ms
Funnel (33) | 83ms (2 ms)
TreeMerger (161) | 70ms (0 ms)
SassCompiler (1) | 59ms
Funnel: App JS Files (1) | 51ms
and good night. thanks again
（BTW: i'm sorry my english is not good.... ）
0xDing
on 16 Jul 2015
I'm getting 113% CPU usage with latest ember-cli for some reason?
Same thing with sourcemapConcat sometimes 50,000ms-70000ms
jfrux
on 18 Jul 2015
OUCH :(
jfrux
on 18 Jul 2015
Ya something is funky. It seems to pull in far to much. I've been adding additional instrumentation to allow us to track down exactly what. I"ll hopefully land that this weekend.
stefanpenner
on 18 Jul 2015
@stefanpenner You rock brother.
Thanks for all you do.
jfrux
on 18 Jul 2015
thanks, we will fix this. It might take a little elbow grease.
stefanpenner
on 18 Jul 2015
@stefanpenner Love it. Do what you do. :dancer:
jfrux
on 18 Jul 2015
Any workarounds? ember-cli is completely unusable on our developers' macbooks at the moment.
bjornstar
on 21 Jul 2015
Any workarounds?
move only stuff you need to vendor, and import from there.
stefanpenner
on 21 Jul 2015
Two minute rebuild time. Something is horribly wrong here.
bjornstar
on 21 Jul 2015
Two minute rebuild time. Something is horribly wrong here.
yup, do you have any new information to add? I will continue to work on this later this week.
If you can provide a sample app that demonstrates your issue that would be great. The above example app revealed several issues.
stefanpenner
on 21 Jul 2015
subscribing
brandonjmckay
on 23 Jul 2015
@stefanpenner I might have an easy way for you to reproduce the issue. I was experiencing the same slow down when I upgraded ember-paper from 0.1.1 to 0.2.6. Build time shot up to more than 30 seconds. Out of desperation I switched to ember-cli-materialize and my build times dropped to less than 2 seconds again...
I then tested it on two blank ember apps and you can clearly see the difference in build times:
Run ember new ember-sample-paper && cd ember-sample-paper && ember install ember-paper, accept all the overwrites from the blueprint and start a server, you'll get something like this:
Build successful - 27814ms.
Slowest Trees | Total
----------------------------------------------+---------------------
Concat: Vendor | 11012ms
SourcemapConcat | 3554ms
SassCompiler | 3292ms
Concat: Test Support JS | 3201ms
SourcemapConcat | 3121ms
Slowest Trees (cumulative) | Total (avg)
----------------------------------------------+---------------------
Concat: Vendor (1) | 11012ms
SourcemapConcat (3) | 6694ms (2231 ms)
SassCompiler (1) | 3292ms
Concat: Test Support JS (1) | 3201ms
The do the same for ember-cli-materialize, ember new ember-sample-cli-materialize && cd ember-sample-cli-materialize && ember install ember-cli-materialize. Accept blueprint conflicts and go for jQuery ^1.11.3 when bower complains. Running ember server will give you something like this:
Build successful - 7983ms.
Slowest Trees | Total
----------------------------------------------+---------------------
Concat: Vendor | 4149ms
Babel | 703ms
Babel | 654ms
Slowest Trees (cumulative) | Total (avg)
----------------------------------------------+---------------------
Concat: Vendor (1) | 4149ms
Babel (7) | 1770ms (252 ms)
_Note my machine was performing time machine backups and other crap, so performance was greatly impacted_...
28 seconds vs 8 seconds, huge difference between the two projects. So something in ember-paper might help diagnose what is going on. Maybe @miguelcobain could help provide info on what changed in later ember-paper versions. I did a quick scan and couldn't find any references to importing web fonts as mentioned in earlier comments, so I'm stuck.
More background info:
$ node -v
v0.12.7
$ npm -v
2.11.3
$ uname -a
Darwin Kenneths-MacBook-Pro.local 14.4.0 Darwin Kernel Version 14.4.0: Thu May 28 11:35:04 PDT 2015; root:xnu-2782.30.5~1/RELEASE_X86_64 x86_64
kennethkalmer
on 23 Jul 2015
@kennethkalmer awesome thanks! With these two examples the problem has become quite accessible.
I'll double check that your scenario has also hit the same pain points i have already identified. It may yield several more.
Just an update, i have some local spikes that mitigate the majority of this slow down. Still noodling on the exact solution (that I can roll out, with minimal / no pain upgrading, and does't regression the fixes that introduced the issue). Its a fun puzzle.
stefanpenner
on 23 Jul 2015
@kennethkalmer @stefanpenner
v0.1.1 and v0.2.6 diff: https://github.com/miguelcobain/ember-paper/compare/v0.1.1...v0.2.6
Main differences between ember-paper 0.1.1 and 0.2.6 are the ember-cli version and that the material icons are now being loaded from google.
However, there is this new large scss file with every icon definition. Don't know if sass is having a hard time on that (unlikely).
Also, we're now loading a file directly from /vendor: https://github.com/miguelcobain/ember-paper/blob/master/index.js#L16
Besides this, there are more scss and component files in the newer version.
I've never had such long times building ember-paper (or anything else). I'm on linux.
miguelcobain
on 23 Jul 2015
@miguelcobain thanks for the detailed report. I just need to carve out some time to address this.
stefanpenner
on 23 Jul 2015
@stefanpenner @miguelcobain thanks! I'll definitely help test if and when needed!
kennethkalmer
on 23 Jul 2015
@stefanpenner
Just an update, i have some local spikes that mitigate the majority of this slow down. Still noodling on the exact solution (that I can roll out, with minimal / no pain upgrading, and does't regression the fixes that introduced the issue). Its a fun puzzle.
Do you mind describing exactly where you think the problem is? Other people, myself included, could use that to help you. I am going through ember-cli massive ~1500 LOC base hopelessly trying to find a crack of light.
ghost
on 24 Jul 2015
Want to confirm that sourceMapConcat line stayed in my compilation until I removed google material design icons and hammerjs from bower_components directory.
Also, should be noted that it was loading them even though I didn't have them in Brocfile or bower.json... what's that about?
Also, I'm using ember-cli 0.2.7 so it's not isolated to 1.13.x.
If sourcemap concat is involved what so ever it seems to have the slowdown.
Once I deleted hammerjs and google material design icons form bower_components the slowness went away. As well as the sourcemap concat line.
:-/ That's all I have for now!
jfrux
on 24 Jul 2015
I was talking with @rwjblue about this last night. I was noticing the same thing with the list-view addon that has a bunch of images in the dummy test app public folder. I thought I heard someone mention that things that shouldn't be (images, svgs, etc) are running through source maps. Haven't had time to investigate.
raytiley
on 24 Jul 2015
Yes @raytiley more is getting pulled in then expected, and also there exist some invalidations upstream that need to be fixed. Like broccoli-config-replacer.
For those interested in helping my time is extremely limited. But high level tasks are as follows. I plan on working on this again after I get the next cli release out. I'll follow up with more details.
- migrate out broccoli plugins to the new broccoli-plugin baseclass
- help finish broccoli-cauliflower as a broccoli-plugin upgrade (I think just tests need porting)
- introduce invalidation logging (like I added to broccoli-caching-writer) to broccoli-cauliflower
- merge Broccoli-cauliflower into broccoli-filter (just think of the former as an experimental branch of the later, that should now be merged)
- upgrade usages of merge-trees to the one that is stable (I believe the most recent version is)
- make broccoli-config-replacer stable (by migrating to the new filter, or if not possible moving to a strategy that is stable)
Please note, although I would love help. It is risky for me to gamble on donating time. To contributors I am unfamiliar with. This should be seen as distrust, rather not following through due to other obligations is common (and fair) but it does often result in a large time cost for me.
I would love to overcome the above, as I get more familiar with contributors I also will invest time in them. I wish I had unlimited time to invest in all, because getting extra help is extremely... Helpful😉
stefanpenner
on 24 Jul 2015
@stefan How come you are the only person working on this? ember-cli has gone out of control… This is just a wrapper for broccoli, but there is more source code in this project than in broccoli and grunt combined. And roughly 10 times more than in Fly.
ghost
on 24 Jul 2015
@stefan How come you are the only person working on this?
I'm not, many contributors -> https://github.com/ember-cli/ember-cli/graphs/contributors just everyone has varying availability.
ember-cli has gone out of control… This is just a wrapper for broccoli,
Sure, if you have suggestions to reduce entropy that would be valuable. But maybe you don't have a good grasp on the project, if it perceived as just a "Wrapper" around broccoli.
ember-cli has gone out of control…
If you don't have constructive & actionable suggestions, I would suggest not exerting energy emitting stray thoughts. It isn't of value here.
stefanpenner
on 24 Jul 2015
@stefanpenner And that was trying to be constructive. Well, there is always deprecate and use your energy on more _ambitious_ enterprises. Advice noted.
ghost
on 24 Jul 2015
Well, there is always deprecate and use your energy on more ambitious enterprises.
maybe our definition of constructive differ...
stefanpenner
on 24 Jul 2015
@stefanpenner I know you are really busy on this project, and although I can see there are quite a number of contributors (after all, this is what ember users are, "for better or worse", stuck with), you seem to be the only one trying to fix bugs and answering to issues. I asked in a couple other open issues regarding specific places where one could help you, but only got the list you posted above. Unfortunately that is not good enough to give you a hand.
I want to know what happens between task1 and task2, because I am experienced "black holes". Instrumentation shows the ms times for each task (concat, babel, merge, etc) but why does stuff stuck in between?
I have started to think the slow builds my team is experiencing have to do more with Broccoli's hard disk abuse and rampant IO dynamics than with ember-cli itself.
if it perceived as just a "Wrapper" around broccoli.
Please help me clarify then. If this is not just a Broccoli wrapper to play well with Ember On Rails rules :wink: , what is it exactly?
Also, one more question. From an earlier comment you replied to:
- Any workarounds?
- move only stuff you need to vendor, and import from there.
What does it mean exactly to move only the stuff one needs to _vendor_?
ghost
on 24 Jul 2015
@stefanpenner I know you are really busy on this project,
Then please be mindful of my time.
I feel you are demanding I teach you the system, in-order for you to provide a hand. Based on this conversation I am dubious if this investment is at all in my (or the larger communities) favor.
Please note, instead of spending the last hour continuing work on the list above, I have spent my time attempting to be accommodating in this thread.
I have started to think the slow builds my team is experiencing have to do more with Broccoli's hard disk abuse and rampant IO dynamics than with ember-cli itself.
It is accurate that the build performance issues are during a broccoli.rebuild, but it is actually not broccoli nor IO that is the problem. For some reason, IO keeps coming up, but the system isn't at all bottlenecked on IO.
Also, only time the system is CPU bottlenecked, is when a misbehaving plugin or two are causing invalidations or accidentally interacting with an unexpected number of files.
Good news is these are totally solvable problems, just requires some elbow grease (enumerated in the above action items).
I want to know what happens between task1 and task2, because I am experienced "black holes". Instrumentation shows the ms times for each task (concat, babel, merge, etc) but why does stuff stuck in between?
As mentioned above, adding invalidation logging to cauliflower-filter and then merging cauliflower-filter into mainline will help illuminate this for you. Now that I am on my laptop, I can share a recent improvement to the logging of broccoli-caching-writer.
If you want to be helpful, please start with that.
In hopes of actually getting work done, I will likely not respond to further comments in this thread. This should not be taken in a negative way, but rather a maintainer triaging their time appropriately.
stefanpenner
on 24 Jul 2015
quick update:
introduce invalidation logging (like I added to broccoli-caching-writer) to broccoli-cauliflower
this task is done, pending review -> https://github.com/caitp/cauliflower-filter/pull/16
help finish broccoli-cauliflower as a broccoli-plugin upgrade (I think just tests need porting)
stefanpenner
on 27 Jul 2015
In case it helps anyone:
I'm on Linux and an empty Ember project builds in 7 seconds on my machine. Installing ember-paper jumped that to 30+ seconds and a second file save while it was building would sometimes freeze the whole thing and require a kill -9. I found that it was entirely due to material-design-icons and since I wasn't using any icons this was a nice workaround:
- remove
material-design-iconsline from bower.json rm -rf bower_components/material-design-iconspkill watchmandue to some sort of IN_Q_OVERFLOW watch errorember serve
Builds are back to 7 seconds.
jerel
on 28 Jul 2015
FWIW, on my MacBook, I tried to remove material-design-icons from ember-paper as well and my rebuilds went down to under a second.
cah-danmonroe
on 28 Jul 2015
@jerel @cah-danmonroe I'm not getting this. ember-paper never had material-design-icons in bower.json. :confused:
Edit:
Forget it. We're adding material-design-icons in the default blueprint of the addon.
This is an official package from google. I followed instructions here which point to bower: http://google.github.io/material-design-icons/#getting-icons
Alternatives appreciated.
miguelcobain
on 28 Jul 2015
So in addition to the above invalidations the are causing spurious cache purges.
broccoli-sourcemap-concat are broccoli-caching-writer are mis-aligned
BCW is not made aware when BSC has only specific files to deal with, resulting in BCW stating and iterator potentially massive trees. A quick local test showed that making BCW aware of BSC's inputFiles results in in nice improvement. Not the whole perf story, but a nice step in the right direction. I suspect combined with my above diagnosis will result in quite a nice boost.
@bucaran ^ – should be an easy win.
stefanpenner
on 31 Jul 2015
https://github.com/ember-cli/broccoli-caching-writer/pull/53 reduces @borisding1994's example apps incremental time from 12s -> 2.5s for me, and initial from 20s -> 3.5s.
remaining chunks of time:
- 800ms funnel – up next for me
- this is twofold, actually a bug in ember-cli causing the funnel to need to deal with more files then it should – will be addressed sometime soon
- an algorithmic change to funnels walkSync will allow for it to work performently in-light of the above issue - will be addressed next.
- 1.7s ember-cli-sass – sometime soon
- which apparently doesn't use any caching and just always rebuilds
- will most likely benefit from https://github.com/ember-cli/broccoli-caching-writer/pull/53 and the algorithmic changes to walkSync
I plan on landing the caching-writer & funnel changes in a new release in the next day or so.
_note_ My guess would be HDD consumers will dramatically benefit from this work.
for those interested, the BCW fix also includes some handy logging to detect these issues quickly: https://github.com/ember-cli/broccoli-caching-writer/commit/2b8b0a5ac21f7a2522b81c5664a1f5eca229c6de visibility is half the battle.
Combined with the new broccoli-plugin work, we should be seeing some nice perf boosts over the next bit.
stefanpenner
on 6 Aug 2015
fwiw its not just ember-cli-sass rebuilding every time, the issue exists over in ember-cli-less as well
There were several tickets opened in ember-cli-less but each was closed..
david-duncan
on 6 Aug 2015
Which very likely means it's in ember-cli-stylus as well which still hasn't added addon support.
runspired
on 7 Aug 2015
800ms funnel – up next for me
https://github.com/joliss/node-walk-sync/pull/5 mitigates the funnel cost nearly entirely.
I have a local WIP that integrates it with Funnel but the code needs some cleanup. I'll be cleaning it up tonight, with a release planned this weekend.
stefanpenner
on 8 Aug 2015
updated #'s with https://github.com/ember-cli/ember-cli/pull/4621
initial build:
Build successful - 3078ms. (was nearly 20,000ms)
Slowest Trees | Total
----------------------------------------------+---------------------
SassCompiler | 2060ms
Slowest Trees (cumulative) | Total (avg)
----------------------------------------------+---------------------
SassCompiler (1) | 2060ms
Babel (3)
incremental build:
Build successful - 1805ms. (was ~12,000ms)
Slowest Trees | Total
----------------------------------------------+---------------------
SassCompiler | 1457ms
Slowest Trees (cumulative) | Total (avg)
----------------------------------------------+---------------------
SassCompiler (1) | 1457ms
this still isn't acceptable, I believe the SassCompiler times can be brought down to cost of "noise"
Interesting Summary:
- > 10s shaved off from both initial and incremental (on SSD, likely more on HDD)
- remaining largest chunk of time appears to be easy pickings
- will be part of the next CLI release
A quick look at SassCompiler reveals another relatively straight-forward set of fixes. I may limit external imports to ones explicitly added to the load-path. Although the ergonomics of that aren't the best, the perf outcome is fantastic. A local prototype reveals with that change the SassCompiler times drop to ~200ms on initial, and nearly nothing on incremental (if no sass changed).
I hope to work on the sass story later this weekend.
relevant pr:
stefanpenner
on 8 Aug 2015
The dude is a machine. Thank you so much stefanpenner.
brandonjmckay
on 10 Aug 2015
v1.13.8 now includes the fixes the ember-cli related the original author reported. The Sass issue is still out-standing, and I will likely investigate that later this week.
updated numbers:
➜ slow-ember-cli-project git:(master) ✗ ember s
version: 1.13.8
Deprecation warning: sassOptions should be moved to your Brocfile
Livereload server on http://localhost:49152
Serving on http://localhost:4200/
Build successful - 8510ms.
Slowest Trees | Total
----------------------------------------------+---------------------
SassCompiler | 7378ms
Slowest Trees (cumulative) | Total (avg)
----------------------------------------------+---------------------
SassCompiler (1) | 7378ms
Job 1, 'ember s' has stopped
➜ slow-ember-cli-project git:(master) ✗ bg
Send job 1 'ember s' to background
➜ slow-ember-cli-project git:(master) ✗ touch app/app.js
➜ slow-ember-cli-project git:(master) ✗ file changed app.js
Build successful - 1877ms.
Slowest Trees | Total
----------------------------------------------+---------------------
SassCompiler | 1531ms
Slowest Trees (cumulative) | Total (avg)
----------------------------------------------+---------------------
SassCompiler (1) | 1531ms
As you can note: Sass(and other CSS preprocessors) still suffer a similar issue, but will be addressed soon.
There is more performance work outstanding, but the issue the original reporter described here should be addressed.
some more focused perf related tracking issues i have opened:
stefanpenner
on 10 Aug 2015
@stefanpenner thanks so much, the initial work you (and everyone else we don't know of) did made a huge difference already!
kennethkalmer
on 11 Aug 2015
Looking good dropped from 20 seconds down to 3 seconds, as an FYI for anyone else who comes here, if you are running ember-cli in windows, make sure you are enabling symlinks ;) http://www.ember-cli.com/user-guide/#symlinks-on-windows
kingpin2k
on 21 Aug 2015
Related issues
musaffa
·
51Comments
nathanpalmer
·
30Comments
GavinJoyce
·
31Comments
calvinmetcalf
·
45Comments
tjhart
·
32Comments