Angular-cli: Build with "ng -prod" is extremely slow

[ankeshdave]

Bug Report or Feature Request (mark with an x)

- [ x] bug report -> please search issues before submitting
- [ ] feature request

Versions.

ng --version
_ _ ____ _ ___
/ \ _ __ __ _ _ _| | __ _ _ __ / ___| | |_ _|
/ △ \ | '_ \ / _| | | | |/ _ | '__| | | | | | |
/ ___ \| | | | (_| | |_| | | (_| | | | |___| |___ | |
/_/ __| |_|__, |__,_|_|__,_|_| ____|_____|___|
|___/
@angular/cli: 1.1.0
node: 7.9.0
os: win32 x64 (Win 10 Enterpirise - 16Gig , i7)
@angular/animations: 4.0.0
@angular/common: 4.0.0
@angular/compiler: 4.0.0
@angular/core: 4.0.0
@angular/forms: 4.0.0
@angular/http: 4.0.0
@angular/platform-browser: 4.0.0
@angular/platform-browser-dynamic: 4.0.0
@angular/router: 4.0.0
@angular/cli: 1.1.0
@angular/compiler-cli: 4.0.0
@angular/language-service: 4.0.0

Repro steps.

$ ng build --prod --aot=false -ec --sourcemap=false
Hash: 552d9867ececa788ac3a
Time: 670222ms
chunk {0} polyfills.d77767cee98c83f90386.bundle.js (polyfills) 294 kB {5} [initial] [rendered]
chunk {1} main.255e1d27fe4d40b56ab2.bundle.js (main) 1.51 MB {4} [initial] [rendered]
chunk {2} scripts.45a4574f31d7b91e5a5e.bundle.js (scripts) 3.88 MB {5} [initial] [rendered]
chunk {3} styles.ac17d766c993c4027892.bundle.css (styles) 228 bytes {5} [initial] [rendered]
chunk {4} vendor.86c029ec4598fb674488.bundle.js (vendor) 8.23 MB [initial] [rendered]
chunk {5} inline.750f0337ff0a9bee9813.bundle.js (inline) 0 bytes [entry] [rendered]

ng build --aot -ec -sm
Hash: 24f744145f250126e699
Time: 62809ms
chunk {0} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 294 kB {5} [initial] [rendered]
chunk {1} main.bundle.js, main.bundle.js.map (main) 3.07 MB {4} [initial] [rendered]
chunk {2} scripts.bundle.js, scripts.bundle.js.map (scripts) 3.88 MB {5} [initial] [rendered]
chunk {3} styles.bundle.css, styles.bundle.css.map (styles) 228 bytes {5} [initial] [rendered]
chunk {4} vendor.bundle.js, vendor.bundle.js.map (vendor) 7.37 MB [initial] [rendered]
chunk {5} inline.bundle.js, inline.bundle.js.map (inline) 0 bytes [entry] [rendered]

670222ms vs 62809ms is a huge difference

The log given by the failure.

There is no failure , the build takes a very long time to complete its stuck at
92% chunk asset optimization and then we have to wait for 10-15 mins to get the build completed.
I have disabled aot and sourcemaps, still the speed is very slow.

Desired functionality.

build should have similar time that to of ng build -aot

Any help and guidance to reduce will be helpful.

[ankeshdave]

@filipesilva Is this the expected behavior, do i need to change something ?

[filipesilva]

Production builds are usually slower than development builds. Besides using --aot, other optimizations are also used.

I do think your build time is too long though. I haven't seen other apps that have such a big difference between prod and dev build time. chunk asset optimization is a phase where sourcemaps are usually handled... but you have them turned off.

I'm unsure of what's happening. Perhaps it's related to https://github.com/webpack/webpack/issues/4863.

Is this a public project that I can look at? You might also have a cyclic dependency perhaps.. if so, https://github.com/angular/angular-cli/pull/6813 could help you.

Note: sourcemaps are disabled in production builds by default, while css extraction is enabled by default.

[elvisbegovic]

Same problem here

ng build --aot=false : 41286ms
ng build --aot=false --prod : 66594ms
ng build --prod : 193155ms

@filipesilva you can check here if you can see import problem : https://plnkr.co/edit/xMf8HXoNIhq3urJ6cPXQ?p=preview

This is one proble "slow" but Other problem is : I don't understand why on dev mode "ng serve" after cli recompile fils changes and finish the browser autorefresh and I need wait 6-7seconds until browser finish refreshing this is very very big and make me crazy

[GoNode5]

@filipesilva how is Angular-cli tested with real life projects? 20-50 modules and 5-10 thirdpary JS Libs?

[hoangit21]

Same issue here. Im using
@angular/cli: 1.4.2
node: 8.5.0
Os: Mac os sierra.
Script: ng build --prod --env=prod

[billdwhite]

Same issue here:
@angular/cli: 1.4.3
node: 8.5.0
OS: Mac OSX 10.13 (High Sierra)
ng build --prod --env=prod --target=production --sourcemaps

 [exec] Date: 2017-09-27T23:19:55.604Z
 [exec] Hash: fa2db26f638074fd1389
 [exec] Time: 476879ms

[billdwhite]

FYI, it speeds up considerably when I leave off the --sourcemaps option. Off course, that's a must have so it still needs a fix......

 [exec] Time: 111347ms

[ShadowManu]

Anything on this @filipesilva? We are running ng build --prod --aot and having times of:

• libs=4.4.x + cli=1.4.9 ~> 12 minutes
• libs=5.0.0-rc.x + cli=1.5.0.rc.x ~> +28 minutes

[mko]

Any update on this? We're even to the point where we're seeing JavaScript heap errors when we compile with sourcemaps turned on.

[thatisuday]

Time: 311644ms
chunk {0} main.71984717b389e49ae42e.bundle.js (main) 2.64 MB [initial] [rendered]
chunk {1} polyfills.e6475d2787bb3154d59c.bundle.js (polyfills) 61 kB [initial] [rendered]
chunk {2} styles.51bed08d75b28990a11e.bundle.css (styles) 671 kB [initial] [rendered]
chunk {3} inline.b8bddd744d2f9e590618.bundle.js (inline) 1.45 kB [entry] [rendered]

Angular CLI: 1.5.2
Node: 7.10.0
OS: win32 x64
Angular: 5.0.2
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

@angular/cdk: 5.0.0-rc0
@angular/cli: 1.5.2
@angular/material: 5.0.0-rc0
@angular-devkit/build-optimizer: 0.0.33
@angular-devkit/core: 0.0.20
@angular-devkit/schematics: 0.0.36
@ngtools/json-schema: 1.1.0
@ngtools/webpack: 1.8.2
@schematics/angular: 0.1.5
typescript: 2.4.2
webpack: 3.8.1

[gozeon]

Date: 2017-11-23T01:59:15.670Z
Hash: dfd7e6a64a5c706ebc84
Time: 461173ms
chunk {0} 0.51b252b5a5d17d9e26c9.chunk.js (common) 358 kB {16}  [rendered]
chunk {1} 1.e4aae86fef81cbdf4571.chunk.js () 490 kB {16}  [rendered]
chunk {2} 2.e0e90ed4a7c76fb9cc64.chunk.js () 443 kB {16}  [rendered]
chunk {3} 3.4f8163065e611b0b5b99.chunk.js () 29.4 kB {16}  [rendered]
chunk {4} 4.85e0c8b989186f994199.chunk.js () 661 kB {16}  [rendered]
chunk {5} 5.ac3ae6b24f5617625496.chunk.js () 54.8 kB {16}  [rendered]
chunk {6} 6.346f88d49d734d317eca.chunk.js () 67.9 kB {16}  [rendered]
chunk {7} 7.f45af7d2f19ea142be6b.chunk.js () 378 kB {16}  [rendered]
chunk {8} 8.67d5955810160bf3abc1.chunk.js () 261 kB {16}  [rendered]
chunk {9} 9.117ecb1bbecbb2b8dae3.chunk.js () 85.6 kB {16}  [rendered]
chunk {10} 10.d26d500eaae787ac8e06.chunk.js () 44.7 kB {16}  [rendered]
chunk {11} 11.bccce4a907647697e438.chunk.js () 11.9 kB {16}  [rendered]
chunk {12} 12.58c78451438f761d9fc3.chunk.js () 24.8 kB {16}  [rendered]
chunk {13} 13.c232605735cecba45737.chunk.js () 361 kB {16}  [rendered]
chunk {14} 14.c3b4c5317a93227ef829.chunk.js () 42.7 kB {16}  [rendered]
chunk {15} polyfills.14a6e475fb60a31cb355.bundle.js (polyfills) 63.7 kB {20} [initial] [rendered]
chunk {16} main.7f2fb8db839c9f4091d4.bundle.js (main) 533 kB {19} [initial] [rendered]
chunk {17} scripts.6082498be40515e405ed.bundle.js (scripts) 109 kB {20} [initial] [rendered]
chunk {18} styles.b93f29cd6d301dfb05ca.bundle.css (styles) 308 kB {20} [initial] [rendered]
chunk {19} vendor.ca648642601664013b21.bundle.js (vendor) 2.81 MB [initial] [rendered]
chunk {20} inline.1cfa0fd976f080e73d87.bundle.js (inline) 1.86 kB [entry] [rendered]

[KissBalazs]

Angular 5

Time: 413322ms
chunk {0} polyfills.1970941146babc162eea.bundle.js (polyfills) 99.8 kB [initial] [rendered]
chunk {1} main.14b317fb1264dafeb9ad.bundle.js (main) 3.96 MB [initial] [rendered]
chunk {2} styles.1d741fb54f277aec9a5f.bundle.css (styles) 738 bytes [initial] [rendered]
chunk {3} inline.51acfa358278247dad6c.bundle.js (inline) 1.45 kB [entry] [rendered]

[filipesilva]

1.6.0 should improve this situation by updating to a version of webpack that has a fix for production builds. Can you update to 1.6.0 and tell me if it's better?

Production builds will always take longer than dev builds though. There's multiple optimization passes that take a long time to provide with smaller bundles.

[alberto-chiesa]

After upgrading to 1.6.0 I went from ~55 secs to ~45 secs for a small project, without css processing. It seems there is an improvement. I don't dare to say it's too significant, but it's a step in the right direction. Btw, is there a recommended upgrade path? I just updated package.json, removed node_modules and package-lock.json and npm installed everything from scratch: without a clean and reinstall nothing would work.

[GoNode5]

@filipesilva no luck time: 167712ms .. ng build --prod
Angular CLI: 1.6.0
Node: 8.7.0
OS: win32 x64
Angular: 5.1.0

@angular/cli: 1.6.0
@angular-devkit/build-optimizer: 0.0.35
@angular-devkit/core: 0.0.22
@angular-devkit/schematics: 0.0.41
@ngtools/json-schema: 1.1.0
@ngtools/webpack: 1.9.0
@schematics/angular: 0.1.10
@schematics/schematics: 0.0.10
typescript-string-enums: 0.3.5
typescript: 2.6.2
webpack-dev-middleware: 1.12.2
webpack-dev-server: 2.9.7
webpack-merge: 4.1.1
webpack-sources: 1.1.0
webpack: 3.10.0

[mvsrinivasan]

Same issue Production Build taking too long to finish
Date: 2017-12-08T19:13:41.142Z
Hash: f6074f34cd93840962b9
Time: 347917ms
angular/cli: 1.5.0
angular/material: 5.0.0-rc0
angular-devkit/build-optimizer: 0.0.32
angular-devkit/core: 0.0.20
angular-devkit/schematics: 0.0.35
ngtools/json-schema: 1.1.0
ngtools/webpack: 1.8.0
schematics/angular: 0.1.2
typescript: 2.6.1
webpack: 3.8.1

[rodrijuarez]

Same issue here:

Time: 565883ms

@angular/[email protected] and Angular 5.1.0

[rezonjov]

on my case it takes a whooping 42 mins to finish

[ScottSpittle]

For me it used to be ~30mins now its ~1h 20mins

[dprandzioch]

Same here, taken 160s before, now it takes 390s with these params: node_modules/@angular/cli/bin/ng build --output-path=build/default/ --target=production --sourcemap

[filipesilva]

That's definitely a long time. Does it get smaller if you pass --build-optimizer=false?

[filipesilva]

@rezonjov @ScottSpittle @dprandzioch are you using Angular 5.x?

[mvsrinivasan]

Partial relief with --build-optimizer=false and cli 1.6

Date: 2017-12-11T16:38:33.792Z
Hash: e0338ceb997073672491
Time: 224756ms (vs 347917ms on cli 1.5)
chunk {0} polyfills.5331a70a99045b106003.bundle.js (polyfills) 61.2 kB [initial] [rendered]
chunk {1} styles.64f371138d201ab1d186.bundle.css (styles) 367 kB [initial] [rendered]
chunk {2} main.18c151adc1813fa3ddd7.bundle.js (main) 155 kB [initial] [rendered]
chunk {3} vendor.fe2debb1be57f2ba9fa3.bundle.js (vendor) 1.21 MB [initial] [rendered]
chunk {4} inline.2b90f1f684dd564677bf.bundle.js (inline) 1.45 kB [entry] [rendered]

[ScottSpittle]

@filipesilva Angular 5.1.0 and CLI 1.6

[EthanYueZhao]

I have the same issue.
Angular 5.1.0 and CLI 1.6
On Angular 4.4, it used to be around 40 seconds.
Right now, it took more than 110 seconds. Usually, it's around 300+ seconds.

[dprandzioch]

@filipesilva Angular 5.1.0 and CLI 1.6 as well (I'm trying to build on MacOS Sierra). The most time waiting, I'm stuck at "92% chunk asset optimization".

[ScottSpittle]

@SV-efi Thanks, turning that off made a huge difference

[filipesilva]

How big are your projects btw? If you are on Linux/OSX/Git bash you can use this command to see your number of files by extension cd src/ && find -type f | sed -e 's/.*\.//' | sort | uniq -c.

We have a test project with a lot of TS files (https://github.com/filipesilva/angular-cli-test-repo) and it creates prod builds in 39m.

These are the files it has by extension:

   1551 html
      1 ico
      8 jpg
      5 json
      5 png
     46 scss
   2637 ts

The times you all are reporting are more than that... Do any of you have a project that I can take a look at and profile?

[ScottSpittle]

@filipesilva I do not have a project you can look at unfortunately, but the output for the above command has provided the following;

      1 eot
      1 gitkeep
    118 html
      1 ico
      4 json
      1 png
     91 scss
      2 svg
    315 ts
     11 ttf
      1 woff
      1 woff2

[dprandzioch]

@filipesilva

$ find . -type f  | sed -e 's/.*\.//' | sort | uniq -c
  12 css
  21 eot
   1 gif
   1 gitkeep
  90 html
   1 ico
  85 js
  28 json
 128 scss
  30 svg
 435 ts
  21 ttf
   1 vimrc
  21 woff
  10 woff2

[rezonjov]

@filipesilva I'm using Angular cli 1.5 and Angular 4.4.6.

Here's the result from cd src/ && find . -type f | sed -e 's/.*\.//' | sort | uniq -c

   1 DS_Store
   1 bak
   1 gitkeep
  93 html
   1 ico
   5 jpg
   3 json
   8 png
  94 scss
 205 ts

Current build time: 41:27 mins using ng build --prod --build-optimizer

EDIT: I remember it used to be around 15 mins 1-2 months back where the project used to be smaller and Angular 4.x and angular 1.3

[gudboisgn]

C:xampp\htdocs\ERPAn>ng build --aot
Date: 2017-12-19T11:10:46.359Z
Hash: e5aa9c132a0567fb1560
Time: 3094966ms

[100cm]

+1
one word:slow,two words: too slow
400000++ ms
how to fix this ,

but i can feel project 's size more slim now

[exzhawk]

Angular 5 angular-cli 1.6.0
small project
``` 11 html
1 ico
2 json
1 png
11 scss
38 ts

stuck at `94% asset optimization`

```Date: 2018-01-02T03:44:22.006Z
Hash: a14d64dbd49b93516533
Time: 297631ms
chunk {0} polyfills.43a6a16e791d2caa0484.bundle.js (polyfills) 60.9 kB [initial] [rendered]
chunk {1} main.a95e3607345f8c7c70f4.bundle.js (main) 876 kB [initial] [rendered]
chunk {2} styles.b0947768ba27143612ba.bundle.css (styles) 46.3 kB [initial] [rendered]
chunk {3} inline.ed783d1e70f001f84c76.bundle.js (inline) 1.45 kB [entry] [rendered]

ng build --prod --build-optimizer=false reduce build time to 254389ms, but still too slow.
Removing "scripts": [ "../node_modules/plotly.js/dist/plotly.min.js" ], make it 54880ms, so is it cost webpack too much time processing this?

[FrancescoBorzi]

I can confirm this issue

[michelre]

I encountered this issue. I needed to use --build-optimizer=false option to have a build time acceptable.

[gnovotny]

Can confirm using --build-optimizer=false speeds up my build by about 10x

[joaopslins]

Can confirm as well, --build-optimizer=false takes the same time as before 5.0.

[elesueur]

The issue is the vendor chunk is now being included the main chunk as far as I can tell [1] and uglify is churning away... Our build regressed from about 3m to >30m... I killed it before it even finished.

Passing --vendor-chunk=true resolves the issue without disabling the optimizer completely.

[1] https://github.com/angular/angular-cli/wiki/build search for build-optimizer

[filipesilva]

@elesueur bear in mind that re-enabling the vendor chunk will greatly reduce the efficacy of Build Optimizer. This was around 40% bigger bundles in our testing.

[elesueur]

@filipesilva yes, I am running some tests on ours now to see what the benefit is in our scenario.

So if my understanding is correct, before, uglify was never run on the vendor libraries (probably because we could assume they were already minified/optimized) and now that the vendor chunk is by default disabled, the optimizer now runs over all of the third-party libraries as well?

Where is the reduction in code size coming from? Tree shaking?

[filipesilva]

@elesueur uglify is ran over all bundles, vendor or otherwise. The size reduction comes from some advanced optimizations in Build Optimizer.

See https://github.com/angular/devkit/tree/master/packages/angular_devkit/build_optimizer for more information. There's also a talk about it (https://www.youtube.com/watch?v=usRrkAzZ108) that goes into a bit more detail on what's happening.

[filipesilva]

We also have some more improvements to production build speed in 1.7, which you can try today via @angular/[email protected].

[elesueur]

Thanks @filipesilva is there any analysis of the build speed or is build speed basically ignored because it's considered a one-time payment for a long-time benefit?

[elesueur]

Here is analysis of our app. The build is of course single threaded so my macbook's dual core processor is only half utilized. The system was mostly idle otherwise. I am running a second datapoint because this morning my first attempt at the 1.6 build was taking much longer than 18 minutes under similar conditions. But long story short, a much longer build for a 2% reduction in code size.

====

After (cli 1.6 with default optimizations and AOT):

Date: 2018-01-19T15:38:57.874Z
Hash: f0ff32f352d583d9a60e
Time: 1064637ms (18 mins)
chunk {scripts} scripts.a09df21daa6632eb9407.bundle.js (scripts) 33.3 kB [initial] [rendered]
chunk {0} 0.d021fc4a7108538dfdf5.chunk.js () 89.3 kB [rendered]
chunk {1} main.b3bf84b1941b7f2c01c2.bundle.js (main) 4.42 MB [initial] [rendered]
chunk {2} polyfills.50f916ca991d321c737c.bundle.js (polyfills) 624 kB [initial] [rendered]
chunk {3} styles.ed292a274fc61d960e6a.bundle.css (styles) 422 kB [initial] [rendered]
chunk {4} inline.c14b5104142c89623613.bundle.js (inline) 1.47 kB [entry] [rendered]

vendor + main = 4.42MB

Before (cli 1.4 with default optimizations and AOT)

Date: 2018-01-19T15:02:38.152Z
Hash: 1be9e9d293dabdb3ca0d
Time: 285541ms (5 mins)
chunk {scripts} scripts.a09df21daa6632eb9407.bundle.js (scripts) 33.3 kB [initial] [rendered]
chunk {0} 0.d021fc4a7108538dfdf5.chunk.js () 89.3 kB [rendered]
chunk {1} polyfills.5b6fa84cc1992d5fb122.bundle.js (polyfills) 624 kB [initial] [rendered]
chunk {2} main.9f0ad465d9688fbae36f.bundle.js (main) 1.59 MB [initial] [rendered]
chunk {3} styles.ed292a274fc61d960e6a.bundle.css (styles) 422 kB [initial] [rendered]
chunk {4} vendor.0d5ee7b13af3505dfd6a.bundle.js (vendor) 2.94 MB [initial] [rendered]
chunk {5} inline.7abf96772a9fd0fe6c28.bundle.js (inline) 1.47 kB [entry] [rendered]

vendor + main = 4.53MB

Difference in time is +13 minutes
Difference in size is -110KB

[filipesilva]

Thanks for the analysis @elesueur, seeing concrete numbers always helps.

My example was Angular.io, where Build optimizer reduced the bundle from ~1200kb to ~600kb. Re-enabling the vendor chunk there made it to up to 850kb. Size reduction varies with projects and is usually bigger the more core angular packages (including material) are used.

I'm looking at using cache and parallelization for Build Optimizer in https://github.com/angular/angular-cli/pull/9295, which should help reduce the build time. @clydin was also looking at a some lightweight transformations in https://github.com/angular/devkit/pull/346.

[clydin]

@elesueur if you have some time, could you also try with 1.7.0-beta.1?

[elesueur]

@clydin does 1.7.0-beta.1 include the changes from @filipesilva's PR9295?

[elesueur]

@clydin with 1.7.0-beta.1 I get a much better time:

Date: 2018-01-19T16:42:19.664Z
Hash: 2591b618214eda399bdc
Time: 303353ms
chunk {scripts} scripts.a09df21daa6632eb9407.bundle.js (scripts) 33.3 kB [initial] [rendered]
chunk {0} 0.83ab10e09bad63e047f5.chunk.js () 89.9 kB [rendered]
chunk {1} main.4531b31adb9a517a8685.bundle.js (main) 4.44 MB [initial] [rendered]
chunk {2} polyfills.50f916ca991d321c737c.bundle.js (polyfills) 624 kB [initial] [rendered]
chunk {3} styles.cfdfe4999edebddcfc3c.bundle.css (styles) 419 kB [initial] [rendered]
chunk {4} inline.5c99c209a7f19ff42247.bundle.js (inline) 1.47 kB [entry] [rendered

and pretty close to the same optimization.

[clydin]

@elesueur the current beta does not contain PR9295. It does contain several other performance optimizations.

[pmoncadaisla]

Very very slow here also, went from 7min to 42+ minutes.
Using @angular/[email protected] and also --build-optimizer=false.

NODE_OPTIONS="--max-old-space-size=8192" ng build --aot --sourcemaps=true --build-optimizer=false --app 0 --locale es --environment=prod --vendor-chunk=true

[anthanh]

Using @angular/[email protected] and [email protected] our build time looks normal again (aot+build optimizer enabled).

Before beta.2 (~50mins)

Hash: 02f1a2706d3f429e32ef
Time: 3121335ms
chunk {0} 0.619b0351c23f5dbface9.chunk.js, 0.619b0351c23f5dbface9.chunk.js.map () 5.41 kB  [rendered]
chunk {1} main.4fa6a69c9838325ec97a.bundle.js, main.4fa6a69c9838325ec97a.bundle.js.map (main) 3.17 MB [initial] [rendered]
chunk {2} polyfills.0d84dd3ba77448d5366c.bundle.js, polyfills.0d84dd3ba77448d5366c.bundle.js.map (polyfills) 221 kB [initial] [rendered]
chunk {3} styles.95a5f4de371f4c77165d.bundle.css, styles.95a5f4de371f4c77165d.bundle.css.map (styles) 69.7 kB [initial] [rendered]
chunk {4} inline.54fc964e24b358bc47f3.bundle.js, inline.54fc964e24b358bc47f3.bundle.js.map (inline) 1.53 kB [entry] [rendered]

After beta.2 (~8mins)

Hash: 91fd480e9565d349ed95
Time: 467219ms
chunk {0} 0.b6c0ee08b80c9db0da44.chunk.js, 0.b6c0ee08b80c9db0da44.chunk.js.map () 5.46 kB  [rendered]
chunk {1} main.67232ef58178f54910ed.bundle.js, main.67232ef58178f54910ed.bundle.js.map (main) 3.4 MB [initial] [rendered]
chunk {2} polyfills.9bc55f47faa8d39d6cf2.bundle.js, polyfills.9bc55f47faa8d39d6cf2.bundle.js.map (polyfills) 222 kB [initial] [rendered]
chunk {3} styles.8195fa9e0bc62be2904e.bundle.css, styles.8195fa9e0bc62be2904e.bundle.css.map (styles) 59.8 kB [initial] [rendered]
chunk {4} inline.c7905c25db038b259ddd.bundle.js, inline.c7905c25db038b259ddd.bundle.js.map (inline) 1.53 kB [entry] [rendered]

Great job!
Thanks @filipesilva

[rafinskipg]

great @anthanh @filipesilva ! Thanks :robot: :rocket:

[filipesilva]

@angular/[email protected] includes a bunch of production perf optimizations:

• https://github.com/angular/devkit/pull/346
• https://github.com/angular/angular-cli/pull/8668
• https://github.com/angular/angular-cli/pull/9295

We hope these will improve production build times overall.

[dustincleveland]

Just installed 1.7.0-beta.3 and my build times are so much better! These improvements were super valuable to me! Thanks so much!

[khanhvu161188]

Confirm 1.7.0-beta.3 is very good. 50 mins reduce to 5 mins

[vesrah]

Can confirm migrating angular-cli from 1.6.6 to 1.7.0-rc.0 resulted in a prod build reduction from 25 minutes to 3 minutes.

[Toub]

Awesome, 1.6.6=>1.7.0-rc.0 reduces my production build time from 18mn to 3mn!

Edit: here is the command I use to build:

ng build --prod --environment=deploy --vendor-chunk=true --locale=fr --progress=false

[dprandzioch]

Can you please write, which build command you use/which configuration for this time improvement? For me, even with 1.7.0-rc.0 build times still are almost the same, but maybe I'm doing something wrong...

[Toub]

@dprandzioch I posted my comand line above.

Just to be sure, did you install 1.7.0-rc.0 locally (local version has priority over global one)?

```bash
npm i --save-dev @anguler/[email protected]
ng --version

[remisture]

1.7.0-rc.0 is much faster for me as well, but for some reason my fonts are not loaded correctly. It seems like the browser loads all versions (eot, svg, ttf, woff, woff2), and non of them are "active".

[PascalTemel]

@remisture This has already been fixed but you'll need to wait for a new release

[elvisbegovic]

@Toub @remisture @PascalTemel reasons for theses improvments are because build currently ignore fonts this decrease build time dramatically, when this enabled again it will still be slow and block at 92% ... (joke)

I don't know how you guys have theses improvments, seems not applied to me,
ps: TBH i don't care a little bit about build time, important to me is serve with aot in big project (>1k files), that is also improved to you ?
first build ng s --aot : 90959ms
rebuild ng s --aot : 27115ms
rerebuild ng s --aot : 23896ms
rererebuild ng s --aot : 0% blocking crashed

[xeofus]

1.6.6 (ng build --prod --aot)
Time: 2552417ms (42 min)
chunk {scripts} scripts.2f2e0b85b06251853901.bundle.js (scripts) 338 kB [initial] [rendered]
chunk {0} main.1090a59971a754842795.bundle.js (main) 6.73 MB [initial] [rendered]
chunk {1} polyfills.8477fbb8c7fe3ad47cff.bundle.js (polyfills) 64.1 kB [initial] [rendered]
chunk {2} styles.ebeede0f32b58cd49beb.bundle.css (styles) 529 kB [initial] [rendered]
chunk {3} inline.a8a43773191a9152089a.bundle.js (inline) 1.45 kB [entry] [rendered]
(~7,6MB)

1.7.0 (ng build --prod --aot)
Time: 139403ms (2 min)
chunk {scripts} scripts.2f2e0b85b06251853901.bundle.js (scripts) 338 kB [initial] [rendered]
chunk {0} main.7855996d44fe44035a13.bundle.js (main) 3.23 MB [initial] [rendered]
chunk {1} polyfills.e412e915007454c18396.bundle.js (polyfills) 64.1 kB [initial] [rendered]
chunk {2} styles.a13a24dba7f782b93952.bundle.css (styles) 522 kB [initial] [rendered]
chunk {3} inline.318b50c57b4eba3d437b.bundle.js (inline) 796 bytes [entry] [rendered]
(~4,1MB)

Great! Thank you!

[subatomicglue]

Our large project is now ~2.35x Faster at building for aot/prod - THANK YOU

Here's our measurements for Angular-CLI 1.6.4 (11.4min / 53MB) vs 1.7.1 (4.8min / 53MB):

Number of files under src/:
   5 DS_Store
   1 eot
   1 gif
   1 gitkeep
 146 html
  38 jpg
   2 json
   1 md
   1 npmignore
   4 otf
  52 png
 149 scss
  16 svg
 319 ts
   1 ttf
   1 woff

Angular CLI: 1.6.4
Node: 7.5.0
OS: darwin x64
Angular: 5.2.0
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

@angular/cli: 1.6.4
@angular-devkit/build-optimizer: 0.0.42
@angular-devkit/core: 0.0.29
@angular-devkit/schematics: 0.0.52
@ngtools/json-schema: 1.1.0
@ngtools/webpack: 1.9.4
@schematics/angular: 0.1.17
typescript: 2.5.3
webpack: 3.10.0

==================================
[AOT & PROD]  (timing)
   node --max_old_space_size=16192 node_modules/.bin/ng build -aot -prod -pr false -op timing-aot-prod

Execution time was 689 s (11 m)

Size of build:
3.1M    timing-aot-prod/assets
4.0K    timing-aot-prod/inline.a70b262748fe0a55d310.bundle.js
 22M    timing-aot-prod/main.34955e8e2b71226ac437.bundle.js
224K    timing-aot-prod/polyfills.769e09b99ce9001df0b0.bundle.js
296K    timing-aot-prod/scripts.bcd65afe20c96c2e4da3.bundle.js
2.3M    timing-aot-prod/0.dd629e968ecf01ac6841.chunk.js
5.6M    timing-aot-prod/1.578685531c3f1f4e6954.chunk.js
1.2M    timing-aot-prod/2.39c050c69186a24a8177.chunk.js
1.4M    timing-aot-prod/3.dce8ef7db7a4e2a16c35.chunk.js
 11M    timing-aot-prod/4.442f379ca140db46fc06.chunk.js
1.3M    timing-aot-prod/5.4c49e82c71452ff05580.chunk.js
4.6M    timing-aot-prod/6.cb7659914c4345fda23a.chunk.js
 32K    timing-aot-prod/index.html

Total size
53M    timing-aot-prod

Angular CLI: 1.7.1
Node: 7.5.0
OS: darwin x64
Angular: 5.2.0
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

@angular/cli: 1.7.1
@angular-devkit/build-optimizer: 0.3.2
@angular-devkit/core: 0.3.2
@angular-devkit/schematics: 0.3.2
@ngtools/json-schema: 1.2.0
@ngtools/webpack: 1.10.1
@schematics/angular: 0.3.2
@schematics/package-update: 0.3.2
typescript: 2.5.3
webpack: 3.11.0

==================================
[AOT & PROD]  (timing)
   node --max_old_space_size=16192 node_modules/.bin/ng build -aot -prod -pr false -op timing-aot-prod

Execution time was 293 s (4 m)

Size of build:
3.1M    timing-aot-prod/assets
4.0K    timing-aot-prod/inline.cd131d510872097ecdeb.bundle.js
 22M    timing-aot-prod/main.f9fe7d98f354f3c411b7.bundle.js
224K    timing-aot-prod/polyfills.9cbf815124d5a3f9dcf2.bundle.js
296K    timing-aot-prod/scripts.bcd65afe20c96c2e4da3.bundle.js
2.3M    timing-aot-prod/0.2dc8a55cddfff64c6380.chunk.js
5.5M    timing-aot-prod/1.fffa7fd0f0133cc15cae.chunk.js
1.2M    timing-aot-prod/2.fecff394c1eb0cafd4f7.chunk.js
1.4M    timing-aot-prod/3.700d00e7c952fd0312d9.chunk.js
 11M    timing-aot-prod/4.278f5a8e5f8aa887b533.chunk.js
1.3M    timing-aot-prod/5.140fbfc2a6af6609ae13.chunk.js
4.6M    timing-aot-prod/6.f3fd2e34889dca730e53.chunk.js
 32K    timing-aot-prod/index.html

Total Size of build:
 53M    timing-aot-prod

[filipesilva]

Happy to hear that 1.7.0 is helping keep prod build time manageable! We put in a lot of prod optimizations in 1.7.0.

I'll leave this issue open for a few more weeks to see if there's anything off with our fixes.

[alberto-chiesa]

I can confirm that now we build with optimizations in almost the same time as before (1.6.x), but with --build-optimizer=false.

Please, keep an eye on this subject, because build times are now acceptable, but "fast" is quite another thing.

Anyway, your work is so invaluable that no "thanks" is enough! And really, really appreciated.
IMO, we just need to be more stable, predictable, fast (at both build and runtime) and slim. New features can come later.

[ShadowManu]

This issue doesn't affect me now in the size of the projects I'm working on, but I wanted to make a little stop to give my token of appreciation to @filipesilva and all the angular cli team and contributors that work hard and try to solve the community problems.

With all the desire to put a smile on your faces: you guys rock!

[filipesilva]

😄

[subatomicglue]

So, I realized I did my testing with 1.7.1 but the discussion above is with 1.7.0...
Would performance be better with 1.7.0 instead? 5min build is faster than 11min, but not really 10x improvement either, so I was wondering

Looks like 1.7.1 is released: https://github.com/angular/angular-cli/releases

[filipesilva]

@subatomicglue it should be the same really. I don't think anything important changed between those to as far as build speed is concerned.

[bharathmuppa]

Guys, I am using 1.7.1 and struck with 92% from half-n-hour. Disabling source-map and build-optimizer is workaround but not a solution. Please provide with better solution for the issue

[alberto-chiesa]

Are you sure you are on 1.7.1 also in node_modules local folder? If you do ng --version what do you read?
Chances are you have 1.7.1 in global only.

[todthomson]

I'm on 1.7.2 and when running ng build --prod --sourcemaps=true --target=production I'm getting stuck on 95% emitting for 30+ minutes both locally (and on the CI server). Works fine on 1.6.8. What I have is a pretty small Angular application. Just letting you know in case you have the solution. More than happy to provide more details if you need them. Cheers.

[filipesilva]

@todthomson is this app something we could look at? Small apps exhibiting this sort of behaviour point towards some hard to find performance bugs.

[todthomson]

@filipesilva I can't provide source (sorry) as it's a project I'm doing on behalf of one of our clients, but I may be able to do a screen share if that would work for you? Please confirm and I'll try to get approval from the client. Cheers. 💯

[filipesilva]

@todthomson could you take a CPU profile perhaps? I have some instructions in https://github.com/angular/angular-cli/issues/8259 that you could use.

[benc-uk]

I've just upgraded my CLI to 1.7.2 (I've double checked the node_modules and --version) and a my build time has gone from about 15 seconds to infinity
It hangs forever at "92% chunk asset optimization", I've waited more than 10 mins before giving up

It's a very small app

[filipesilva]

@benc-uk is this app something you could share so we can debug?

[benc-uk]

OK now I look stupid, it's working!
I did a ng build --prod --build-optimizer=false which ran OK, and since then a regular ng build --prod runs in about 12 seconds again

[filipesilva]

Glad to hear it's sorted 👍

[clabough]

I'm having the issue.

Using ng build --prod it finishes in 74112ms.
Using ng build --prod --sourcemaps true it hangs at 92% chuck asset optimization forever (left it running overnight).
Using ng build --prod --sourcemaps true --build-optimizer false, it stops at 92% chunk asset optimization for a few minutes, then hangs at 95% emitting.

CPU at about 80% during chunk asset optimization, then drops to 30% during emitting.

146 .ts
54 .html
6 .css
1 .gitkeep
1 .json

ng --version
Angular CLI: 1.7.2
Node: 8.9.4
OS: win32 x64
Angular: 5.2.7
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router
@ angular/cli: 1.7.2
@ angular-devkit/build-optimizer: 0.3.2
@ angular-devkit/core: 0.3.2
@ angular-devkit/schematics: 0.3.2
@ ngtools/json-schema: 1.2.0
@ ngtools/webpack: 1.10.1
@ schematics/angular: 0.3.2
@ schematics/package-update: 0.3.2
typescript: 2.5.3
webpack: 3.11.0

[pgrm]

currently our application builds with ng build --prod with angular cli 1.7.2 within 2.5 to 3 minutes, but building with ng build --prod --sourcemaps true takes ~7 minutes longer (so ~10 min) - although this is much faster than angular cli 1.6, where build without sourcemaps took ~5-7 minutes and with sourcemaps ~12-15 minutes, it seems like only one part of the build was improved and the large part (source map generation) didn't change.

We are generating source maps for production builds in order to publish them to exception handling services like sentry to rollbar, so it would be really cool if that part could also be improved or made parallel somehow.

[todthomson]

FYI I tried again with 1.7.3 and I'm still experiencing "infinite builds" where --sourcemaps true.

I've taken a look at https://github.com/angular/angular-cli/issues/8259 but I'm not 100% sure how to adapt these instructions to working with ng build rather than ng serve.

[Londovir]

Just did an upgrade to 1.7.3, and definitely seeing sourcemaps dragging down the build speed. Without sourcemaps, running a prod-build optimizer build takes about 74823ms. Enabling source maps on the exact same build kicks it up to 363124ms, about 5 times longer. Reasonably small project (about 129 source files, using Angular Material, rxjs, flex-layout, date-fns as the main library dependencies).

Wish there was a way to speed up sourcemaps without quintupling the build time.

[clydin]

Unfortunately, to be able to support the necessary optimizations with source maps enabled requires additional functionality to be exposed from within TypeScript itself. There is ongoing work to improve the situation, however, there is currently no timetable at this point.

[clabough]

Well, after spending most of two days making a copy of my project, then stripping modules/components out one at a time to track down the culprit, I got down to 2 feature modules and 1 common module and discovered that doing an ng build --prod -sm has an issue that causes it to hang (at 95% emitting) if any [styleUrls] contain CSS (empty css files are ok).

Looks like this is issue #8314

I put everything back to how I have my production project, modified all my components to not use [styleUrls], then ran ng build --prod -sm and after 4990841ms (83 min), it finally finished.

My /src/app stats are:

145 .ts (approx 10,200 non-blank lines)
55 .html (9.675 non-blank lines)
6 .css

If I also use --build-optimizer false, the time falls to 470616ms (7.8 min)

[bharathmuppa]

@alberto-chiesa yes I am sure and it is still the same .Hi @clydin Hope it will be resolved soon

[alberto-chiesa]

I think that a lot of performance problems will be solved once we can upgrade Webpack to 4.

I saw some quite impressive benchmarks (execution times reduced by 60% to 90%).
Unfortunately, this requires some work on many plugins. Nothing major, but there is a lot of small moving parts to be checked and minor fixes for Webpack breaking changes.

[100cm]

1.7.3 still very slow.

[joaopslins]

1.7.3 fine here. Thanks guys!

[HarelM]

Can't move to version 1.7.x due to #9980, so I can't check if the issue was resolved. :-/

[benc-uk]

I removed my node_modules (I needed a clean up) and updated to CLI 1.7.3 and the problem has come back! The only way I can get a build to complete now is with --build-optimizer=false :tired_face:

[smoses2]

I ran into this with production builds (with source maps) hanging at 95% after upgrading to angular-cli 1.7.4 on Windows. Setting --sourcemaps=false, resolved the build issue for me.

[clabough]

@smoses2 Hanging at 95% is a different issue than building slow. For me, I fixed the hanging at 95% issue by switching away from using styleurls - I either embedded the stylesheet in the style property, or added a style link in the page itself. If you're using styleurls, run a test build with them removed.

[hahn-kev]

Do we think this an issue that's going to get fixed in 1.7.x? or is this something that won't get fixed until the next major release?

[benc-uk]

I removed all styleUrls from my components as a test, and build prod finally ran, but it took about 3 minutes to finish. I added the styleUrls back, and now build prod runs fine again! and in about 10 seconds.

This is really strange

[chizhenming]

Can you provide a fast compiled package.json Thank you

@filipesilva

[clabough]

@hahn-kev I imagine they're pretty heads-down trying to get 6.0 out the door before ngConf. Hopefully the switch to Webpack 4 will clear this issue up.

[elvirdolic]

Did anyone tried with latest RC? Any performance increase?

[rightisleft]

I cant believe this is single threaded...

[rightisleft]

This is a 3 week old project: Time: 379949ms Or 6+ minutes...

    _                      _                 ____ _     ___
   / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
  / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
 / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
/_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
               |___/

Angular CLI: 1.7.4
Node: 8.11.1
OS: linux x64
Angular:

> ng build --prod --base-href /ingest/ --deploy-url /ingest/ --preserve-symlinks

 10% building modules 6/6 modules 0 activeWARNING: The `text-hide()` mixin has been deprecated as of v4.1.0. It will be removed entirely in v5.
         on line 10 of node_modules/bootstrap/scss/mixins/_text-hide.scss, in mixin `text-hide`
         from line 57 of node_modules/bootstrap/scss/utilities/_text.scss
         from line 14 of node_modules/bootstrap/scss/_utilities.scss
         from line 41 of node_modules/bootstrap/scss/bootstrap.scss
         from line 1 of stdin

Date: 2018-04-26T16:12:10.883Z
Hash: a01a8f6065671dfa4f1b
Time: 379949ms
chunk {0} main.e9b908f1aa112ccae986.bundle.js (main) 2.22 MB [initial] [rendered]
chunk {1} polyfills.194f3db5c95bb635c66b.bundle.js (polyfills) 59.7 kB [initial] [rendered]
chunk {2} styles.5124363d8f33a39f555c.bundle.css (styles) 221 kB [initial] [rendered]
chunk {3} inline.02c5a2b14b98749c9aca.bundle.js (inline) 804 bytes [entry] [rendered]

[dharmendrap7]

Thanks @clabough

ng build --prod-optimizer=false
it worked for me.

[clabough]

@dharmendrap7 Glad to hear it. I've since started using ng build --prod -sm --build-optimizer false --aot false

It's even much faster to build and oddly it results in smaller bundle/chunk files. Might be a tad slower to bootstrap though - but not noticeably.

Just found that Firefox is really slow bootstrapping non-AOT compiled code.

[clabough]

After upgrading to 6.0, ng build --prod --source-map is still taking 40 minutes. ng build --prod --source-map --build-optimizer false takes 8 minutes.

[rightisleft]

https://github.com/angular/angular-cli/issues/8344#issuecomment-387558945

[pablorsk]

• node_modules/.bin/ng build --prod --build-optimizer false: 70s
• node_modules/.bin/ng build --prod: 63s
• node_modules/.bin/ng build --prod -sm --build-optimizer false --aot false: 74s
• node_modules/.bin/ng build --prod --sourcemaps false --build-optimizer false --aot false: 28s :ok_hand:
• node_modules/.bin/ng build --prod --sourcemaps false: 60s

Any better config for a fast build? I will only try check if all code its OK, on CI., for example disable chunk assets, I don't see any option for that.

Of course building modules and chunk assets processing takes a lot of time.

[alberto-chiesa]

Have you tried disabling source maps? In my experiments it made a lot of difference.

[ampeixoto]

I just updated the CLI version from 1.5.4 to 1.7.4 and the build time went down from more than 30min to less than 5min! And I didn´t deactivate the build optimizer (ng build -prod)

Angular CLI: 1.7.4
Node: 8.11.1
OS: win32 x64
Angular: 5.2.0
... animations, common, compiler, compiler-cli, core, forms
... http, platform-browser, platform-browser-dynamic
... platform-server, router

@angular/cdk: 2.0.0-beta.12
@angular/cli: 1.7.4
@angular/language-service: 4.4.6
@angular-devkit/build-optimizer: 0.3.2
@angular-devkit/core: 0.3.2
@angular-devkit/schematics: 0.3.2
@ngtools/json-schema: 1.2.0
@ngtools/webpack: 1.10.2
@schematics/angular: 0.3.2
@schematics/package-update: 0.3.2
typescript: 2.6.2
webpack: 3.11.0

[morr0350]

Just upgraded to Angular 6 and also having problems with 'optimization: true' in our production build. Not only is the build slow (699865ms), but the app no longer works with build optimization set to true. We use Microsoft Authentication Library for Js and suddenly get mysterious "B2cAuthorityUriInvalidPath" errors from said library. Setting 'optimization: false' actually made the build take longer (769290ms) but the app now works. In both cases, I have aot set to true and sourcemaps set to true.

Angular CLI: 6.0.7
Node: 8.11.1
OS: win32 x64
Angular: 6.0.3
... animations, common, compiler, compiler-cli, core, forms
... platform-browser, platform-browser-dynamic, router

Package                           Version
-----------------------------------------------------------

@angular-devkit/architect: 0.6.5
@angular-devkit/build-angular: 0.6.5
@angular-devkit/build-optimizer: 0.6.5
@angular-devkit/core: 0.6.5
@angular-devkit/schematics: 0.6.7
@angular/cdk: 6.2.0
@angular/cli: 6.0.7
@angular/flex-layout: 6.0.0-beta.16
@angular/material: 6.2.0
@ngtools/webpack: 6.0.5
@schematics/angular: 0.6.7
@schematics/update: 0.6.7
rxjs: 6.2.0
typescript: 2.7.2
webpack: 4.8.3

[andreyselitsky]

Folks, we've swtiched to ng6 with a hope that build will be faster because of webpack4.
In reality - almost no difference, could you explain it?

[pgrm]

@andreyselitsky for me it actually seems to be marginally slower - with sourcemaps disabled it jumped from ~3min to ~3.5min and with sourcemaps enabled from ~13min to ~14min but those are average numbers and it could be caused also by something else, otherwise, as you said - definitely not the big win I hoped for because of webpack4

[LukeBillo]

We've had this issue for a long time across multiple projects - some big, some very small. It doesn't seem to be affected by the size of the project at all, and still occurs on Angular 6. Using ng build --prod takes absolutely insane amounts of time for the uglify plugin to finish.

Is there any fix for this on the roadmap without having to turn optimizations off?

[tuurbo]

This may help someone.

All 4 of my CPU cores were running at 100% when I ran ng build --prod, which caused my laptop to throttle down from 3.7 ghz to 0.7 ghz due to high CPU temperatures, which was causing very long build times.

I ended up installing ThrottleStop and lowered my "Turbo Ratio Limits" when 3+ cores were in high use (to lower temps). Also adjusted my laptop fan profiles to increase fan speed when I had very high cpu usage.

DISCLAIMER: Be very careful with ThrottleStop, it gives you complete control of your CPU, which means you could fry your (or your companies) hardware if you don't know what your doing.

CLI: 6.0.7
OS: Windows 10
CPU: Intel I7-7820HQ
MEM: 32GB

[subatomicglue]

For people with crazy slow build times with build -prod I’ll leave you with some advice that helped us: We were including bootstrap into every component’s scss file which was causing bootstrap to be included ~160 times in our project. Not only was the build output large (50mb or so unzipped) but the build took crazy amounts of time, 30 min. Now we are down to couple minutes and ~4MB uncompressed.

If you have doubts you can open your main bundle in a text editor and start asking yourself if you are seeing a lot of redundant stuff. Go from there. That’s what tipped us off.

[mvsrinivasan]

This has reached epic proportions ! Anybody seeing this slow ?.

[ThorstenHans]

Build times became a nightmare with ng6. The problem seems to be something related to source-maps generation. Which I can disable for prod builds, but for development time and debug, those are essential.

CLI: 6.0.8
OS: macOS 10.13.4
CPU: 3,1 GHz Intel Core i7
MEM: 16GB

90% of the time it's generating source-maps with the following output

92% after chunk asset optimization SourceMapDevToolPlugin main.js generate SourceMap

It's draining the developer performance dramatically and starts to become a show stopper.

I'd really love to get some update from the team here. Are you guys aware of this? Is there some investment to speed up the build times?

[spmsupun]

this fixed the issue ng build --prod --build-optimizer true --aot true

[ThorstenHans]

no, not really. Still seeing build times which are terrible.

[maxime1992]

I do have an error occurring only on CI for some reason, within our E2E tests (cypress).

The error displayed is not saying much and I can't get the stacktrace.

I do have Sentry though and all the errors are reported there which is great! But... As sourcemap is not working I can let you imagine that checking an uglified source code is far from ideal.

Sentry does support sourcemaps :raised_hands: But I can't provide them as it's not even long, it's just crashing while building with sourcemaps on. On CI, but also on my machine which has 32gb of RAM and quite a good config in general.

92% after chunk asset optimization SourceMapDevToolPlugin main.3b1b07dccf0137765ba6.js generate SourceMap

#
# Fatal error in , line 0
# API fatal error handler returned after process out of memory
#
#
#
#FailureMessage Object: 0x7f89267de9b0Illegal instruction (core dumped)

[nirdshabo]

@maxime1992 i bet its your its node v8 memory limit who sh*ts it and not your machine memory limit
its a known problem with v8 actualy... its can be a real bitch when dealing large/long strings without chunking (node has a fixed string length limit afaik)

in my own opinion v8 can be a real pain handling FILES and parsing them ... cause of those limits.
some refs
https://futurestud.io/tutorials/node-js-increase-the-memory-limit-for-your-process
https://bugs.chromium.org/p/v8/issues/detail?id=847

[thenslerse]

I have also seen a huge increase since migrating to NG6:-

when running

ng build --prod --output-path ~/Desktop/KaiSuite/public

Date: 2018-07-14T09:30:25.804Z
Hash: d580dd49dcd4e6917e1b
Time: 607608ms
chunk {0} runtime.a66f828dca56eeb90e02.js (runtime) 1.05 kB [entry] [rendered]
chunk {1} styles.6e13815b16b2c751820b.css (styles) 572 kB [initial] [rendered]
chunk {2} polyfills.7a0e6866a34e280f48e7.js (polyfills) 59.6 kB [initial] [rendered]
chunk {3} main.8dfd3d83f3bae347a6a4.js (main) 1.02 MB [initial] [rendered]



md5-00b0c2f6e24c195e5434b8e31f39bc03



WARNING in Invalid background value at 147:97. Ignoring.
WARNING in Invalid background value at 171:56164. Ignoring.



md5-befa16717a3fbbae6779a1fbea7fbbc6



Angular CLI: 6.0.8
Node: 8.10.0
OS: linux x64
Angular: 6.0.7



md5-24d7ce565fb239f31090503f2b33051c



      1 /browserslist
      1 gitkeep
     24 html
      1 ico
      4 jpg
      1 js
      3 json
      6 png
     24 scss
     59 ts



md5-5f8512bfe0107d834bcf67ded02ee631



Unknown option: '--sourcemaps'

and --build-optimizer=false results are no different.

[AshMcConnell]

:arrow_up: I think it has changed to --source-maps @thenslerse

[cghislai]

Recent updates greatly improved the situation here. Note that the build takes quite some RAM, ~2Go here.

I don't know whats wrong with your build, but for reference here is my config:

angular.json config

$ ./node_modules/.bin/json -D'/' -f angular.json /projects/gestemps-front/architect/build/configurations/prod
{
  "fileReplacements": [
    {
      "replace": "projects/gestemps-front/src/environments/environment.ts",
      "with": "projects/gestemps-front/src/environments/environment.prod.ts"
    }
  ],
  "optimization": true,
  "outputHashing": "all",
  "sourceMap": false,
  "extractCss": true,
  "namedChunks": false,
  "aot": true,
  "extractLicenses": true,
  "vendorChunk": false,
  "buildOptimizer": true,
  "serviceWorker": true
}

Build output

$ find projects/gestemps-front/src/ -type f | sed -e 's/.*\.//' | sort | uniq -c
      4 eot
      1 gitkeep
    212 html
      1 ico
      1 jpg
      1 json
     48 png
    248 scss
     10 svg
    667 ts
     26 ttf
      4 woff
      4 woff2
$ ng build -c prod gestemps-front

Date: 2018-07-24T14:48:59.473Z
Hash: ea9d61387d2af3ceabd9
Time: 99707ms
chunk {22} 22.b59b168f3e26e58f571f.js () 7.26 kB  [rendered]
chunk {scripts} scripts.abc521399cddc9dfff22.js (scripts) 213 kB  [rendered]
chunk {0} common.f9ace648256335434c07.js (common) 37.8 kB  [rendered]
chunk {1} 1.5499a61f5e0ceef51961.js () 32.1 kB  [rendered]
chunk {2} 2.eba1d4227d25e1530c33.js () 18.1 kB  [rendered]
chunk {3} 3.e1f9d5b9a8f8e7380574.js () 23.7 kB  [rendered]
chunk {4} 4.d42fe05fbd7e1f8813f7.js () 108 kB  [rendered]
chunk {5} 5.23850a79a8216953a8a7.js () 226 kB  [rendered]
chunk {6} 6.4177f7ded01dad458a17.js () 24.1 kB  [rendered]
chunk {7} 7.3a4dadc888d3ecf86a20.js () 43.8 kB  [rendered]
chunk {8} 8.b5f37ef38c0266a0bff2.js () 26.2 kB  [rendered]
chunk {9} 9.a9f9f788ab594d3ecfdf.js () 16.2 kB  [rendered]
chunk {10} 10.e486711476a941ac9dec.js () 96.9 kB  [rendered]
chunk {11} 11.8e127c54d67e9335e8c3.js () 30.9 kB  [rendered]
chunk {12} 12.9f4ed703f1168dccf5cf.js () 80.1 kB  [rendered]
chunk {13} 13.6db615c37ecc172c5fa6.js () 30.2 kB  [rendered]
chunk {14} 14.052b7686366bb2f26d24.js () 38.3 kB  [rendered]
chunk {15} 15.e2245a819668c46a01c4.js () 37.5 kB  [rendered]
chunk {16} 16.7c535aedcaff7253bd08.js () 106 kB  [rendered]
chunk {17} 17.06a436c11b3a72156b8d.js () 112 kB  [rendered]
chunk {18} 18.e6819dd4cc3210927d4a.js () 39.7 kB  [rendered]
chunk {19} 19.d696738f06a391d0cb80.js () 39.3 kB  [rendered]
chunk {20} 20.3e01f7381cc58596f370.js () 92.2 kB  [rendered]
chunk {21} 21.ee46112cf361a8bca156.js () 7.4 kB  [rendered]
chunk {23} 23.e23d4566a33d8f9c8885.js () 43.9 kB  [rendered]
chunk {24} 24.aec6824d1689bdd4eb64.js () 25.3 kB  [rendered]
chunk {25} 25.8be5ea48dca99238076e.js () 62.8 kB  [rendered]
chunk {26} 26.7a0050c51f531d1139fd.js () 46.1 kB  [rendered]
chunk {27} 27.f2741b48d64773848311.js () 79.1 kB  [rendered]
chunk {28} 28.f706a121c03421ba3186.js () 51.8 kB  [rendered]
chunk {29} 29.6dc6da300b9570ccca0a.js () 47.1 kB  [rendered]
chunk {30} 30.bc47c5ac2ec1d443d19a.js () 56.4 kB  [rendered]
chunk {31} 31.9d8ed35e97ae1001f6c5.js () 91.6 kB  [rendered]
chunk {32} 32.3b4c316116b309503cb6.js () 116 kB  [rendered]
chunk {33} 33.8cfe4f67375db93fa434.js () 37.6 kB  [rendered]
chunk {34} 34.6ce051529c0407f5b008.js () 129 kB  [rendered]
chunk {35} 35.ba4d73fed5e6e11a29b4.js () 6.17 kB  [rendered]
chunk {36} 36.13018565227fa5fd45f9.js () 5.8 kB  [rendered]
chunk {37} 37.c14950ef72fdab47b533.js () 264 kB  [rendered]
chunk {38} 38.99504dfac9f13b5e20e9.js () 86.8 kB  [rendered]
chunk {39} 39.0bf52b854d08e0037da5.js () 367 kB  [rendered]
chunk {40} runtime.116953fa95d6d856c696.js (runtime) 2.85 kB [entry] [rendered]
chunk {41} styles.10b82f02a98f259af35b.css (styles) 258 kB [initial] [rendered]
chunk {42} polyfills.6f9dd0fcb5982023a125.js (polyfills) 101 kB [initial] [rendered]
chunk {43} main.4d929acb0b2780c3ca2d.js (main) 2.13 MB [initial] [rendered]

Versions

Angular CLI: 6.0.8
Node: 10.6.0
OS: linux x64
Angular: 6.0.9

[Brandon-Born]

Hey folks, I had an out of memory build issue and stumbled across a fix for the slow build (for us at least).
It turns out, throwing memory at it fixed the problem. Here's our build command:

node --max_old_space_size=5048 ./node_modules/@angular/cli/bin/ng build --prod --sourcemaps false

YMMV and it's a bit hacky but I hope it helps someone out there.

[haochi]

Here's a data point for @Brandon-Born's suggestion. It decreased my project's prod build time from 50+ minutes to around 13 mins. 👍

Angular CLI: 1.6.3
Node: 8.9.4
OS: darwin x64
Angular: 5.2.1

[jpsk]

I had same issue.

The problem was in my app.scss file and the way I imported a library:

@import "../../node_modules/bootstrap/scss/bootstrap"
Time: 224909ms

vs:

@import "~bootstrap/scss/bootstrap";
Time: 19256ms

Angular CLI: 6.1.3
Node: 9.7.0
OS: darwin x64
Angular: 6.1.3

[ShadowManu]

This thread is just huge and I'm still subscribed to it just to know where are some weird culprits that blow things out of proportion without a discernible reason. Sorry for trying to invoke you @filipesilva but maybe the last comment from @jpsk can be one of the biggest culprits for the existence of this issue?

[filipesilva]

@ShadowManu I had a try at @jpsk's comment on a new project and saw no discernable difference on a prod build (11544ms vs 11724ms).

Something I know can have a big difference is the version of uglify we use, which should be improved by https://github.com/angular/angular-cli/pull/11996.

Increasing the memory limit like @Brandon-Born does is not a hack, and something you should expect to do at some point. Node processes have a default memory limit of about 1.7gigs. When a node process starts getting close to the memory limit it needs to spend more and more time doing garbage collection to free up memory, which in turn makes it run more slowly.

Bigger projects will use up more memory than smaller projects so at some point a project will hit the memory limit. I personally use a npm script for this:

"ng-high-memory": "node --max_old_space_size=8000 ./node_modules/@angular/cli/bin/ng",

It's hard to act on many of these reports since there is no objective reproduction that we can follow. It's not that we don't believe that builds can be slow. But builds can be slow for many reasons. It's exceedingly likely that the projects that are seeing the most dramatic build times are suffering from some odd problem somewhere in the pipeline that is not a general problem, and for those we really need to reproduce it to fix it.

General performance improvements are of course always helpful, and we try to do them as much as possible. But the absolute best way of getting us to fix issues, especially hard to fix ones, is to offer concrete repros.

[clabough]

Here's my latest numbers using the 6.1.5.

node --max_old_space_size=8000 ./node_modules/@angular/cli/bin/ng build --prod --source-map

_Time: 3,117,385ms (52min)_

ng build --prod --source-map

_Time: 8,416,746ms (140min)_

Hopefully the switch from uglify will really help this.

[inthegarage]

The memory change/advice bought our prod build from 48 minutes to a cool 10. just by adding the max_old_space_size variable. Good work @Brandon-Born!
Oddly we weren't seeing this on DEV's windows PCs.
I'll await the terser change too, but this has been quite a useful page it has to be said.

[clabough]

With these:

@angular-devkit/architect 0.8.1
@angular-devkit/build-angular 0.8.1
@angular-devkit/build-optimizer 0.8.1
@angular-devkit/build-webpack 0.8.1
@angular-devkit/core 0.8.1
@angular-devkit/schematics 0.8.1
@angular/cli 6.2.1
@ngtools/webpack 6.2.1
@schematics/angular 0.8.1
@schematics/update 0.8.1
rxjs 6.3.2
typescript 2.9.2
webpack 4.18.0

Running node --max_old_space_size=12000 ./node_modules/@angular/cli/bin/ng build --prod --source-map

I'm now down to Time: 470558ms (8 min).

Much better and tolerable. Seems like max_old_space_size should be configurable within tsconfig.app.json

[sivabudh]

Just wanted to add that for my company's case, it was because our CI/CD server only had 2GB of RAM. With only 2GB, the build never finished. When we asked the CI/CD service provider to bump the RAM up to 3GB to 4GB, our build finished in roughly 3 minutes.

So if you are seeing this issue on your build server, but not on your laptop, first check how much RAM you have on the build server.

[iCodr8]

@jpsk Thank you for your solution in https://github.com/angular/angular-cli/issues/6795#issuecomment-416236486

The fix was to use ~ instead of ../../node_modules/ in all of the scss project files.

In my case I had a build time from 2378409ms (~40 mins) and now it needs only 96053ms (~1,5 mins).

[robert-king]

It's very strange for me - i have had 11 minute builds for the last month.. but 2 days ago i had some 40 second builds, and now back to 11 minute builds. I didn't change many things - so no idea how i got he 40 second builds..

[kellycrowther]

All of the OSX developers on our team were experiencing very high CPU usage, and very slow ng build and ng serve times (3-8 minutes for initial build and/or recompile). Bumping node to version 10.x via nvm, removing node_modules from the project, and reinstalling with an npm install dropped initial build time to about 30 seconds and recompile to about 12 seconds. This is on rather large, angular 6 application

[hbarve1]

as by reading all the comments and seeing my own angular 6 build time, I'm concluding that UglifyJSPlugin at 92% is taking most of the time and RAM as well, it is consuming more that 2GB of RAM for just one build.

[nirdshabo]

All of the OSX developers on our team were experiencing very high CPU usage, and very slow ng build and ng serve times (3-8 minutes for initial build and/or recompile). Bumping node to version 10.x via nvm, removing node_modules from the project, and reinstalling with an npm install dropped initial build time to about 30 seconds and recompile to about 12 seconds. This is on rather large, angular 6 application

Saved my day @kellycrowther . Thanks, m8. I confirm that on os x with nodejs v10.11.0 I can see a significant improvement

[aldrindizonh]

same issue :(

Date: 2018-10-15T09:17:21.275Z
Hash: a8e9f7577a381cf027bd
Time: 338960ms
chunk {0} 0.e38e03f2d9c6fd65370c.chunk.js (common) 1.29 MB [rendered]
chunk {1} 1.35b3fdbd653e9c99e87e.chunk.js () 168 kB [rendered]
chunk {2} 2.78c04f64dba92e87b001.chunk.js () 12.7 kB [rendered]
chunk {3} 3.ef5382adf15030bf4d7f.chunk.js () 41.1 kB [rendered]
chunk {4} 4.ac5e7a11066bd3bce58a.chunk.js () 713 kB [rendered]
chunk {5} 5.261500741b47979be46e.chunk.js () 867 kB [rendered]

[DaveLomber]

Same issue on OS X,
last week it was around 10 mins but then something happened and now it's around 50 mins..
That's annoying...
Going to try what @kellycrowther proposed

[negberts]

Our build with these parameters (ng build --prod --aot=false --optimization=true --sourceMap=false --buildOptimizer=false) creates 71 bundles in just over 2 minutes with a total size of 11MB for all bundles.

Now we turn the --aot flag to true and what happens?? The build takes about 43 minutes, comes up with 84 bundles and has a total size of 22MB???? Isn't AOT supposed to minimize the size by excluding what is not being used? But instead it takes 41 minutes to create twice as many code? What's going on here?

[filipesilva]

We had a particularly nasty regression with 7.0 listed as https://github.com/angular/angular-cli/issues/12646. The main problem there should be fixed by using @angular/core@~7.0.2 and [email protected].

@negberts not sure what's happening in your project really. Is it open source, or somewhere I could look at?

[artaommahe]

Isn't AOT supposed to minimize the size by excluding what is not being used

nope, aot is about pre-compiling your templates and boosting up app/modules startup by removing runtime compiler and runtime compilation phase

[negberts]

@artaommahe that's probably what I meant. ;-) But to be honest, I have the experience that the --aot=false version is actually starting up faster.. Not very strange I think if you compare the total size of the bundles…

@filipesilva unfortunately it is not :-( But all packages are up-to-date. I've attached the build outcomes. You need anything else? Angular.json? Package.json?

ngbuildcompare.xlsx

[awerlang]

@negberts aot precompiles the HTML templates into straight forward js files which keep the view updated. These are larger than the templates themselves, so there's a point where the combined size surpasses the size of the runtime compiler.
It saves time at runtime because it's a step done in advance.

But you pointed out the bundles are so much bigger you consider the non-aot is faster to startup. Which may be true for uncached assets, plus the difference in parsing a larger file. I build into a single bundle, so your experience with so many bundles ought to be different.

I suggest you compare these options on different devices as well. Mobile and desktop, and keep the one that perform better.

[HarelM]

Since recent versions I can't use appveyor CI no more since the build runs more than an hour.
I've updated to latest version (from 6 to 7) hoping it would improve but it just got worse.
I've tried to use build-optimizer=false but the build fails (What??).
I'd love for some insights, I'm desperate...
Code can be found here:
https://github.com/IsraelHikingMap/Site
Appveyour build can be found here:
https://ci.appveyor.com/project/IsraelHikingHost/site
The relevant branch which I'm currently working on is called openlayers.
Please let me know if you need anything else.
Production build used to run for about 10 minutes in angular version 4-5 I think...
I've sent a heart breaking mail to appveyor and asked them to increase the timeout, hope they would help out, but I would really like my build to take less time, this is not a huge app...

[hahn-kev]

@HarelM 2 things you could try that seemed to help me. Upgrade @angular-devkit/build-angular you're on version 8, but version 10 is out (not sure why this isn't updated when you update the cli or anything else)

You're also mixing scss and css. Which doesn't seem like it would be a big deal, but when I converted all my css files over to scss, I saw my build time drop quite a bit.

I just tried it out on your repo and didn't see much change, but the build only took a little over a minute for me.

[HarelM]

@hahn-kev Thanks for the super quick response!
As can be seen here on the openlayers branch the devkit version is 0.11:
https://github.com/IsraelHikingMap/Site/blob/dd78814bc11b86fc66157943e79e3a3d8e85488d/IsraelHiking.Web/package.json#L60
I'll try to migrate all my css file to scss, although I have a generated one from icomoon which I don't fancy migrating every time I change it, but if it will reduce the build time I'll give it a go.

but the build only took a little over a minute for me.

Did you build it with --prod?
In any case, the appveyor servers are limited on resources and it is taking a very long time there. They were kind enough to extend the timeout to 90 minutes just now, I hope it will be enough...

[hahn-kev]

@HarelM Yeah I did build it with --prod but this is a nice desktop, so definitely faster then what you get from appveyor.

Oh ok, yeah I didn't build the openlayers branch. But I had really slow build times when using source maps when I had scss and css. But it doesn't look like you have source maps on, so maybe you're hitting a different issue?

[HarelM]

I now see that the debug builds when using the --watch flag in angular 7 are now significantly slower (about 10-15 seconds instead of 2-3). I'm considering downgrading back to 6... :-(

[HarelM]

TL;DR: use increase-memory-limit package.
After a long trial and error I got to this issue: #5618 which started to happen on my builds.
I'll post a nice solution in the issue above, but apparently allowing node to have more memory reduces the build time significantly - build that was taking 43 minutes now finishes in about 5 minutes! :-)))
Hope this will help others as well.

[jamesona]

@thenslerse, sorry to necro-tag you here, but were you by chance using semantic-ui in this project? I get the exact same two warnings in my app.

WARNING in Invalid background value at 147:97. Ignoring.
WARNING in Invalid background value at 171:56164. Ignoring.

[muthufmass]

angular universal is too slow to build, takes more than 20-30 mins to build and its not smooth to build and release. cant it be simplified ? I do the client build , then it does a server build, then again it will do a webpack to package a server.js which goes with all plugins that we add up , close to 24-30 mb single file. definitely needs a simplified way or fast build flow.

[yeahrow]

@Brandon-Born Thank you!!!

[skylarmb]

I cannot even compile a small boilerplate-based project (angular-material, a 2-3 dozen very small modules, <5k LoC total) on a 1gb ram EC2 instance, even with every possible memory intensive option disabled:

"optimization": false,
"outputHashing": "all",
"sourceMap": false,
"extractCss": false,
"namedChunks": false,
"aot": false,
"extractLicenses": false,
"vendorChunk": false,
"buildOptimizer": false

Even with all these flags disabled, on my local machine this takes about 1.2gb of ram and about 30 second to build on an i7 based machine. Unacceptable.

[alberto-chiesa]

@skylarmb Unacceptable by who? Depending upon which absolute standard? The build of Javascript applications is complex. 1GB RAM is absolutely nothing. You should expect to have build occupy a lot of resources. The issue is time, not RAM usage. 30 seconds to build, considering you are starting at least 4 processors (tsc, sass, the whole npm thing, Uglify, WebPack, and who knows what else), is perfectly acceptable if there is a way to keep the development build shorter (which usually is the case).

With JS, you are exchanging resources for flexibility and convenience.

If you want to use less RAM, use GoLang. ;)

[inthegarage]

@alberto-chiesa (trolling accepted :-)) Whilst there are no real benchmarks for such processing from the angular team, we can clearly see severe regression whilst compiling. An angular 7 project with a similar amount of files to a Angular 5 one takes 5x as long here on our development environments. It should also be noted that this is after memory and other optimizations, without those it was taking nearly 8x as long. We've gone backwards here.
Both Jenkins boxes have similar capabilities and enough resources for a standard compile.
Angular 7 is very slow in its compilation times, there's enough evidence on this post alone for that.
The unacceptable part is that development teams are being held up, boxes overloaded and it looks generally inefficient.
A couple of files as the poster suggests should take no longer than the Sass build, which for us is around 5s. Casting my mind back over the years of compilation, I don't think we are being unreasonable with our demands here.

[alberto-chiesa]

@inthegarage I'm not saying that there are no problems. I'm following this thread from the very beginning. I'm just saying that considering unacceptable that the Angular build requires more than 1 GB of RAM is naive. The issue is time: if the build is 5x slower, it's a problem (a huge one). But, if I can get the same build time throwing in more RAM, it becomes less of a problem.
Again, I don't want to troll anyone, and I'm well aware of long build times, but the build hardware target can't be "a 1gb ram EC2 instance". If you say: "build time is too slow", I'm 100% with you.
If you say "build takes too much RAM", I can just say "it would be nicer if it took less, but I really don't care that much".
I hope I have clarified this point. No trolling intended :)

[inthegarage]

@alberto-chiesa Agreed, there are some naive assumptions and when an IDE can gamefully hog 3GB+, to expect a compiler to use anything less is not realistic. All points clarified.
I also wish there was more urgency and activity with this problem. We seem to be in a "Jam tomorrow" situation. 1st it was wait for terser, second it was try angular 7, then mess about with some settings. Nothing has reduced the build times correctly and that's very frustrating.

[rafa-as]

Hi all,

After upgrade all the dependencies our compilations times are significantly increased, from ~2min to more than 25min (we have not waited so long).

Checking which dependency have the problem, we found the last version of Terser. We solve the problem fixing it to the 3.8.1.

[filipesilva]

Hey all,

There were two performance regressions recently that affected 7.x.

The first was in CLI 7.0 was affected by https://github.com/angular/angular-cli/issues/12646, fixed by https://github.com/angular/angular/pull/26734 and https://github.com/Microsoft/TypeScript/pull/28028.

Using at least [email protected], @angular/[email protected] and `@angular-devkit/[email protected] as per https://github.com/angular/angular-cli/issues/12646#issuecomment-435189691 should take care of it.

CLI 7.1 was affected by https://github.com/angular/angular-cli/issues/13102, which will be fixed by https://github.com/angular/angular-cli/pull/13309, and should be out in next patch versions of the CLI build system (@angular-devkit/[email protected]).

I think that in the past having this single issue for all performance related problems was useful, but it's not as useful anymore. More than a year has passed with a lot of new versions of Angular CLI in between then and now.

During that time a number of performance regressions came up, were fixed, and new ones came up. Having all the feedback in a single thread makes it hard to get meaningful reports, or to inform people of what versions the regressions that affect them were fixed.

At the time of writing this comment, there are 116 hidden comments:

Having so many hidden comments makes it hard for people to find information in anything but the latest comments. But on the other hand I don't think most people would go through all of the comments anyway. New users that see this thread mostly read the first and latest comments and lose things said in between.

So instead of having this mega-issue, we'll try to have current high impact performance issues pinned at the top of the issue tracker:

Generally speaking, if you are using the latest version of the CLI build system (that's @angular-devkit/build-angular and should be on your package.json) and notice a performance problem, please check the pinned issues. If you don't see anything performance related there, please open a new one.

Another thing to keep in mind is memory usage, which has a similar issue in https://github.com/angular/angular-cli/issues/5618. Bigger projects take more memory to build, and when you are close to the memory limit performance may suffer. Please bear in mind that project size is not only your source code, but also includes all of the libraries you use.

Please check my comment in https://github.com/angular/angular-cli/issues/5618#issuecomment-450151214 to see how you can increase the memory limit.

So for the reasons stated above (high volume of comments, hidden comments, hard to report and inform people) I'm also locking this issue to prevent this comment from being lost as new comments come in.

Thank you to everyone that reported and helped diagnose these performance problem. If you encounter any new ones, please open a new issue so we can give that specific regression our full attention and provide a resolution for it.