Components: Update Schematics to version 8 fails with Out-of-Memory exception

[ComradeCow]

Reproduction

Steps to reproduce:

1. Start with a "large" Angular v7 workspace
2. Update @angular/core etc to v8
3. Run ng update @angular/material

"Large" is a relative term, of course. We're using a monorepo style workspace (using the Nrwl's NX tools) for our Angular Applications. We have 390 *.component.ts and *.directive.ts files.

Expected Behavior

The update to successfully run the upgrade schematics.

Actual Behavior

After it installs the @angular/material and @angular/cdk packages this is displayed to the terminal:

    ** Executing migrations for package '@angular/cdk' **

<--- Last few GCs --->

[15728:0000017D08BC7A90]   978327 ms: Mark-sweep 2016.3 (2069.9) -> 2000.2 (2069.9) MB, 772.3 / 0.0 ms  (average mu = 0.273, current mu = 0.240) allocation failure scavenge might not succeed
[15728:0000017D08BC7A90]   979306 ms: Mark-sweep 2016.2 (2069.9) -> 2000.4 (2070.4) MB, 720.2 / 0.0 ms  (average mu = 0.269, current mu = 0.264) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 00007FF600E82C16]
Security context: 0x03f554a193e1 <JSObject>
    1: createScanner [000003B48610A701] [C:\git\Daktronics-Web-Platform\node_modules\typescript\lib\typescript.js:~7066] [pc=000002E34F358EE6](this=0x01dc41080509 <Object map = 0000028D62B90EE9>,1,0x02b136100709 <false>,0,0x03ce79b0e971 <String[425]\: `.control-container {\r\n  background: linear-gradient(var(--background-standard), var(--background-standa...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20190529.132332.15728.0.001.json
Node.js report completed
 1: 00007FF60033674F napi_wrap+120927
 2: 00007FF6002DFE26 uv_loop_fork+44550
 3: 00007FF6002E0CC6 uv_loop_fork+48294
 4: 00007FF600618BCE private: void __cdecl v8::internal::Parser::ReportMessageAt(struct v8::internal::Scanner::Location,enum v8::internal::MessageTemplate,class v8::internal::AstRawString const * __ptr64,enum v8::internal::ParseErrorType) __ptr64+206
 5: 00007FF6005ECC4A public: void __cdecl v8::internal::wasm::StreamingDecoder::NotifyCompilationEnded(void) __ptr64+666
 6: 00007FF60068309C public: class v8::internal::HeapObject __cdecl v8::internal::Heap::CreateFillerObjectAt(unsigned __int64,int,enum v8::internal::ClearRecordedSlots,enum v8::internal::ClearFreedMemoryMode) __ptr64+2972
 7: 00007FF60068AE0F public: void __cdecl v8::internal::LocalEmbedderHeapTracer::NotifyV8MarkingWorklistWasEmpty(void) __ptr64+5375
 8: 00007FF600681483 public: bool __cdecl v8::internal::Heap::CollectGarbage(enum v8::internal::AllocationSpace,enum v8::internal::GarbageCollectionReason,enum v8::GCCallbackFlags) __ptr64+1235
 9: 00007FF60067FB84 public: void __cdecl v8::internal::GCTracer::AddScopeSample(enum v8::internal::GCTracer::Scope::ScopeId,double) __ptr64+1316
10: 00007FF6006C07B3 public: class v8::internal::Handle<class v8::internal::HeapObject> __cdecl v8::internal::Factory::NewFillerObject(int,bool,enum v8::internal::AllocationSpace) __ptr64+67
11: 00007FF6009E76E6 public: void __cdecl v8::internal::OptimizingCompileDispatcher::Unblock(void) __ptr64+116966
12: 00007FF600E82C16 public: static class v8::internal::Vector<char const > __cdecl v8::internal::NativesCollection<0>::GetScriptsSource(void)+662454

I uploaded the report.json to a Gist here.

This ends up running for a _long_ time. The NodeJS process continually increases memory usage until it crosses the 2GB threshold and then fails out. Unfortunately I can't share link to a reproducing repository.

I also tried running this on an old Mac laptop we have in the office (late 2011), and it failed with a similar Out of Memory error.

Since I can't upload a reproducing repo, please let me know if there's any other information that can help!

Environment

Angular CLI: 8.0.0
Node: 12.3.1
OS: win32 x64
Angular: 8.0.0
... animations, cdk, cli, common, compiler, compiler-cli, core
... forms, language-service, material, platform-browser
... platform-browser-dynamic, router

Package Version

@angular-devkit/architect 0.800.0
@angular-devkit/build-angular 0.800.0
@angular-devkit/build-ng-packagr 0.800.0
@angular-devkit/build-optimizer 0.800.0
@angular-devkit/build-webpack 0.800.0
@angular-devkit/core 8.0.0
@angular-devkit/schematics 8.0.0
@ngtools/json-schema 1.1.0
@ngtools/webpack 8.0.0
@schematics/angular 8.0.0
@schematics/update 0.800.0
ng-packagr 5.2.0
rxjs 6.5.2
typescript 3.4.5
webpack 4.30.0

[jymdman]

There are a couple of issues open in @angular/cli regarding memory issues.
Try to increase the memory limit for node like here angular/cli#14543

[ComradeCow]

Increasing the memory limit let the schematics finish out.

To save people a click, I ran this command:

node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng update @angular/material

I watched it run for about 1.5 hours before I had to leave at the end of the day. The node.exe process topped out around 5.7GB at one point, and finished with 145 files modified.

Since my issue is solved I'm fine closing out this bug - seems like this is more of an issue with angular/cli?

[intellix]

I had to do the above for @angular/cdk. It took around 10 minutes on 8gb old space and it only changed 2 files after all that compute: package.json, package-lock.json

Similar thing for @angular/material really. It almost takes a minute to scan my TS file to fix:

import { x } from '@angular/material';

to

import { x } from '@angular/material/accordion';

Not sure if it's related to the performance regression mentioned in TS 3.4 but there's definitely something weird. Felt like I needed to send it to IBM Watson to change 1 line in a couple of files of my project

[devversion]

Yeah that is definitely something we need to look into. At first glance this looks related to how the migrations leverage tslint to analyze source-files and perform the replacements.

We still use tslint because when ng update was in the early stages, the pattern for doing replacements was to use TSLint. Especially because the tslint rules could also be used within Google.

In the future.. I definitely want to switch away from tslint and just use the TypeScript compiler API. This gives us a couple of benefits (e.g. global analysis phase; files don't need to be re-checked multiple-times etc.)

[devversion]

Looked more into this and actually inspected what objects are allocated. It's definitely caused by tslint as it re-creates the whole TypeScript program for every updated source-file. This impacts performance significantly as re-creating the program w/ type checker etc. results in ~3seconds per source-file.

Additionally tslint executes all lint rules for a given source-file until there are no remaining fixes. This causes an additional slow-down. Additionally TypeScript 3.4.x seems to be currently affected by memory leaks which can cause OOM exceptions when creating a TypeScript program (e.g. see here)

[Xaz16]

Hi guys
@devversion any updates on this? It is really painful due to a big size of our project :)

[bcarson]

Chiming in with @Xaz16, I'm also running into this issue in a large project.

It's been 18 days, any updates or workarounds you can share?

Thank you @devversion, you're the best! We believe in you!! 💪🏻👍🏻

[devversion]

The workaround is to increase the memory https://github.com/angular/components/issues/16154#issuecomment-497335439 unless there are some other known issues with this? I brought this up multiple times in our team meeting but we decided that it's not a priority for us to switch away from TSLint right now as the current focus is other stuff (see README.md). I wish we could switch away from TSLint as soon as possible but it just involves a lot of work and we decided to not prioritize this right now since there is a workaround.

[bcarson]

Thank you @devversion!

[philipjfulcher]

@devversion Thank you to you and the team for working on Angular Material. :)

You're right that there's a workaround for the out of memory error. However, large projects are taking an extremely long time to migrate. I work on the large project that @bcarson mentioned. We have over 1000 components in the project, and several hundred references to @angular/material. I spent the weekend starting and restarting the Material v8 migration process and never got it to complete. It would hang up after a awhile, or run for two hours and only modify two files. I let it run for up to 6 hours but a manual search showed there were still hundreds of references to @angular/material to go.

Is there any way to reduce the time this takes? You've said the team just can't focus on moving away from tslint right now, but is there anyway to get through this long migration process for large projects?

[devversion]

@philipjfulcher You can try something that I found while investigating the performance issues caused by using tslint. It's not guaranteed that it will work but it might be worth trying for your project.

Go to your project node_modules and find the linter.js file in the tslint/ package. Inside that file you need to look for a call to updateProgram (as seen here). This one can be temporarily removed since our migrations in Angular Material are guaranteed to not generate new code that needs to be migrated by other upgrade rules. Please let me know if that helped.

[devversion]

Also as mentioned in previous posts. TypeScript v3.4 had significant performance issues and upgrading to TypeScript v3.5 before running the migrations will also help with the OOM exceptions.

[specnazzz]

I don't know if my issue is related to this issue, because I don't get the OOM exception (maybe because I already increased the --max_old_space_size option before running it), but when running ng update @angular/material I'm always stuck on the ** Executing migrations for package '@angular/cdk' ** line and was stopping it after about an hour. Based on @devversion comments, maybe it's just slow and I should have left it running longer. We have around 6000 typescript files.

@devversion I did the changes you proposed (e.g. upgrade to typescript 3.5 and removed the updateProgram call from linter.js) and run the ng update @angular/material again. Will let it run through the night to see if it completes. I'm running it on a Windows machine.

[specnazzz]

@devversion the first time around I didn't properly upgraded to typescript 3.5 so it was still using typescript 3.4 and was running for 14 hours and was still stuck on the line ** Executing migrations for package '@angular/cdk' **. The second time I properly upgraded to typescript 3.5 and it finally upgraded @angular/material "only" after 4.5 hours. Both times I removed the updateProgram function call, so I think it's safe to say that removing this function didn't help. If the issue is tslint then, tslint is adding a massive amount of overhead for changing just imports.

[devversion]

@specnazzz Yeah there is still a massive overhead with tslint as it re-runs all enabled rules once replacements for a source file have been applied. This is still a significant issue if you have ~6000 typescript files...

We actually also decided to prioritize this issue and we'll be working on switching away from tslint as soon as possible. The goal is to make the upgrade as seamless as possible. Especially for big projects!

[philipjfulcher]

@devversion Thank you for prioritizing this! I ended up writing my own migration script that I was able to use for both @angular/material and ngrx-data.

[devversion]

Here is the PR that reworks ng update to no longer use tslint. https://github.com/angular/components/pull/16433

[Hardik5Bavishi]

Step : 1 -
Open C:Usersusername %AppData%Roamingnpm

Or

Window + r and copy/paste %AppData% then find npm folder(it can be inside the roaming folder).

Step: 2 -
Copy/paste the following code into your ng.cmd:

@IF EXIST "%~dp0\node.exe" (
    "%~dp0\node.exe" --max_old_space_size=8048 "%~dp0\node_modules\@angular\cli\bin\ng" %* 
) ELSE (
    @SETLOCAL 
    @SET PATHEXT=%PATHEXT:;.JS;=;% 
    node --max_old_space_size=8048 "%~dp0\node_modules\@angular\cli\bin\ng" %* 
)

It works for me.

Thank you!

[devversion]

Closing as we reworked ng-update to no longer use tslint. I would be interested in hearing how the update now goes for big applications. Thanks!

[angular-automatic-lock-bot[bot]]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{_This action has been performed automatically by a bot._}