Angular-cli: Out of memory exception when upgrading to v8 with ng-update

[mattlewis92]

🐞 Bug report

Command (mark with an x)

- [ ] new
- [ ] build
- [ ] serve
- [ ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [x] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Is this a regression?

No

Description

When trying to update to angular 8 I get a cryptic error message thrown when running the static query migration:

ng update @angular/cli @angular/core codelyzer --next --force --verbose
...
------ Static Query migration ------
In preparation for Ivy, developers can now explicitly specify the
timing of their queries. Read more about this here:
https://github.com/angular/angular/pull/28810

There are two available migration strategies that can be selected:
• Template strategy  -  migration tool (short-term gains, rare corrections)
• Usage strategy  -  best practices (long-term gains, manual corrections)
For an easy migration, the template strategy is recommended. The usage
strategy can be used for best practices and a code base that will be more
flexible to changes going forward.
? What migration strategy do you want to use? Template strategy

Cannot read property 'some' of undefined

Even using the verbose flag I can't seem to get a stacktrace to track down where the error is coming from, is there way of getting that so I can track down the issue and provide a minimal repro?

🔬 Minimal Reproduction

ng update @angular/cli @angular/core codelyzer --next --force --verbose

I get that this isn't enough to reproduce the issue your end, but if you can point me to how I can get a stack trace back out then I can narrow it down further to give a repro. Thanks!

🔥 Exception or Error

Cannot read property 'some' of undefined

🌍 Your Environment

Angular CLI: 8.0.0-rc.1
Node: 11.13.0
OS: darwin x64
Angular: 8.0.0-rc.1
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router, service-worker

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.800.0-rc.1
@angular-devkit/build-angular     0.800.0-rc.1
@angular-devkit/build-optimizer   0.800.0-rc.1
@angular-devkit/build-webpack     0.800.0-rc.1
@angular-devkit/core              8.0.0-rc.1
@angular-devkit/schematics        8.0.0-rc.1
@angular/cdk                      8.0.0-beta.0
@ngtools/webpack                  8.0.0-rc.1
@schematics/angular               8.0.0-rc.1
@schematics/update                0.800.0-rc.1
rxjs                              6.5.1
typescript                        3.4.5
webpack                           4.30.0

Anything else relevant?

[JoostK]

Thanks for reporting. I think the following code is causing this problem:

https://github.com/angular/angular/blob/364250e7a6a29b4961d9c3cffd23f0a6fde22e48/packages/core/schematics/migrations/static-queries/transform.ts#L31-L39

That cast to ts.ObjectLiteralExpression is not checked, so I'm guessing you have a ViewChild or ContentChild where the second parameter to the decorator is not such an object literal.

/cc @devversion

[devversion]

Looking into it. Thanks for the issue.

[devversion]

@mattlewis92 When you have a chance, can you please check if you have any ViewChild or ContentChild queries having a second parameter that is not an object literal (like Joost thinks)

That would help a lot. Thank you!

[mattlewis92]

Thanks so much for the fast reply! ❤️ I found this in our code base, removing it fixed the issue and it worked great 😄 (4000+ files changed)

@ViewChild('cropper', undefined) cropper: ImageCropperComponent;

Side note: ng update failed with an out of memory issue. Using node --max_old_space_size=8192 node_modules/.bin/ng update fixed it, and I guess anyone doing AoT builds will be used to having to use this to hack to get it to work, but it might be worth documenting this somewhere (thankfully it looks like node 12 is no longer subject to this issue as it gets configured automatically)

[clydin]

@mattlewis92 Could you share the command used to perform the update? It would be greatly preferred if the memory did not need to be configured in this regard. (And yes; Node 12 brings some nice improvements in this and many other areas).
Also, if you happen to have some time to test something for us (since you have a project that can reproduce the problem), does the memory issue happen when updating just the CLI or the framework plus CLI? (i.e., ng update @angular/cli --next or ng update @angular/cli @angular/core --next)

[devversion]

@mattlewis92 Thanks for providing that code snippet and for the feedback! :smile: In order to prevent such confusion in the future I've created a fix within the migration schematic: see: https://github.com/angular/angular/pull/30178

[mattlewis92]

@mattlewis92 Could you share the command used to perform the update? It would be greatly preferred if the memory did not need to be configured in this regard. (And yes; Node 12 brings some nice improvements in this and many other areas).
Also, if you happen to have some time to test something for us (since you have a project that can reproduce the problem), does the memory issue happen when updating just the CLI or the framework plus CLI? (i.e., ng update @angular/cli --next or ng update @angular/cli @angular/core --next)

It's the Static Query migration migration that's causing the out of memory exception (same error we get whenever doing AoT builds):

added 7 packages from 6 contributors, updated 12 packages and audited 877597 packages in 14.295s
found 2 low severity vulnerabilities
  run `npm audit fix` to fix them, or `npm audit` for details
            ------ Static Query migration ------
            In preparation for Ivy, developers can now explicitly specify the
            timing of their queries. Read more about this here:
            https://github.com/angular/angular/pull/28810

            There are two available migration strategies that can be selected:
              • Template strategy  -  migration tool (short-term gains, rare corrections)
              • Usage strategy  -  best practices (long-term gains, manual corrections)
            For an easy migration, the template strategy is recommended. The usage
            strategy can be used for best practices and a code base that will be more
            flexible to changes going forward.
? What migration strategy do you want to use? Template strategy


<--- Last few GCs --->

[85409:0x103ae5000]    81733 ms: Scavenge 1349.7 (1418.0) -> 1349.3 (1419.0) MB, 4.7 / 0.0 ms  (average mu = 0.076, current mu = 0.068) allocation failure
[85409:0x103ae5000]    81760 ms: Scavenge 1350.0 (1419.0) -> 1349.7 (1419.0) MB, 4.9 / 0.0 ms  (average mu = 0.076, current mu = 0.068) allocation failure
[85409:0x103ae5000]    81789 ms: Scavenge 1350.4 (1419.0) -> 1350.0 (1419.5) MB, 5.2 / 0.0 ms  (average mu = 0.076, current mu = 0.068) allocation failure


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x1981da9cfc7d]
Security context: 0x369b3451d9d1 <JSObject>
    1: new constructor(aka NodeObject) [0x369b27b5fdb9] [/Users/mattlewis/Code/clickup/frontend/node_modules/typescript/lib/typescript.js:~120160] [pc=0x1981dc045098](this=0x369b7d0d7ce1 <NodeObject map = 0x369b2bcd6b21>,0,892,892)
    2: ConstructFrame [pc: 0x1981da989d53]
    3: parseObjectLiteralElement(aka parseObjectLiteralElement) [0x369b477383d9] [/Us...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x1000660f9 node::Abort() [/usr/local/bin/node]
 2: 0x100066785 node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x1001805d7 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x100180578 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x100443a74 v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/bin/node]
 6: 0x1004454ff v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/bin/node]
 7: 0x100442dd0 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
 8: 0x100441b7a v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 9: 0x100440f92 v8::internal::Heap::HandleGCRequest() [/usr/local/bin/node]
10: 0x1004105ea v8::internal::StackGuard::HandleInterrupts() [/usr/local/bin/node]
11: 0x100608995 v8::internal::Runtime_StackGuard(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
12: 0x1981da9cfc7d
13: 0x1981dc045098
Abort trap: 6

Works:

ng update @angular/cli --next --force #needs --force tho to avoid the newer TS version incompatibility with angular 7
node --max_old_space_size=8192 node_modules/.bin/ng update @angular/core --next --force
node --max_old_space_size=8192 node_modules/.bin/ng update @angular/cli @angular/core codelyzer --next --force

Doesn't work (fails with out of memory error from above)

ng update @angular/cli @angular/core codelyzer --next --force
ng update @angular/core --next --force

@mattlewis92 Thanks for providing that code snippet and for the feedback! 😄 In order to prevent such confusion in the future I've created a fix within the migration schematic: see: angular/angular#30178

Thanks for the insanely fast turnaround, you rock! 👍

[devversion]

Thanks @mattlewis92 for mentioning that the standard AOT compilation also fails with the same exception. The static-query schematic actually leverages the AOT compiler. So that explains why there is a memory issue.

Also the schematic fix has been merged on the framework side. I assume we still want to keep that issue open for tracking the memory issue right? cc. @filipesilva should we rename the issue then?

[mattlewis92]

@devversion nice one, thank you! I can file a new issue for you to track if that's easier?

[filipesilva]

I don't quite follow why this issue (or a new issue) should be tracking the node OOM error. We've had many such issues in the past and they mostly boil down to the same thing: node has a default memory limit and large applications will reach it. I tried to explain it in https://github.com/angular/angular-cli/issues/5618. It's not a hack, it's part of using the node platform.

[mattlewis92]

Yeah that's a good point, AFAICT there's no way of catching this type of error and printing a nicer warning to the user and the issue will solve itself once node 12 is the minimum required version for the CLI 🤷‍♂

[clydin]

I still think it would be useful to analyze where the memory is being used in this case. Especially considering only analysis is taking place here and not code generation (and no webpack involvement). Even though the static query migration is triggering the out of memory issue, there is the possibility that there’s not much memory available before that particular migration even executes. For example, updating just the CLI on an empty project appears to use close to 300MB itself.

[filipesilva]

Yes, I agree that the analysis performed during the upgrade should always take less memory than the build, because the build does analysis+emit+all other webpack stuff.

I'm not sure if that assumptions is being broken here though. It sounds like both build and update hit the memory limit, but I didn't see anything about how much it is overstepped. @clydin if you're trying to look into that I recommend using our benchmark package (https://github.com/angular/angular-cli/tree/master/packages/angular_devkit/benchmark), it will give you peak CPU and memory usage.

[filipesilva]

We're adding a note about this problem, and the workaround, on the release blog post to increase awareness. We'll also leave it pinned for a while on the issue tracker to make sure people see it.

Another way to address this problem is to use Node 12.1. CLI 8, and the latest versions of CLI 7, will have Node 12.1 support.

[jasedwards]

I am confused if there was ever an actual solution to this? Here is what I tried with all failing:

1. update to node 12. still crashes after 2 gb
2. set --max_old_space. I set it to 20 gb but still crashed after 2gb
   Is there something else I am supposed to try? I was on version 7. The one thing that maybe causing problems is that I am using nrwl nx. so it runs @nrwl/schematics first. but it seems the mentioned steps above should still work going that route.

[filipesilva]

We thought Node 12 would automatically adjust the needed memory, but in https://github.com/angular/angular-cli/issues/13734#issuecomment-500895699 we see that it maxes out at 2gb anyway.

Are you sure you're setting the memory option correctly though? It should be node --max_old_space_size=4096 ./node_modules/@angular/cli/bin/ng update.

[jasedwards]

I was definately using the correct syntax. Interestingly enough after it failed I then ran node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng update @angular/core --from 7 --to 8 --migrate-only and it appears to be using the 8192 instead of 2gb. so I am wondering if the nrwl update kicks off a different process and that wipes out the usage of max_old_space_size? I will open an issue there. thanks.

[clydin]

@jasedwards I believe the nwrl process does indeed execute an ng update internally.

[angular-automatic-lock-bot[bot]]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{_This action has been performed automatically by a bot._}