Graphback: Heap out of memory error when running tests in my local machine

[machi1990]

When running the test locally, I am getting a heap out of memory error

 RUNS   graphql-serve  packages/graphql-serve/tests/GraphbackServer.test.ts

<--- Last few GCs --->

[100208:0x4b85480]    55571 ms: Mark-sweep 2031.0 (2064.1) -> 2025.9 (2052.6) MB, 463.3 / 0.1 ms  (+ 128.7 ms in 28 steps since start of marking, biggest step 5.2 ms, walltime since start of marking 606 ms) (average mu = 0.191, current mu = 0.096) allocat[100208:0x4b85480]    56193 ms: Mark-sweep 2030.6 (2052.6) -> 2027.4 (2051.8) MB, 610.8 / 0.1 ms  (average mu = 0.112, current mu = 0.018) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x13cb519]
Security context: 0x08039c9008d1 <JSObject>
    1: new Map(aka Map) [0x8039c90a1f9](this=0x3deebcc00591 <the_hole>)
    2: ConstructFrame [pc: 0x134c7fa]
    3: createSymbolTable [0x18b36a0c4409] [/home/manyanda/Documents/Projects/aerogear/graphback/node_modules/typescript/lib/typescript.js:~12970] [pc=0x3082f17874e4](this=0x07d449cfcf79 <Object map = 0x3cf48f738d9>,0x3deebcc004b1 <undefined>)
    4: decla...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xa07f90 node::Abort() [/usr/local/bin/node]
 2: 0xa0839c node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0xb80d9e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0xb81119 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0xd2d875  [/usr/local/bin/node]
 6: 0xd2df06 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/usr/local/bin/node]
 7: 0xd3a785 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
 8: 0xd3b635 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 9: 0xd3e0ec v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
10: 0xd04cbb v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/usr/local/bin/node]
11: 0x10464be v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
12: 0x13cb519  [/usr/local/bin/node]

This was also observed in CI and fixed by https://github.com/aerogear/graphback/commit/4a16de7ea20accb8daf0b4922d95e862ab999770, and can be fixed locally using a simalar approach

NODE_OPTIONS="--max_old_space_size=4096" yarn test

I am wondering what could have happened to cause the memory increase, also should we increase the heap by default (for ci and locally) or document it the option ?

/cc @craicoverflow @wtrocki

[wtrocki]

need to update metadata library

[machi1990]

need to update metadata library

Why? You think that's where the memory increase is coming from?

[craicoverflow]

need to update metadata library

It is up to date I believe.

[craicoverflow]

I think it could be how jest tests are executed.

jest -w 1 is executed from root as one large test suite. If we use yarn workspaces to execute each package test sequentially it might not run out of memory

[craicoverflow]

I'll take a look

[machi1990]

Cool thanks.

[craicoverflow]

So jest -w 1 is equivalent to jest --runInBand - meaning it was running tests sequentially anyway. The issue must be something else. Still looking into it.

[craicoverflow]

See #1420

Since this is not a blocker, and the initial fix https://github.com/aerogear/graphback/commit/4a16de7ea20accb8daf0b4922d95e862ab999770 means master is building fine, i will leave this PR for now. There is a deeper problem in our tests that is causing some errors.