Compodoc: [BUG] Out-of-memory error: FATAL ERROR: invalid table size Allocation failed - JavaScript heap out of memory in v8::internal::Runtime_MapGrow

[earshinov]

Overview of the issue

Compodoc build fails in our project with the following error:

$ yarn run build-compodoc
...
 <--- Last few GCs --->
 [56:0x45df2e0]    53832 ms: Scavenge 4178.1 (4324.6) -> 4166.2 (4326.6) MB, 6.8 / 0.0 ms  (average mu = 0.998, current mu = 0.999) allocation failure 
 [56:0x45df2e0]    53866 ms: Scavenge 4180.0 (4326.6) -> 4168.2 (4328.6) MB, 6.8 / 0.0 ms  (average mu = 0.998, current mu = 0.999) allocation failure 
 [56:0x45df2e0]    53900 ms: Scavenge 4181.9 (4328.6) -> 4170.0 (4331.1) MB, 6.8 / 0.0 ms  (average mu = 0.998, current mu = 0.999) allocation failure 
 <--- JS stacktrace --->
 ==== JS stack trace =========================================
     0: ExitFrame [pc: 0x334d92e5be1d]
 Security context: 0x1a59a349e6e9 <JSObject>
     1: set [0x1a59a3491ea9](this=0x160fe02b71a1 <Map map = 0x11cd6fb84359>,0x00fc30a1afd9 <Object map = 0x317f3f9ec1>,0x3a7b5590f889 <Object map = 0x11cd6fb82571>)
     2: set(aka mapCacheSet) [0x3a9bc742c649] [/opt/grt/node_modules/lodash/lodash.js:~2203] [pc=0x334d9362f3eb](this=0x160fe02b7121 <MapCache map = 0xc193ae5a2b1>,key=0x00fc30a1afd9 <Object map = ...
 FATAL ERROR: invalid table size Allocation failed - JavaScript heap out of memory
  1: 0x8f9d10 node::Abort() [/usr/local/bin/node]
  2: 0x8f9d5c  [/usr/local/bin/node]
  3: 0xaffd0e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
  4: 0xafff44 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
  5: 0xef4152  [/usr/local/bin/node]
  6: 0x1059ca2 v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, 2>::Allocate(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [/usr/local/bin/node]
  7: 0x105ce36 v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, 2>::Rehash(v8::internal::Handle<v8::internal::OrderedHashMap>, int) [/usr/local/bin/node]
  8: 0x105d459 v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, 2>::EnsureGrowable(v8::internal::Handle<v8::internal::OrderedHashMap>) [/usr/local/bin/node]
  9: 0x11419ce v8::internal::Runtime_MapGrow(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
 10: 0x334d92e5be1d 
 Aborted (core dumped)
 error Command failed with exit code 134.
 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
 ERROR: Job failed: exit code 1

Operating System, Node.js, npm, compodoc version(s)

The build is performed on a CI server in a Docker container:

$ lsb_release -d
Description:    Ubuntu 18.04.2 LTS
$ node --version
v10.16.0

compodoc 1.1.11

Angular configuration, a package.json file in the root folder

package.json:

{
  ...
  "scripts": {
    ...
    "compodoc": "compodoc -p src/tsconfig.compodoc.json -d compodoc/ -s --port=6007",
    "build-compodoc": "node --max_old_space_size=8192--trace-gc ./node_modules/@compodoc/compodoc/bin/index-cli.js -p src/tsconfig.compodoc.json -d compodoc/"
  },
  ...
}

tsconfig.compodoc.json:

// The "files" section is commented out as per
// https://github.com/compodoc/compodoc/issues/831
{
  "extends": "../tsconfig.json",
  "compilerOptions": {
    "outDir": "../out-tsc/app",
    "baseUrl": ""
  },
  //"files": [
  //  "main.ts",
  //  "polyfills.ts"
  //],
  "exclude": [
    "test.ts",
    "**/*.spec.ts",
    "**/*.stories.ts"
  ]
}

The base tsconfig.json is pretty standard:

{
  "compileOnSave": false,
  "compilerOptions": {
    "noImplicitReturns": true,
    "outDir": "./dist/out-tsc",
    "baseUrl": "src",
    "sourceMap": true,
    "declaration": false,
    "moduleResolution": "node",
    "experimentalDecorators": true,
    "target": "es2015",
    "typeRoots": [
      "node_modules/@types"
    ],
    "lib": [
      "es2017",
      "dom"
    ],
    "module": "esnext"
  },
  "angularCompilerOptions": {
    "enableIvy": false,
    "fullTemplateTypeCheck": true,
    "disableTypeScriptVersionCheck": true,
    "strictInjectionParameters": true,
    "strictTemplates": true
  }
}

Compodoc installed globally or locally ?

Locally.

If possible sourcecode of the file where it breaks

Not applicable.

If possible your terminal logs before the error

Please see above.

Motivation for or Use Case

Our project is not small, but it isn't large either. I think that more users are likely to face this memory issue in the future, as more and more people start using compodoc.

Reproduce the error

I don't have a repro project, but I am ready to assist with testing potential fixes.

Related issues

None that I know of.

Suggest a Fix

As you can see in package.json above, I already tried to bump --max-old-space-size, but it didn't help. After I did some digging, it seems that Node.js / V8 have non-configurable memory limit for collections: https://github.com/nodejs/node/issues/14590#issuecomment-319584875. That's where "FATAL ERROR: invalid table size" and "Runtime_MapGrow" in the stack trace above come from.

Maybe you compodoc developers know where this huge Map instance resides? If so, the issue can perhaps be fixed by splitting this data structure into a number of smaller ones.

[Bnaffeti]

hello, i have the same issue :(

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had recent activity. Please file a new issue if you are encountering a similar or related problem. Thank you for your contributions.