Typescript: JavaScript heap out of memory when compiling typesafe-joi on v3.4.x

[hjkcai]

TypeScript Version:

3.4.1, 3.4.2, 3.5.0-dev.20190406

Reproduction:

https://github.com/hjkcai/typesafe-joi/tree/feature/split-files

git clone https://github.com/hjkcai/typesafe-joi.git -b feature/split-files --depth=1
yarn
yarn test

Update: the published version also has this problem. Reproduce using the published version:

mkdir typesafe-joi-issue && cd typesafe-joi-issue
yarn add typescript@next @types/node typesafe-joi joi
echo '{"compilerOptions":{"strict":true}}' > tsconfig.json
echo 'import * as Joi from "typesafe-joi"; console.log(Joi)' > index.ts
npx tsc -p .

Expected behavior:

tsc should work. It works in ts 3.3.

Actual behavior:

• tsc got out of memory when running tsc -p .
• tsserver does not response in vscode (tsserver is responsive only in the first opened file. It does not response after opening a second file. There is no useful information when setting trace to verbose, only showing TypeScript Server: canceled request with sequence number xxx)

<--- Last few GCs --->

[237:0x2a86850]    32677 ms: Mark-sweep 1375.1 (1448.2) -> 1375.1 (1448.2) MB, 781.3 / 0.0 ms  allocation failure GC in old space requested
[237:0x2a86850]    33476 ms: Mark-sweep 1375.1 (1448.2) -> 1375.1 (1432.2) MB, 798.5 / 0.0 ms  last resort GC in old space requested
[237:0x2a86850]    34299 ms: Mark-sweep 1375.1 (1432.2) -> 1375.1 (1432.2) MB, 823.4 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x1488b018427d]
    1: StubFrame [pc: 0x1488b018b394]
Security context: 0x17670b2206a9 <JSObject>
    2: getUnionType(aka getUnionType) [/mnt/c/Users/hjkcai/Projects/typesafe-joi/node_modules/typescript/lib/tsc.js:~33033] [pc=0x1488b06faf5d](this=0x20e577e022e1 <undefined>,types=0x331caa7af049 <JSArray[2]>,unionReduction=1,aliasSymbol=0x20e577e022e1 <undefined>,aliasTypeArguments=0x20e577e022e1 <undefined>)
    3: in...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x88050c [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewRawOneByteString(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::Factory::NewStringFromOneByte(v8::internal::Vector<unsigned char const>, v8::internal::PretenureFlag) [node]
 7: v8::internal::Factory::NumberToString(v8::internal::Handle<v8::internal::Object>, bool) [node]
 8: v8::internal::Runtime_NumberToStringSkipCache(int, v8::internal::Object**, v8::internal::Isolate*) [node]
 9: 0x1488b018427d
[1]    237 abort (core dumped)  npx tsc -p .

Related Issues:

I saw lots of regression issues about recursion and type inference. typesafe-joi uses lots of recursions. Maybe they are somehow related?

typesafe-joi was crashed once (#28873). @j-oliveras @ahejlsberg Please help!

[hjkcai]

I am debugging TypeScript using the tag v3.4.1. I pause the execution of tsc randomly in vscode and analyze the call stack. Here are what I know about what TypeScript is doing:

1. Look into the file src/schema/object.ts
2. Check the heritage element ObjectSchemaType<ObjectSchema, Value> of ObjectSchema
3. Check ObjectSchemaType.allow against AbstractSchema.allow
4. allow is using the type ObjectSchema
5. Back to 2 with somewhere in the type parameters substituted with ?
6. After several rounds, TypeScript finally completes checking allow
7. Start checking ObjectSchemaType.keys
8. and so on...

Seems that ts is doing type check very slowly so every time I pause the stack tells me the same thing.

[hjkcai]

Increasing heap memory limit does not help. tsc will eat up all memory. There seems a infinity loop consuming lots of memory.

node --max-old-space-size=8192 node_modules/typescript/bin/tsc -p typesafe-joi-issue

[Veetaha]

The same issue with me, the new version of tsc has ruined (overly limited) my complex type hierarchy so that the compiler crashes when tries to process it.

[nicholasrice]

Also experiencing this issue - v3.3.3 works as expected.

[teves-castro]

Same here when compiling application with large type hierarchies.
Previous versions (3.3) of tsc work fine.

[zuzusik]

After upgrading to TS 3.4.3 we started to have out of memory issues with karma 3.0.0, karma-webpack 3.0.5 and awesome-typescript-loader 5.2.1

Downgrading to TS 3.3.3 solved the issue for us

Here is the issue we had:

06:24:09 10 04 2019 10:24:10.023:DEBUG [preprocessor.sourcemap]: base64-encoded source map for /usr/src/app/test/unit/index.ts
06:24:14 
06:24:14 <--- Last few GCs --->
06:24:14 
06:24:14 [22:0x41366c0]    93441 ms: Mark-sweep 1136.6 (1442.4) -> 1136.2 (1432.9) MB, 1065.9 / 0.1 ms  allocation failure GC in old space requested
06:24:14 [22:0x41366c0]    94547 ms: Mark-sweep 1136.2 (1432.9) -> 1136.2 (1363.9) MB, 1105.6 / 0.0 ms  last resort GC in old space requested
06:24:14 [22:0x41366c0]    95623 ms: Mark-sweep 1136.2 (1363.9) -> 1136.2 (1334.4) MB, 1076.1 / 0.2 ms  last resort GC in old space requested
06:24:14 
06:24:14 
06:24:14 <--- JS stacktrace --->
06:24:14 
06:24:14 ==== JS stack trace =========================================
06:24:14 
06:24:14 Security context: 0xfecfb0a57c1 <JSObject>
06:24:14     1: toString [buffer.js:611] [bytecode=0x198b036e3f71 offset=31](this=0x28d61df02251 <Uint8Array map = 0x89f7ce11f59>,encoding=0x2bf242822d1 <undefined>,start=0x2bf242822d1 <undefined>,end=0x2bf242822d1 <undefined>)
06:24:14     2: arguments adaptor frame: 0->3
06:24:14     3: /* anonymous */(aka /* anonymous */) [/usr/src/app/node_modules/karma-webpack/lib/karma-webpack.js:324] [bytecode=0x1a6fc83e59b9 offset=...
06:24:14 
06:24:14 FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
06:24:14  1: node::Abort() [gulp]
06:24:14  2: 0x11e7fec [gulp]
06:24:14  3: v8::Utils::ReportOOMFailure(char const*, bool) [gulp]
06:24:14  4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [gulp]
06:24:14  5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [gulp]
06:24:14  6: v8::internal::Factory::NewStringFromUtf8(v8::internal::Vector<char const>, v8::internal::PretenureFlag) [gulp]
06:24:14  7: v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int) [gulp]
06:24:14  8: node::StringBytes::Encode(v8::Isolate*, char const*, unsigned long, node::encoding, v8::Local<v8::Value>*) [gulp]
06:24:14  9: 0x1207c96 [gulp]
06:24:14 10: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [gulp]
06:24:14 11: 0xb7a9ac [gulp]
06:24:14 12: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [gulp]
06:24:14 13: 0xc10cfc842fd

[RyanCavanaugh]

Getting a minimal repro

[mmiszy]

@RyanCavanaugh it seems this oneliner does it: https://github.com/Microsoft/TypeScript/issues/30831

[RyanCavanaugh]

Here's the smallest (!) repro I have

interface AbstractSchema<S, V> {
  m1<T> (v: T): SchemaType<S, Exclude<V, T>>;
  m2<T> (v: T): SchemaType<S, T>;
}

type SchemaType<S, V> = S extends object ? AnySchema<V> : never;
interface AnySchema<V> extends AnySchemaType<AnySchema<undefined>, V> { }
interface AnySchemaType<S extends AbstractSchema<any, any>, V> extends AbstractSchema<S, V> { }

[RyanCavanaugh]

@mmiszy #30831 seems unrelated; this still OOMs with the fix in #30769 applied

[ahejlsberg]

The issue is runaway recursion in isTypeIdenticalTo which is set off by code that attempts to simplify conditional types. I just put up a PR that switches the check to just use object identity.