Hermes: RangeError: Maximum call stack size exceeded when require/load very big json file

[dongyuwei]

https://github.com/dongyuwei/tiny_english_dictionary/blob/hermes/trie-service.js#L2
const words = require('./words_with_frequency_and_translation_and_ipa.json'); will throw error:

RangeError: Maximum call stack size exceeded, js engine:hermes

The json file size is 21M, while its data structure is very simple.

Steps to reproduce the bug:

1. yarn start
2. start an android emulator
3. yarn android

Tested with yarn 1.17.3 and nodejs 12.12.0 on Mac 10.14.5.

[dongyuwei]

Exception call stack:

[tmikov]

Thanks for reporting this. It seems like we have a problem with compiling large JSON files. We are looking into it and will post an update here.

Meanwhile, as a workaround can you parse as a string with JSON.parse()?

[dongyuwei]

Thanks for your reply, I'll try and let you know today. On road now.

[dongyuwei]

I just verified JSON.parse works. See https://github.com/dongyuwei/tiny_english_dictionary/blob/hermes/trie-service.js#L18, I use react-native-local-resource to load the stringified json txt, then use JSON.parse to decode it.

[sofiageo]

In my case I have a 14MB file which builds fine in debug mode but Hermes hangs when trying to optimize. I had to turn off hermes optimizations for the build to complete. Let me know if I can help to somehow debug this for you.

[dongyuwei]

There is a unit test to parse json: /unittests/Parser/JSONParserTest.cpp, but I don't know how to run the test.

@tmikov can you modify and run the test with my big json file words_with_frequency_and_translation_and_ipa.json? From the call stack of the exception, can't see any direct relationship with hermes.

[tmikov]

We investigated this when it was reported and we believe we understand the bug. We haven't forgotten it, and we have been workin to address it, but progress has been slow because the fix is non-trivial. There are two separate problems:

• Without optimizations, the generated code for large nested object literals consume inordinate amount of register stack, causing the stack overflow exception.
• With optimizations, the stack usage is much better, but one particular optimization pass exhibits degenerate behavior on the same kind of object literals.

The workaround is to use JSON.parse() with a string or a resource.

I will look into escalating this, so the fix can land sooner.