Flow: Flow check: Subprocess(51791): fail 16Unhandled exception: SharedMem.Hash_table_full <- error when using linked packages

[villesau]

I'm are getting following error when running flow check, this means I'm not able to use 0.84.0 for typechecking our codebase properly:

$ flow check
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Subprocess(51791): fail 16Unhandled exception: SharedMem.Hash_table_full
Raised by primitive operation at file "hack/heap/sharedMem.ml", line 467, characters 58-66
Called from file "hack/heap/workerCancel.ml", line 30, characters 6-10
Re-raised at file "src/core/lwt.ml", line 3008, characters 20-29
Called from file "src/unix/lwt_main.ml", line 42, characters 8-18
Called from file "src/commands/checkCommands.ml", line 108, characters 57-132
Called from file "src/flow.ml", line 86, characters 6-34

error Command failed with exit code 110.

This might be related to what I get when running Flow in IntelliJ: https://github.com/facebook/flow/issues/5640#issuecomment-435321983
and to hanging that happens in flow-typed travis:
https://github.com/facebook/flow/issues/7108

I'm not entirely sure wether the errors started when updated to macOS Mojave or when updating to 0.84.0 Flow.

[villesau]

If it helps, here are logs that I'm getting when running flow:

flow.log:

[2018-11-02 13:46:36.787] argv=/Users/path/to/project/node_modules/flow-bin/flow-osx-v0.84.0/flow start --flowconfig-name .flowconfig --temp-dir /tmp/flow /Users/path/to/project
[2018-11-02 13:46:36.798] Initializing Server (This might take some time)
[2018-11-02 13:46:36.798] executable=/Users/path/to/project/node_modules/flow-bin/flow-osx-v0.84.0/flow
[2018-11-02 13:46:36.798] version=0.84.0
[2018-11-02 13:46:36.801] No saved state available
[2018-11-02 13:46:36.806] Parsing
[2018-11-02 13:47:02.242] File /Users/path/to/lib/smartly-redux-first-router/node_modules/node-uuid/lib/sha1-browser.js is malformed
[2018-11-02 13:47:06.174] Building package heap
[2018-11-02 13:47:06.731] Loading libraries
[2018-11-02 13:47:08.748] Resolving dependencies
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Subprocess(59418): fail 16Unhandled exception: SharedMem.Hash_table_full
Raised by primitive operation at file "hack/heap/sharedMem.ml", line 467, characters 58-66
Called from file "hack/heap/workerCancel.ml", line 30, characters 6-10
Re-raised at file "src/core/lwt.ml", line 3008, characters 20-29
Called from file "src/unix/lwt_main.ml", line 42, characters 8-18
Called from file "src/server/server.ml", line 158, characters 6-54

/private/tmp/flow/projectnameandhash.monitor_log:

Nov  2 13:46:36.725 [info] argv=/Users/path/to/project/node_modules/flow-bin/flow-osx-v0.84.0/flow start --flowconfig-name .flowconfig --temp-dir /tmp/flow /Users/path/to/project
Nov  2 13:46:36.725 [info] Creating a new Flow server
Nov  2 13:46:51.324 [info] File watcher reported 3 files changed
Nov  2 13:47:13.250 [error] Closing connection 'server #1' due to uncaught exception in read loop: End_of_file
Nov  2 13:47:13.250 [error] backtrace:
Nov  2 13:47:13.250 [error] Raised at file "map.ml", line 130, characters 10-25
Nov  2 13:47:13.250 [error] Called from file "src/unix/lwt_engine.ml", line 318, characters 8-36
Nov  2 13:47:13.250 [error]
Unable to obtain kernel buffer: Operation not permitted
usage: sudo dmesg
Nov  2 13:47:13.268 [error] Flow server (pid 59411) exited with code Unknown_error (110)
Nov  2 13:47:13.268 [info] Monitor is exiting (Dying along with server)
Nov  2 13:47:13.268 [info] Broadcasting to threads and waiting 1 second for them to exit

Maybe worth noting that smartly-redux-first-router is yarn linked package.

E: when unlinking the package, things started to work again. Maybe Flow does not properly support linked packages?

[vjpr]

I'm seeing the same thing with 0.81 and 0.85 and macOS Sierra.

I have lots of symlinks.

---

$ tail -f /private/tmp/flow

[2018-11-03 12:53:21.137] Building package heap
[2018-11-03 12:53:22.077] Loading libraries
[2018-11-03 12:53:23.027] Resolving dependencies
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Worker exited (code: 16)
Subprocess(16356): fail 16Unhandled exception: SharedMem.Hash_table_full
Raised by primitive operation at file "hack/heap/sharedMem.ml", line 467, characters 58-66
Called from file "hack/heap/workerCancel.ml", line 30, characters 6-10
Re-raised at file "src/core/lwt.ml", line 3008, characters 20-29
Called from file "src/unix/lwt_main.ml", line 42, characters 8-18
Called from file "src/server/server.ml", line 158, characters 6-54


==> /private/tmp/flow/xxx.monitor_log <==
Nov  3 12:53:27.518 [error] Closing connection 'server #1' due to uncaught exception in read loop: End_of_file
Nov  3 12:53:27.518 [error] backtrace:
Nov  3 12:53:27.518 [error] Raised at file "map.ml", line 130, characters 10-25
Nov  3 12:53:27.518 [error] Called from file "src/unix/lwt_engine.ml", line 318, characters 8-36
Nov  3 12:53:27.518 [error]
Unable to obtain kernel buffer: Operation not permitted
usage: sudo dmesg
Nov  3 12:53:27.548 [error] Flow server (pid 16354) exited with code Unknown_error (110)
Nov  3 12:53:27.548 [info] Monitor is exiting (Dying along with server)
Nov  3 12:53:27.548 [info] Broadcasting to threads and waiting 1 second for them to exit

[vjpr]

Fixed with https://github.com/facebook/flow/issues/3143#issuecomment-270946030

[options]
sharedmemory.hash_table_pow=21

[villesau]

No sure if possible, but the error message could point to that direction directly.