Flow: Memory Leak / High Memory Usage Makes Flow Unusable

[joevbruno]

I'm running into similar problems as a number of other issues have reported:

4046, #3576, #3565

Flow is consuming all available memory. I'm not getting the impossibly high numbers of 40+ GB of memory as some, but it's bring my machine to a standstill, and I'm not getting any results from flow.

I get the same result regardless of how I run flow (I've tried uninstalling all options and re-installing each individually):

• Tried uninstall / installing globally with NPM
• Tried brew install
• Tried running from ./node_modules/.bin/flow
• Tried running with flow-for-vscode (in VSCode)

I've tried versions 0.45, 0.46, and 0.47 of flow using each of the above approaches individually. Always the same results

Here's my setup:
Mac OSX Sierra 10.12.5
2.6 GHz Intel Core i5
8 GB 1600 MHz DDR3
VSCode Version 1.12.2 (1.12.2)

I'm running this in a decently sized project:

---------- Result ------------

        Physical :  83606
          Source :  67992
         Comment :  10504

Single-line comment : 2043
Block comment : 8463
Mixed : 338
Empty : 5968
To Do : 62

Number of files read : 870

---

Approximately, 100 - 200 files (of the 870) have /* @flow */ and have flow types.

Here's my flow config:

[include]

[ignore]
<PROJECT_ROOT>/uploads/.*
<PROJECT_ROOT>/tools/.*
<PROJECT_ROOT>/data/.*
<PROJECT_ROOT>/build/.*
<PROJECT_ROOT>/.storybook/.*
<PROJECT_ROOT>/.github/.*

#Also tried Ignoring node modules, which also seems to have no effect
<PROJECT_ROOT>/node_modules/*

[libs]
flow-typed
./node_modules/immutable/dist/immutable.js.flow

[options]

# Saw this in a post and tried it but doesn't seem to have any effect
sharedmemory.hash_table_pow=21

esproposal.class_static_fields=enable
esproposal.class_instance_fields=enable
esproposal.decorators=ignore

unsafe.enable_getters_and_setters=true

munge_underscores=false

# These are Webpack / Babel Aliases
module.name_mapper='^components' -> '<PROJECT_ROOT>/src/common/components'
module.name_mapper='^hocs' -> '<PROJECT_ROOT>/src/common/hocs/index.js'
module.name_mapper='^selectors' -> '<PROJECT_ROOT>/src/common/redux/selectors/index.js'
module.name_mapper='^api' -> '<PROJECT_ROOT>/src/common/api/index.js'
module.name_mapper='^actions' -> '<PROJECT_ROOT>/src/common/redux/actions/index.js'
module.name_mapper='^resources' -> '<PROJECT_ROOT>/src/common/config/resources/index.js'
module.name_mapper='^utilities' -> '<PROJECT_ROOT>/src/common/utilities/index.js'

suppress_type=$FlowIssue
suppress_type=$FlowFixMe
suppress_type=$FixMe

[version]
^0.46.0

I've tried stripping out everything from the flowconfig but the version and ignoring node_modules and I still get the memory issue.

Also tried ./node_modules/.bin/flow check --verbose-indent and don't see any errors - just a lot or processing (with flow-types libs - using "flow-typed": "2.1.2".

I'm not sure what else to try 😞

Any suggestions?

[joevbruno]

@lll000111 or anyone else have any ideas?

[0rvar]

I have the same problem, running Windows 10 and Flow 0.47.0

[frontsideair]

I started having this problem recently, as of 0.53.1 on Linux (NixOS).

[ozgunarslantas]

I also have it with 0.51.1 on macOS.

[asolove]

Is anyone having this problem able to share the codebase where it occurs? It's a little baffling because I've worked on some huge Flow codebases and not bumped into it, so there must be some specific cases other than just size that trigger it. Would be useful to start gathering cases and trying across different Flow versions, OSes, etc.

[jcready]

[frontsideair]

Since I started having this recently, I did a binary search on recent commits and pinpointed the issue to a few lines. I have no idea how this is triggered, but the problematic lines are roughly like this:

        const as = map.get(pathA[0]) || []
        const bs = map.get(pathB[0]) || []
        const cs = map.get(keyC) || []

        // as.push(val)
        map.set(pathA[0], [...as, val])
        // bs.push(val)
        map.set(pathB[0], [...bs, val])
        // cs.push(val)
        map.set(pathC, [...cs, val])

Changing this code by replacing the commented out lines with the ones directly below solved the nontermination issue. I could maybe try to corner the issue further if I have time again.

[asolove]

Huh, that's very interesting. I wonder whether adding explicit type annotations onto map or as helps?

[frontsideair]

Maybe it could, but I already worked around the issue. The bigger problem is non-termination of flow, and I don't have any idea why my use case triggered this. :/

[asolove]

Hmm, darn. Unfortunately, without a full chunk of code that can reproduce, this is going to be very hard to track down. Is it possible to do a screen share session where we try to narrow it down? I'm happy to promise/sign that I won't share anything about the code other than what we learn about this Flow issue.

[frontsideair]

I managed to corner the issue to a small function. A stack overflow exception is immediately thrown in the console. I hope this helps.

[frontsideair]

Function isn't necessary either, weird.

[asolove]

Ah, that's an interesting one. To save everyone else from loading that page and bricking their browser window, the code that triggers this is of the form:

const ys = new Map();
const y = ys.get('a');

ys.set('a', [...y]);
ys.set('a', [...y]);
ys.set('a', [...y]);

The problem goes away if you:

• Change [...y] to y.
• Call ys.set fewer than three times.
• Add explicit types to ys or y.

If you do any of these things, you get the error I would expect, which is:

6: ys.set('a', [...y]);
                   ^ property `@@iterator` of $Iterable. Property not found in possibly undefined value
6: ys.set('a', [...y]);
                   ^ undefined

Since ys.get returns V | void, directly spreading that object into an array doesn't seem like it should ever be allowed.

But it seems like Flow somehow gets into an infinite loop while checking this. Flow shouldn't get into an infinite loop, but at least in this example, it's only doing so for already-invalid code.

[frontsideair]

If I change the ys.get('a') line to ys.get('a') || [] it still doesn't terminate. And when I remove some sets Flow says there are no errors; so it's reproducible for valid code as well. In any case, we can agree that it shouldn't get into an infinite loop.

[asolove]

Ah, awesome, hadn't thought of that way to get it to reproduce validly. Thanks! Now if I can just figure out how to debug the flow server, I can try to explain why...

[barbalex]

This seems to be rather prominent in the issues. Maybe related issues found after a five minute search:

• https://github.com/facebook/flow/issues/5271
• https://github.com/facebook/flow/issues/4370
• https://github.com/facebook/flow/issues/4070
• https://github.com/facebook/flow/issues/4046
• https://github.com/facebook/flow/issues/3565
• https://github.com/facebook/flow/issues/3370
• https://github.com/facebook/flow/issues/2152

[aleclarson]

Seeing this with v0.59.0 on macOS v10.13 where the flow process takes GB of memory.

Edit: The issue for me was all=true made Flow parse every file in my node_modules directory. 😱
I filed an issue (#5421) asking for [libs] to not be affected by all=true.

[JofArnold]

If you specific the types for all args in that code does the problem go away?