Flow-typed: run_def_tests.sh does not work on a fresh clone of repo.

[fabulator]

I was trying to install flow-typed locally to add a new library definitions but I could not make it work. I run following commands in git bash:

git clone [email protected]:flowtype/flow-typed.git
cd flow-typed
./run_def_tests.sh

Output was:

...
 FAIL  src\commands\__tests__\runTests-test.js (7.283s)
  ● run-tests (command) › regression 1385 › returns error code if $ExpectError test doesn't fail

    Error
      Error: ENOENT: no such file or directory, scandir 'F:\test\flow-typed\cli\.flow-bins-cache'
...

It can be solved by creating folder .flow-bins-cache manually. But I think that empty folder should be in repo so it works by default.

[AndrewSouthpaw]

runTests-test.js fails for me on a fresh clone, but not with that error, and creating an empty .flow-bins-cache didn't fix the issue, since both runTests-test.js and install-test.js need actual flow bins in order to work correctly, but those don't seem to download until we run libdef tests. Makes a tidy catch-22 for getting started.

Some options that jump out to me:

1. Stop having ./run_def_tests.sh as the most obvious entry point for contributors. It does a bunch of stuff that we generally don't care about, we just want to test a particular library. Create a different script that focuses exclusively on installing what's necessary and running the libdef tests without the cli tests.
2. Download the flow bins before running the cli tests.

Thoughts, team?

[LoganBarnett]

I recently ran into this problem as well and had to create (or recreate - it's been a minute) .flow-bins-cache. I don't have flow-bin nor flow-typed installed globally as a matter of ensuring I'm never running the wrong version for a local repo, so my workflow has been this:

1. pushd into cli.
2. Run dist/cli.js run-tests <libname>.

And that seems to do the trick. It skips all the setup stuff that ./run_def_tests.sh does (which can really hose my system - it's like if it runs out of memory once, it's done forever).

So far point 1 I'm in agreement. For point 2, I'd be okay with just stuffing the flow bins into the repo directly. It's versioned, and one less thing that can fail. But that means that ordinary consumers would have that inflicted upon them as well? That would make checked-in versions of the bins undesirable.

[AndrewSouthpaw]

I'm hesitant to check in flow bins, it's 176 MB right now. That makes it unnecessarily expensive to clone.

I'll look into creating the following functionality:

1. Easily able to run tests for a libdef without all the other stuff
2. Able to run ./run_def_tests.sh on a fresh clone and have it not create the catch 22 described above