Reason: Compiling the world (on npm install) takes unreasonably long

[jaredly]

22 minutes on a 2015 MBP w/ 8 gigs of ram.
aint nobody got time for that.

$ time npm install git+https://github.com/facebook/reason
[ miles of logspam ]
npm install git+https://github.com/facebook/reason  1237.75s user 318.99s system 115% cpu 22:33.43 total

Possible remedies:

• have the npm package just be a simple js file that downloads a prebuilt release binary from github or somewhere
• have ocaml & deps prebuilt, but build reason on install (probably won't take too long?)

[jordwalke]

A couple of things:

• I think something is going on here. I likely have a far weaker laptop than you (12" macbook, basically glorified iPad), and yet it only takes me 8:50s. I did git clone https://github.com/facebook/reason.git && cd reason && time npm install, but that shouldn't matter. For some reason, your install is taking 2.5x as long, when I would expect it to be faster (any decent laptop should take around 7 minutes).
• This is compiling ocamlmerlin, as well as utop. utop is a huge part of this time (probably around 1/3 of the time). I have a github issue open for a lighterweight repl that is not as awesome, but just uses rlwrap on top of the default REPL.
• We have a strategy in mind to achieve greater parallelism while npm installing (I noticed my total CPU is only at 30% while installing). We think this can cut out 2/3 of _all_ Reason package (and OCaml dependency) install times. (This is not the most immediate thing we're going to improve, but see the next point).
• We are going to check in binaries for certain big ticket items (I'm thinking Jenga, and utop are both good candidates. I imagine this, plus parallelizing the installs, could cut the compiling Reason/Ocaml from scratch time to about 3 minutes max.
• Even though we'll be checking in binaries for big parts of the install, it's really good that we are focused on total sandboxed installs. We always need the ability to rebuild from scratch on any host without network connectivity. Some hosts do not allow precompiling binaries because of bizarre conflicts. Even if we didn't improve the install times by checking in binaries, I can't tell you how many times I've had to spend 30 minutes debugging my global environment that was screwed up by a non-sandboxed model. I would take a sandboxed model that took 30 minutes on the first install (but instant afterwards), in exchange for a universal global install that makes me spend 30 minutes debugging it. For the sandbox, I know it is guaranteed to work and have no conflicts - that's time I can spend doing something else. For global universes which tend to have conflicts, that 30 minutes is time I have to be expending energy debugging stuff. Not to minimize your concern! The time you're seeing is not consistent with what I'm seeing, and we're not even close to being done optimizing. 3 minutes max for first install is a good goal.

[yunxing]

Working on this right now. Let's set the goal to 3 minutes on a reasonable performant computer.

[chenglou]

@jaredly what if you switch to this diff instead? https://github.com/facebook/reason/commit/c061aa509a0c218ee1914ae69f25741fa7b908a1

Master might be having some trouble

[jaredly]

I think 8 minutes is still pretty annoying :/

[chenglou]

@jaredly it is, but don't forget that we're basically doing the equivalent of bundling a node.js distribution per project here. Technically we could be as fast as any node project in terms of installation speed (minus the compiling of deps of course), if we mandate to install Reason globally.

[jaredly]

ok, tried again & it took 8 minutes.

npm install git+https://github.com/facebook/reason  530.67s user 148.46s system 117% cpu 9:36.09 total

Which is still a long time. Are people against doing a reason-bin similar to jenga-bin?

[jordwalke]

I'm not against a reason-bin, but I believe we can get the installation time down to three minutes if we have people install rtop as a separate package. I think three minutes is totally reasonable, given that it would almost be _guaranteed_ to work. I mentioned earlier that I'll take an installation that reliably works at 30 minutes consistently unattended over one that occasionally makes me waste hours debugging the global, stateful environment.

[jordwalke]

Oh, and even if we do include building rtop as part of the stock install, I think the parallel installation trick could get everything down to about 4 or 5 minutes (maybe less?)

[jaredly]

@jordwalke you'd be fine w/ a reproducible one that takes 30 minutes, but our pool of potential early (and mid) adopters would be put off at anything over 5, and there are some who would ditch it after 1.
So at least our create-reason-app should probably DL a simple precompiled binary :)

[bobzhang]

Actually I think the core of reason parser can be installed less than 1 minute, its dependency on ocaml-re/menhir can be easily eliminated

[chenglou]

Stale. We're either inside BS now or prebuilt