Rescript-compiler: RFC: a stdlib with the same interface for better performance

[bobzhang]

Just landed an optimization which allows user to keep the old curried interface but gain performance improvement for free.

For example, map in stdlib

let rec map f xs = 
  match xs with 
  | [ ] -> [ ]
  | u::us -> f u :: map f us

---rewritten as -->

let rec map f xs  =
  match xs with
  | [ ] -> []
  | u::us -> f u [@bs] :: map f us
let map f xs = map (fun[@bs] x -> f x) xs

Should we patch the stdlib for better performance?
I did some benchmark, it gives 20% ~80% performance enhancement
CC @freebroccolo @chenglou

Note the interface is exactly the same, so it is totally transparent to user,
we need to justify the maintenance overhead

[chenglou]

That's quite a bit of perf for such a simple change =D
However, I'm worried that this might be a slippery slope. There are likely a ton of [@bs] tag optimization you can do in the stdlib (?) and by all mean it'll end up being a BS-specific stdlib in the future, which would suck.

[chenglou]

Is there _any_ way to fine-tune the uncurrying optimizations, aside from/including the heuristics used in the docs?

cc @jordwalke @yunxing. Jordan has a proposal that distinguishes currying from non-currying on the syntax level (I also raised such concern for other reasons: I'd like to avoid accidentally currying my function by forgetting to pass an argument, then have the type system point to the wrong location).

[bobzhang]

Good thing is bs annotated code also compiled under vanilla ocaml with the same types. User will not observe any difference since those BS don't appear in mli. But yes, there is maintainance issue.
Maybe we can write a ppx to make the path easy to apply

[ghost]

The performance improvement is pretty impressive. Really nice.

But I too worry a little bit about maintenance as far as patching the standard library is concerned.

It may be worth it still but I wonder if it would instead be possible to apply most of these transformations automatically somehow (either before or after compilation). I imagine certain cases would be difficult to handle but if you could get some analysis that worked the majority of the time then you could at least maintain a smaller set of changes.

If you had some semi-automated approach maybe you could re-export most of the stdlib in a different set of modules (perhaps in a different BS stdlib package) and just fix up a few of the edge cases manually.

[bobzhang]

I think in the middle term, we should provide an optimized stdlib, (Not just currying/uncurrying, there other issues with regard to stdlib, see #625). We need deliver more value for people who use BuckleScript, readability is great, but performance is also important, I have confidence that we can deliver better performance with reasonable effort.

About the maintenance, yes, it is tedious work, but it is dead simple, mechanical and hard to get it wrong (the compiler will catch all these errors), there is no extra work needed from the user side.

Some ideas to reduce the maintenance overhead:

[@@bs.internal.uncurry {f} ] (* an ad-hoc ppx to tell compiler wrap `f` *)
let rec map f xs  =
  match xs with
  | [ ] -> []
  | u::us -> f u :: map f us

[bobzhang]

lets do this in lambda layer

let rec map/1 f xs = 
  match xs with 
  | [ ] -> [ ]
  | u::us -> f u :: map/1 f us

let rec map/0 f xs = 
  match xs with 
  | [ ] -> [ ]
  | u::us -> f u [@bs] :: map/0 f us
let map/1 f xs = map/0 (primitive/1 f ) xs

We should also do it in the early phases, so there is a high change map/1 get inlined

[bobzhang]

for the escape analysis, if the argument is used in the same position in the self recursive function, then it is still considered non-escaped

[bobzhang]

optimizations of alias elimination will remove most aliases