Rescript-compiler: Future of Bucklescript

[sigarthur]

Bucklescript is great! However Still not reached a level of maturity the seems to be production ready for the masses. There are not so subtle warts with the developer experience. These are some that I have seen mentioned on discord and between colleagues of mine since we started using bucklescript at our company internally.

• bsb -make-world - this command fills my terminal with information that isn't particularly useful to me. Bucklescript should hide this information and just tell me when its done building.

• Error Messages - the error messages kinda suck. Following improvements are huge:

  • Format the error message code/type definitions when printing them to the console and add code and type highlighting.
  • Show where the error occurs in my code, underline what is causing this error.
  • Provide suggestions on how to fix the errors
  • Syntax errors are particularly bad and type errors could be waaaay better
  • Bucklescripts competition is elm. If you want to appeal to developers to pick bucklescript the developer experience must be on par or better. Bucklescripts developer experience is bad in this regard. Inspiration can be taken from elm error messages, reasonml bettererrors error format, create-react-app redbox

• Production Mode Setting - currently theres no production mode in bucklescript that handles everything for us.

  • We have to defer to a js bundler/buildsystem and manage 2 build system (bucklescript and either webpack, rollup, gcc). This is less than ideal and given the js fatigue everyone in js land has, no one that is sane will be willing to jump ship just to manage 2 build systems. Although at the moment I think elm uses webpack for development features, but doesn't rely on it in production.
  • For bucklescript to be become as easy to use and as appealing as humanly possible it should take on some of the responsibility that is usually left to js bundler/buildsystem space. if it did it would be immediately more appealing than elm or purescript. Especially if it could intelligently automatically run our code through GCC and use GCC code splitting features and advanced mode creating the most optimized sized bundles possible it would be a show stopper.
  • Also someone mentioned on discord for bucklescript to run our code through prepack and gcc to provide code that starts up fast in the js vm and is optimized to be as few bytes as possible . This seems outlandish and unlikely but if bucklescript were to pull off would be magical and second to none.

• Useful and Developer experience additions

  • When building our code bucklescript should show the following information after every compilation:
  • Build: Success | Failure (with appropriate red or green highlighting)
  • Time to build: Float
  • Overall Build Size: Float (in kb)
  • Inspiration

[OvermindDL1]

Bucklescript should hide this information and just tell me when its done building.

I quite like all that spew... >.>

But I could see a -q added or so to quiet normal messages and only report errors.

the error messages kinda suck. Following improvements are huge:

Not a Bucklescript thing, that is an OCaml thing, you should use the BetterErrors package to make them better (could be included with bucklescript though?).

Bucklescripts competition is elm.

Eh, I'd say it is more of the React/TypeScript/PureScript audience, Elm is fairly small among it all.

We have to defer to a js bundler/buildsystem and manage 2 build system (bucklescript and either webpack, rollup, gcc). This is less than ideal and given the js fatigue everyone in js land has, no one that is sane will be willing to jump ship just to manage 2 build systems. Although at the moment I think elm uses webpack for development features, but doesn't rely on it in production.

We don't always want a bundler/buildsystem.

I personally just keep it as raw ECMAScript modules when served over http2 for example.

Also someone mentioned on discord for bucklescript to run our code through prepack and gcc to provide code that starts up fast in the js vm and is optimized to be as few bytes as possible . This seems outlandish and unlikely but if bucklescript were to pull off would be magical and second to none.

Already trivial to do, just a single extra line (well, 2 lines including the dependency) in the npm's package.json file.

I really do not think all this bundling and stuff would ever be in bucklescript's purview. Everyone has their own build systems, I just use npm scripts, and they want it to work well when imported piecemeal, to not auto-bundle and compress it all, which would cause excess code to be included when they could just keep the modules they want, nor does it handle assets packing or processing and so forth. Every project is different and this is not something that you can have reasonable defaults for that would be used by even 10% (perhaps even 1%) of users.

When building our code bucklescript should show the following information after every compilation:
Build: Success | Failure (with appropriate red or green highlighting)
Time to build: Float
Overall Build Size: Float (in kb)

Time is nice, but I already get that in my system.

Build size is entirely superfluous, after all not all modules are touched, so should it then touch and stat the files just to get their sizes, or what if you are only including certain modules, or is it of pre compiled code, or what? This is another part that I'm not sure is useful in any way either...?

_/me is not an author of bucklescript, just a user_

[sigarthur]

I quite like all that spew... >.>

Yea, i'm not sure how helpful "spew" is.

But I could see a -q added or so to quiet normal messages and only report errors.

I think leaning towards a --verbose is more pragmatic, most of that "spew" isn't helpful to people building their websites or apps.

Not a Bucklescript thing, that is an OCaml thing, you should use the BetterErrors package to make them better (could be included with bucklescript though?).

Maybe Bucklescript can bring BetterErrors into the fold or collaborate with BetterErrors.

Eh, I'd say it is more of the React/TypeScript/PureScript audience, Elm is fairly small among it all.

Arent those all competition in addition to elm? Thus gaurenteeing a better experience is equally as helpful.

We don't always want a bundler/buildsystem.
I personally just keep it as raw ECMAScript modules when served over http2 for example.

I'd wager to say that there isn't a large number of people/companies on http2 in comparison to http1.X. If you want raw modules maybe a --raw. Also not to mention raw ECMAScript modules are not supported in most browser or are behind flags. Please don't bring up safari, thats like 3.3% browser share.

I really do not think all this bundling and stuff would ever be in bucklescript's purview. Everyone has their own build systems, I just use npm scripts, and they want it to work well when imported piecemeal, to not auto-bundle and compress it all, which would cause excess code to be included when they could just keep the modules they want, nor does it handle assets packing or processing and so forth. Every project is different and this is not something that you can have reasonable defaults for that would be used by even 10% (perhaps even 1%) of users.

Everyone in Js land either uses Webpack/Rollup/Browserfify. If Bucklescript audience is Javascript folks this is a MUST. Like I said if you don't want to bundle maybe adding a --raw flag and youre a happy person once again.

Build size is entirely superfluous, after all not all modules are touched, so should it then touch and stat the files just to get their sizes, or what if you are only including certain modules, or is it of pre compiled code, or what? This is another part that I'm not sure is useful in any way either...?

Buildsize as it relates to files that are outputted, Similar to webpack build size output. Size in Javascript community is huge thing and currently every bundler/buildsystem is optimizing this the best they can. If bucklescript doesn't do this once again lagging behind.

I want bucklescript to win just as bad as anyone else and it's really fast and useful for reason. However it is hard to pitch project managers and engineering teams to use bucklescript if it adds to overhead to our build pipeline. However __replacing__ webpack in our build pipeline would be a HUGE and favorable option that would win over engineering teams and project managers because webpack is complex and difficult to configure in comparison to bucklescript. @bobzhang I hope this is seriously considered as I want to use bucklescript outside of our internal channels or weekend pet projects.

[bobzhang]

hi @sigarthur thanks for your suggestions

bsb -make-world - this command fills my terminal with information that isn't particularly useful to me.

We can have a quiet mode (#1799)

Error Messages - the error messages kinda suck. Following improvements are huge:
Show where the error occurs in my code, underline what is causing this error.

Did you use vscode , we provide a problem matcher which provide such functionality. We are working with @chenglou to absorb betterErrors upstream. (Syntax errrors and type errors are what we inherited from ocaml)

We have to defer to a js bundler/buildsystem and manage 2 build system (bucklescript and either webpack, rollup, gcc).

Since BuckleScript are mostly targeting js devs, they probably will feel more comfortable using their existing build tools, webpack or rollup, and with chrome 60 having native es6 module support, probably you don't need bundler any more during dev time. Also developing a high quality bundler is non-trivial, since you are not just bundling generated JS files from OCaml(which is easy), you also need bundle all third party external js libs, (we can vendor in rollup though #1800, but I suggest we do not bundle during dev, bucklescript support amdjs/es6 module which does not need bundle at all during dev, so you can enjoy the fast feedback loop)

Build: Success | Failure (with appropriate red or green highlighting)

You should try out vscode with our tasks.json, which fits your goal, we should document it properly though

[sigarthur]

@bobzhang
I'm not suggesting building a bundler. I'm suggesting bucklescript automatically run the JS it produces through GCC or existing bundlers with code splitting features that GCC has had for a very long time. Depending on browsers that have es modules is not realistic. Many users don't have the newest browsers and we can't have our sites just not work from one day to another. The way the javascript landscape is shaping up is Smaller + Faster is better. The only feature I imagine Bucklescript needing is automatically creating service worker file it it is using GCC underneath. I hope Bucklescript will take inspiration from ClojureScript community that is building sophisticated devtools over GCC, ESModules, and the new web api's (Service Workers) giving clojurescript developers a pit of success when creating apps.

As far as the quiet mode goes, A verbose mode that shows all that information would be better. For the average dev most of the information shown when building is rather verbose and unnecessary.

[anmonteiro]

Here's my 2cts: BuckleScript should optimize for 1 thing and 1 thing only, UNIX style. Vendoring a single solution with BS will make other issues pop up requesting different tooling. And then how to deal with mixed codebases, i.e. adopt BS incrementally when it clearly favors 1 tool over others?

IMHO bundling is a downstream concern in userland and it should remain there.

[OvermindDL1]

I think leaning towards a --verbose is more pragmatic, most of that "spew" isn't helpful to people building their websites or apps.

Perhaps, but perhaps add that to a global configuration instead defaulting to following the stock OCaml compiler as it really is just the OCaml compiler.

I'd wager to say that there isn't a large number of people/companies on http2 in comparison to http1.X. If you want raw modules maybe a --raw. Also not to mention raw ECMAScript modules are not supported in most browser or are behind flags. Please don't bring up safari, thats like 3.3% browser share.

It's not just that, I have bucklescript compile, take it's output and run two passes on it (just normal npm scripts, no build system, a single line each in package.json), one copies it to my assets output in the http2 output section, the other passes it to rollup, then closure, then outputs to assets in the http1 output section. I also have other passes running concurrently that handle images and fonts and such as well to the requisite sections, all via normal npm scripts, no build system. I absolutely would not want bucklescript changing the build style it does now (except maybe customizing the output directory to stop overlapping with the primary source in lib).

Everyone in Js land either uses Webpack/Rollup/Browserfify. If Bucklescript audience is Javascript folks this is a MUST. Like I said if you don't want to bundle maybe adding a --raw flag and youre a happy person once again.

Exactly, lots of options, don't force one of them on someone, plus there is running the modules straight on node as well. ;-)

Buildsize as it relates to files that are outputted, Similar to webpack build size output. Size in Javascript community is huge thing and currently every bundler/buildsystem is optimizing this the best they can. If bucklescript doesn't do this once again lagging behind.

That is the point of closure and so forth, not the transpiler.

However it is hard to pitch project managers and engineering teams to use bucklescript if it adds to overhead to our build pipeline.

Uh, what overhead? I literally add a single line...
https://github.com/OvermindDL1/overbots/blob/master/package.json#L24

However replacing webpack in our build pipeline would be a HUGE and favorable option that would win over engineering teams and project managers because webpack is complex and difficult to configure in comparison to bucklescript.

Those are entirely orthogonal. I could never see the bucklescript build system naturally handling the variety of other javascript parsers, closure, image minizers and packers, html packers, etc... etc... Absolutely not!

I'm suggesting bucklescript automatically run the JS it produces through GCC or existing bundlers with code splitting features that GCC has had for a very long time.

Then let the rest of your build system handle that. I just call them directly, even on a single line in a package.json script entry, it is not a hard thing and every project will be different. Like this absolutely would not work with the system at my place of work.

Here's my 2cts: BuckleScript should optimize for 1 thing and 1 thing only, UNIX style. Vendoring a single solution with BS will make other issues pop up requesting different tooling. And then how to deal with mixed codebases, i.e. adopt BS incrementally when it clearly favors 1 tool over others?

Exactly this, closure is not even the only option out there. The OCaml compiler follows standard unix philosophy as those are trivially composable, as rollup also follows among others, even closure, hence why you do not need a build system like webpack or so for that (I really dislike webpack, I can do more work in less code and faster than webpack...).

IMHO bundling is a downstream concern in userland and it should remain there.

Precisely.

[sigarthur]

@bobzhang @OvermindDL1 @anmonteiro

There are 2 types of people that use Bucklescript:

1. People that are using Reason/OCaml in an existing javascript project
2. People that are starting a brand new Project in Reason/OCaml without dependencies on javascript build tools.

I agree with you all that people that fall into the number one should indeed integrate with their existing build step.

HOWEVER

People that fall under number 2 likely are doing so because the want to take advantage of OCaml speed (Bucklescript vs Babel+Webpack) and easy setup (Bucklescript vs Babel Plugins/Presets+ Webpack)

Maybe a bundler shouldn't be included into Bucklescript directly, but maybe a standalone bundler that uses gcc underneath and consumes build artifacts that Bucklescript produces that will be useful to gcc in order to create slim and fast bundles and leverages gcc code splitting feature whilst avoiding developers from configuring gcc themselves (due to gcc history of being hard to configure with advanced mode and intelligent codesplits) is useful to these people. I know it would be for me and the ability to use Bucklescript at work. I'm only arguing for people in category 2.

[Pauan]

@sigarthur I agree. I've used all of the JS tools extensively (Webpack, Closure, Rollup, Babel, TypeScript, etc.), but I really like how easy it is to setup and use BuckleScript.

Bundling is such a necessary tool that I think BuckleScript should have great bundling support out of the box (even if that just means a thin wrapper over an existing bundler).

Not having to deal with bundler configuration would be a huge advantage over the JavaScript ecosystem.

Having watch mode Just Work(tm) with the BuckleScript compiler and bundler would also be a huge advantage. Right now you have to run Webpack watch mode and BuckleScript watch mode in parallel, which is a bit annoying (in addition to the extra configuration and setup).

Of course if somebody has special needs, they can choose to not use BuckleScript's built-in bundler and instead use whatever bundler they want.

[bassjacob]

If you're using webpack, why not just use bs-loader, which abstracts everything away for you?

[kennetpostigo]

@bassjacob I think @sigarthur is saying that is what people who fall under "category 1" should do. But what about people in "category 2"? I agree with this, so long as it's a separate module and not built into bucklescript itself.

[bassjacob]

Ah, understood. I think webpack+bs-loader answers this problem regardless (both for cat1 and cat2) in terms of effort against payoff. We could ship a simple lib that just wraps webpack and bs-loader directly and it would be 99% of what's being asked for here, right? But if there's enough community interest in an alternative, then sure, why not? 😄

[Pauan]

@bassjacob I was unaware of bs-loader. I think it would be good if bs-loader was officially supported by BuckleScript and was mentioned in the docs.

Of course it would still be a separate npm package, so people can choose to not use it if they want.

Having a simple wrapper for it is not a bad idea either, because it simplifies the common case.

Having bs-loader be officially supported and easy to use would solve my personal gripes.

[bassjacob]

makes total sense 😄 as far as I know, bs-loader has only recently been considered stable (@rrdelaney, is it considered stable enough for wider promition?) If it is, would be good to mention it in both the bucklescript and the reason docs as a good way to go.

[rrdelaney]

All outstanding issues with bs-loader have been closed for two weeks now, and no new ones have been reported for a while. I also worked on the perf a lot a couple weeks ago so it's not slow, so I'd say it's pretty production ready 🙂

[OvermindDL1]

People that are using Reason/OCaml in an existing javascript project
People that are starting a brand new Project in Reason/OCaml without dependencies on javascript build tools.

I do both, routinely actually. :-)

Maybe a bundler shouldn't be included into Bucklescript directly, but maybe a standalone bundler that uses gcc underneath and consumes build artifacts that Bucklescript produces that will be useful to gcc in order to create slim and fast bundles and leverages gcc code splitting feature whilst avoiding developers from configuring gcc themselves (due to gcc history of being hard to configure with advanced mode and intelligent codesplits) is useful to these people. I know it would be for me and the ability to use Bucklescript at work. I'm only arguing for people in category 2.

That all sounds to be like just a template, why not make a template for such a setup? It seems the perfect use-case for it. Certainly not built into the compiler though, but as a template, certainly.

I do the above in, say, my overbots tutorial project, all via npm scripts, no webpack or anything utterly stupid like webpack, and that style could easily be a template.

But it absolutely should not be built in to the build system.

[DemiMarie]

@sigarthur @OvermindDL1 Here is my suggestion (as an outsider):

1. Have the basic compiler executable not do any bundling. It should, however, provide whatever hooks are needed to make it possible.
2. Provide a wrapper tool, which uses the basic compiler (either by invoking it as a subprocess, or by calling it as a library), and which does provide these high level features.

Both should be officially documented and supported.

[bobzhang]

some of those issues are already addressed. I will close this issue since it is not actionable. you are encouraged to discuss more in discord.gg/reasonml or https://discuss.ocaml.org/