Rescript-compiler: Using existing ocaml packages

[little-arhat]

Hi,

So, I've seen http://bloomberg.github.io/bucklescript/Manual.html#_build_ocaml_library_as_a_npm_package but it's unclear, how I can use existing OCaml package (say, containers or Lwt)? Should I fork them, change Makefiles and convert them to npm modules using bsc? Or there is better way?

Thanks

[bobzhang]

you don't need change the build system, only a build script would be good enough.
we have some useful flags like

bsc -bs-files *.ml *.mli

[little-arhat]

Are there any plans to automate this for users? Like, getting package from opam and automatically compile it to cmj, putting into right place, etc?

Right now, forking package doesn't look very appealing, imo.

[bobzhang]

@little-arhat you should check with @chenglou , they are working on npm-ml which might automate such process

[little-arhat]

I think the good way to handle this is to teach opam switch-specific tricks, like compiling deps with bsc. So, after installing bucklescript switch, one can then install ocaml dependencies, which would be compiled to cmj or any other format, that bucklescript can use.

[chrisdavies]

Does bsc support lwt? When I add this open Lwt.Infix in my ml files, the compiler complains:

Unbound module Lwt

I tried adding this line to my bsc call: -bs-package-include lwt and I get this:

Pacage lwt not found or lwt/lib/ocaml does not exist

(With package misspelled.)

opam install lwt tells me that lwt is already installed:

[NOTE] Package lwt is already installed (current version is 2.5.2).

Any ideas? I'd really like to be able to use the >>= operator.

[little-arhat]

@chrisdavies that's what this issue is about — currently, to use existing ocaml package, you have to compile it yourself, using bsc, create npm package and then feed the name to -bs-package-include. There is http://github.com/npm-ml community which collects forks of ocaml packages with fixes necessary to use it with bucklescript. but this solution doesn't look very effective, imo.

[bobzhang]

@chrisdavies we have some bindings for promise in case you missed it (with >>= operator): https://github.com/bloomberg/bucklescript-addons/blob/master/bindings/bs-promise/README.md
@little-arhat it is a little more complicated than that, the nodeJs module system is path sensitive (it does not work well with opam). I think Facebook reason team are working very hard to make ocaml packages available on npm, maybe we can wait for a while. But at the same time, you have a much larger sets of libraries all on npm, create bindings you need and use it immediately.

[chrisdavies]

@bobzhang Looking at the source for the link you sent, I don't see a definition for >>=, though I do see it being used. Do you know where >>= is defined?

By the way, so far, I'm really impressed with Bucklescript.

[chrisdavies]

@little-arhat Thanks. It wasn't clear to me what the answer was, even though I did see that lwt was part of the question. :) Now it's clear.

[bobzhang]

@chrisdavies https://github.com/bloomberg/bucklescript-addons/blob/master/bindings/bs-promise/src/bs_promise.mli#L33

[chrisdavies]

Interesting. Hadn't looked at the mli. It looks like >>= means something different in Bucklescript than in lwt? Looks like Bucklescript uses it as sugar on top of promises? I'm intending to use it as a monadic bind over Some/None. I'll just stick to using my own bind helper function for now.

Thanks!

[bobzhang]

(>>=) is a plain function, you can redefine it any way you like!

[jordwalke]

This issue that I created would make it trivial to use any existing OCaml package (as long as it doesn't require something like C stubs)

https://github.com/bloomberg/bucklescript/issues/638

[cchantep]

Where bsc is looking for package specified with -bs-package-include ?

[bobzhang]

It's looking for a package published on npm

[cchantep]

No way to use an ocaml pkg install using opam?

[bobzhang]

@cchantep checkout the new build system, it is really easy to convert an existing ocaml package to npm using bsb.exe

[cchantep]

Thanks for the advice. I don't see the doc about how to make an existing package available in this way. Can you indicate a link to such doc?

[cchantep]

Typically in a multi-module project, I would like to re-use some code from a server-side/ocaml module (let call it mypkg) from the UI developed in BS. For now I'm blocked when calling bsb:

Error: Package mypkg not found or mypkg/lib/ocaml does not exist or please set npm_config_prefix correctly

[coolmenu]

how about this issue ? Can i use bsb compile janes core library? thanks.

[kstarikov]

@bobzhang is there any progress on this issue?

[OvermindDL1]

/me is also curious about bringing in PPX's from opam...

[shinzui]

What's required to make progress on this? It became a huge pain point after adopting BuckleScript for backend services. I hope it can be prioritized as soon as the compiler is upgraded to support OCaml 4.06.1.

[outkine]

npm-ml has also disappeared, and npm-opam is deprecated. What is the current idiomatic way to use an opam package? https://github.com/BuckleScript/bucklescript/issues/706#issuecomment-244072958?

[outkine]

Looking at the bsb-native README, it looks like it's possible to include ocaml libs using the ocamlfind-dependencies bsconfig.js option, but only in the native builds.

[dysinger]

I'm also really wanting to share code between server (reason native) and client (reason/react)

[jordwalke]

For depending on ppx's, esy could be used if this issue is addressed. For being able to compile arbitrary opam packages using BuckleScript, that's a whole other issue which I don't know the answer to. That one seems really hard given the architectural decisions of bucklescript.

[yawaramin]

Let me recap a bit. BuckleScript is OCaml. It should be able to compile any _idiomatic_ OCaml code. If it can't, that's a bug (@bobzhang please correct me if I'm wrong).

As such, BuckleScript inherently supports using any existing OCaml package (by which I mean collection of .ml/.mli/.re/.rei). What BuckleScript does _not_ support is using _opam_ packages in an automated way. This makes sense because BuckleScript and opam are two different dependency management systems–e.g. you can't automatically reuse Bower packages as npm packages or vice-versa.

At this point, let me just emphasize that this is OK. BuckleScript is doing something different from opam and dune. It's targeting JS devs to try to onboard them to a safe, sound static type system and compiler which we all happen to think is the best around. It's not necessarily targeting the existing opam/dune community, that is already well-served by JSOO. If you want to depend heavily on opam packages, _you probably want JSOO._

Given the above situation, if you want to reuse opam packages in BuckleScript, you will need to explore custom solutions. I hate to say it, but you might even need to vendor the packages with something like git submodules. That's the situation right now.

Edit: let me just add that if you want to _share_ code between backend and frontend in the same repo, there's nothing stopping that today–just put the shared code somewhere both build systems can find them. See https://github.com/atongen/setml for a great example of this.

[bobzhang]

There are some technical challenges to support OPAM packages.
The way that JSOO works is that it generates a huge js file. BuckleScript generate one js file per module, more importantly, the NodeJS module resolution is path sensitive, so there is no obvious way without taking over the library's build system (I would be interested if you have some solutions).

[jordwalke]

As such, BuckleScript inherently supports using any existing OCaml package (by which I mean collection of .ml/.mli/.re/.rei).

@yawaramin This is not what people typically mean when they say "package" though. In fact, they don't really even mean opam package. What they usually mean is a compiled set of source files that have been organized into an output with an ocamlfind config.

@bobzhang I think the easiest path would be to somehow allow bs compiler to consume ocaml packages that were compiled with jsoo. It might not be the highest priority, but it would be good to think about longer term. If the two systems can agree on a convention for representation (or can be configured to agree) then it might be doable. BuckleScript wouldn't need to care how those were compiled, it can perhaps trust that the representation is correct given a signature file or something.

[thangngoc89]

@jordwalke from my limited knowledges of jsoo. You can't even share the same type between 2 separate build of jsoo

[hhugo]

You can't even share the same type between 2 separate build of jsoo.

I don't understand what you mean. Can you maybe give a concrete example ?

[yawaramin]

I just re-read all the comments on this issue. I think we should close it; from @bobzhang 's comments it hasn't been on the BuckleScript roadmap to support opam packages in an automated way. If someone would like to come up with a technical proposal (or even better, PR) that's great but in the meantime this is spinning in place :-)

[chenglou]

Ok yeah, let's close this for now. There are similar, easier issues we can solve to advance this one too