Dune: Install package without opam-installer in Windows

[bikallem]

I am on windows using 'msvs ocaml'. Opam AFAIK can't be built with msvs tools yet and I am not sure if opam ever will be a native(msvs/mingw) executable such that we can use it in a normal windows cmd terminal. In the interim or as an alternative solution for windows, can jbuilder use ocamlfind install perhaps? Is there interest in this feature?

[dra27]

Related to #179 and #202.

You should be able to build opam-installer using MSVC OCaml. I think all my patches for opam-installer specifically are also merged.

It may be taking some time (I'm literally just finalising patches for Windows support for the internal solver introduced in opam2 beta5), but mingw/msvc native opam2 is nearly ready...

[bobot]

@dra27 Is the current state of opam-installer satisfactory for where things should be installed on windows when there is no opam? Or do you have a pointer for that ?

[dra27]

No - if you're installing by hand, you'd need to be giving --prefix

[dra27]

(as in giving jbuilder a --prefix to pass on to opam-installer)

[jeremiedimino]

At some point we should implement jbuilder install directly in jbuilder. That will make things simpler

[bobot]

@dra27 I should have been more explicit, my question was that I am currently implementing jbuilder install directly in jbuilder and I'm trying to avoid regressions by keeping the old behaviors of opam-installer (when it makes sense). The harder is to choose the good default when opam is not present/used.

What would be the right default --prefix for you?

[dra27]

@bobot - ah, I see. At the moment, having just checked the sources, the default for opam-installer is /usr/local - i.e. the default is broken, so you don't have worry about regressions!

The problem is that there is no universal default for where one might install things on Windows. The best default for prefix would be just to take the directory above where the ocaml executables are found. You could possibly add a sanity check that $(where ocamlc)/../lib is also $(ocamlc -where) (or just use $(ocamlc -where) for --libdir and $(where ocamlc)/.. for --prefix)

[bikallem]

@dra27 Ah, I just noticed the opam-installer.opam in opam repository in github. Will try to just build that package and see if it works. I assume installing opam-installer itself is a manual process, i.e. copy/rename to PATH? Which is what I did for jbuilder, i.e. copy and rename main.exe from _build folder to PATH.

@diml

At some point we should implement jbuilder install directly in jbuilder. That will make things simpler

Indeed. Building/using jbuilder on msvs windows as native executable is pretty solid. So I was a bit surprised that it depended on opam-installer to implement its install functionality. Eliminating this dependecy would make jbuilder truly self-hosting and most usable on Windows when opam is not around and/or not working correctly.

[bikallem]

@bobot Are you also trying to not depend on ocamlfind? If not then maybe jbuilder can install in the location where ocamlfind install installs artefacts. If you are looking to remove ocamlfind dependency on windows, then that is even better - less dependency is better most of the time.

Are you current working on implementing jbuilder install without opam-installer?

Edit: grammar.

[bobot]

@bikallem I think jbuilder should use whatever is present (ocamlfind, opam) to give better default, but it should not depend on anything. If ocamlfind is present, ocamlfind printconf destdir is a good default for libdir. Otherwise @dra27 paths are good default.

[rgrinberg]

At some point we should implement jbuilder install directly in jbuilder. That will make things simpler

Last time we discussed this, I thought the consensus was to vendor opam-installer itself. Which was made possibly after some recent patches. Is this what you mean? Or do you want to write opam-installer from-scratch?

[jeremiedimino]

Ah, I wasn't aware of this. @bobot told me he was looking at this, so I let him comment on this. I don't mind either way, but I would imagine that what opam-installer does is not too hard to reproduce? The advantage of reproducing it is that we could use the same framework for reporting errors, etc... It will be more integrated I think.

[bobot]

Last time we discussed this, I thought the consensus was to vendor opam-installer itself.

I didn't remember that. The main problem with this feature is the choice of the default, and I think opam-installer doesn't add enough features for adding a layer of choices.

[rgrinberg]

What kind of choices are you looking for that opam-installer doesn't
support? It could be improved.

Sure, I think having our own implementation with better errors would be
nice. But maintaining the same external interface as opam-installer will
also be good.

On Mon, Dec 18, 2017, 8:35 PM François Bobot [e-mail]
wrote:

Last time we discussed this, I thought the consensus was to vendor
opam-installer itself.

I didn't remember that. The main problem with this feature is the choice
of the default, and I think opam-installer doesn't add enough features
for adding a layer of choices.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/janestreet/jbuilder/issues/372#issuecomment-352426873,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIe-_JRTA_kVlsf1ruc8oe42b3NXepVks5tBmoXgaJpZM4RE5U0
.

[bobot]

What kind of choices are you looking for that opam-installer doesn't support? It could be improved

By choice I mean, choice of the default that depends on the environment:

• jbuilder does the detection of program and switch present (ocamlfind, opam),
• opam-installer does also the detection

Currently the default of opam-installer are broken, when installed on bare debian, bare windows, or for generating debian package (ex: libdir, /usr/local/, ... ).

We could provide all the arguments to opam-installer but in that case, the amount of code used in opam-installer is minimal compared to the amount of code used (opam library). We can't let opam-installer (with its current interface) do the detection if we want to support cases where you need the installation directory (inlined in the binary) during compilation.

Finally opam-installer could read the database of opam (to get information about opam switch) instead of using the opam binary. I prefer to have a clear cut so that we don't have problem of incompatible versions. We could write an ocaml-installer that is not directly dependent of any tool, but first the good default must be found and agreed. For that jbuilder infrastructure is sufficient.

But maintaining the same external interface as opam-installer will also be good.

Jbuilder will keep generating the .install files. It doesn't need to read them. Which interface do you have in mind?

[jeremiedimino]

Let's not bother having a separate binary for installation. If people want the exact behavior of opam-installer, they can use opam-installer.

[ArchangeGabriel]

I’m currently packaging some OCaml libs for a distro (ArchLinux), and having jbuilder install depending on opam is not really nice. My use case is that I run OCAMLFIND_DESTDIR="${pkgdir}$(ocamlfind printconf destdir)" jbuilder install in a chroot environment where there is no OPAM set, and this results in:

[ERROR] No OPAM root found at /build/.opam.
        Please run 'opam init' to initialize the state of OPAM, or specify
        '--root'.
        See 'opam init --help' for details.
opam-installer easy-format.install (exit 1)

at each build and I’m not sure whether that matters or not, so I run something like:

export OPAMROOT="${srcdir}"/opam
opam init -n

first, which then results in:

Checking for available remotes: none.
  - you won't be able to use rsync and local repositories unless you install the
    rsync command on your system.
  - you won't be able to use git repositories unless you install the git command on
    your system.
  - you won't be able to use mercurial repositories unless you install the hg
    command on your system.
  - you won't be able to use darcs repositories unless you install the darcs
    command on your system.


=-=- Fetching repository information =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[default: http]
[default: http]
[default: http]
[default: http]
[default] synchronized from https://opam.ocaml.org

=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
∗  installed base-bigarray.base
∗  installed base-threads.base
∗  installed base-unix.base
Done.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

1. To configure OPAM in the current shell session, you need to run:

      eval `opam config env`

2. To correctly configure OPAM for subsequent use, add the following
   line to your profile file (for instance ~/.bash_profile):

      . /build/ocaml-easy-format/src/opam/opam-init/init.sh > /dev/null 2> /dev/null || true

3. To avoid issues related to non-system installations of `ocamlfind`
   add the following lines to ~/.ocamlinit (create it if necessary):

      let () =
        try Topdirs.dir_directory (Sys.getenv "OCAML_TOPLEVEL_PATH")
        with Not_found -> ()
      ;;

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

which means network access, something we might frown upon in the future during make or make install steps.

I’m still able to package the libs fine, but this should be replaced by jbuilder doing the right things on its own indeed.

[bikallem]

Closing this since https://github.com/ocaml/dune/pull/941 fixes the issue.