Dune: Cross Compilation & C Stubs

[rgrinberg]

I'm investigating porting ocaml-ssl to dune and the new configurator and @toots has brought up some good points about supporting cross compilation in https://github.com/savonet/ocaml-ssl/issues/35

In particular, how the current cross compilation scheme works:

We're using the standard autoconf paradigm, so when passing --host= option, all compilation binaries are prefixed with , i.e. gcc becomes -gcc, pkg-config becomes -pkg-config etc. In most situations this is enough to make it work. Also, autoconf knows to disable checks such as running cross-compiled binaries and such.

Which points that our cross compilation scheme is ignores pkg-config entirely.

and that our current assumptions about using ocaml's CC may be naive in the context of windows.

Sounds good. Using ocamlc configuration might be tricky for the linker, however. On windows, in particular, the linker for OCaml code is flexlink and will prolly not work as expected for straight-up C targets.

Points worth considering:

• Should we support autoconf's scheme for binary selection when x compiling?

• What's a good way to make configurator aware of x compilation? Should we pass the correct pkg-config binary to it? or should it somehow find it if it's in a cross compilation context?

• The assertions about windows & $(CC) need to be verified.

A successful port of ocaml-ssl to dune should be quite valuable. It's a widely used package, a key part of the lwt ecosystem, and a stepping stone to get any kind of uable http client running on windoze. Lastly, it will be a good test for our C stubs stack touching on cross compilation, configurator, macports, win32, etc.

cc'ing:

@diml for the first 2 points
@dra27 as our windows oracle
@avsm regarding pkg-config
@whitequark as the general x compilation expert.

[whitequark]

Using ocamlc configuration might be tricky for the linker, however. On windows, in particular, the linker for OCaml code is flexlink and will prolly not work as expected for straight-up C targets.

No, this should be fine. No OCaml libraries I know of build C-only binaries, and partial linking via $(PARTIALLD) specified in Makefile.config just works.

What's a good way to make configurator aware of x compilation? Should we pass the correct pkg-config binary to it? or should it somehow find it if it's in a cross compilation context?

Avoid prefixed binaries, those are a useless and unnecessary relic of GNU dark ages. pkg-config respects the PKG_CONFIG_PATH and PKG_CONFIG_SYSROOT_DIR environment variables, these are the proper way to configure it for cross-compilation.

[toots]

While I'm certainly a supporter of questioning the status-quo and avoiding the cathedral syndrome, I have to say the triplet-prefixed binaries seems a little more mainstream to me than a relic of GNU dark ages :-) They make sense from a system setup point of view (allows them to cohabit in the PATH of the host with the regular compiler tools without a risk of confusion) and they're definitely still being pushed in recent stacks, for instance dockcross here: https://github.com/dockcross/dockcross/blob/master/linux-arm64/Toolchain.cmake or MXE.

[whitequark]

@toots While supporting the compiler triple-prefixed binaries makes sense (after all you probably don't want to drop support for gcc), requiring all tools to use triple-prefixed binaries or expecting that by default doesn't. Pretty much every tool except GNU gcc and binutils can target multiple architectures behind a switch, and requiring the user to have several copies of these binaries (which bloat installations, get out of sync, etc) is not something dune should do.

[jeremiedimino]

What's a good way to make configurator aware of x compilation? Should we pass the correct pkg-config binary to it? or should it somehow find it if it's in a cross compilation context?

I think we should pass whatever information from Context.t that configurator needs to understand the configuration. Now that configurator is part of Dune it's easier to do.

[rgrinberg]

Okay, so it definitely seems simpler not to assume the user has multiple binaries installed unless it's necessary.

The question still remains though, where does dune get the information for which pkg-config binary or which pkg-config environment variables to set. We could just add more variables to findlib.conf?

[whitequark]

More findlib.conf variables would be very convenient for the opam-cross-* repositories.

[rgrinberg]

I see that some dune based packages already work in opam-cross-windows btw. For example, pcre:

build: [
  ["jbuilder-windows" "subst"]{pinned}
  ["env" "PKG_CONFIG_PATH=%{conf-gcc-windows:c-lib}%/pkgconfig" 
   "jbuilder-windows" "build" "--install-prefix" "windows-sysroot" "-p" "pcre" "-j" jobs "-x" "windows"]
]

@whitequark @toots remind me again what is jbuilder-windows? Why can't normal jbuilder be used?

Also curious, why PKG_CONFIG_PATH rather than PKG_CONFIG_LIBDIR? Seems like we don't want to make the host packages visible. I.e. only the packages in %{conf-gcc-windows:c-lib}%/pkgconfig should be availble

[toots]

jbuilder-windows is just jbuilder built from a fixed commit that supports the -x option switch, to differentiate it from mainstream jbuilder.

[rgrinberg]

jbuilder-windows is just jbuilder built from a fixed commit that supports the -x option switch, to differentiate it from mainstream jbuilder.

Mainstream jbuilder also supports -x, does it not?

[toots]

Maybe now :-)

[whitequark]

I believe there are still some bugs in upstream jbuilder that prevents it from being used for cross-compilation, since https://github.com/ocaml-cross/opam-cross-windows/pull/56 fails.

[whitequark]

Also curious, why PKG_CONFIG_PATH rather than PKG_CONFIG_LIBDIR?

You're right of course, it should be PKG_CONFIG_LIBDIR.

[rgrinberg]

Fairly certain that bug has been fixed. $ jbuilder -x is indeed present upstream.

Also, where do variables like this come from: %{conf-gcc-windows:host}%?

[whitequark]

@rgrinberg https://github.com/ocaml-cross/opam-cross-windows/blob/master/packages/conf-gcc-windows.1/files/generate-config.sh

[toots]

@rgrinberg Looks like mainstream jbuilder is stil missing the --install-prefix option: https://travis-ci.org/ocaml-cross/opam-cross-windows/builds/370112727

[rgrinberg]

@toots there's no longer a need for it. it's inferred from the toolchain. So -x windows implies --install-prefix windows-sysroot.

[rgrinberg]

Okay, so far we just need agree on a scheme to pass PKG_CONFIG_LIBDIR via findlib.conf. Should we agree on a way to set any env var perhaps? Or stick to just this one for now.

[whitequark]

I would say go for arbitrary environment variables.

[toots]

In my experience, being able to set arbitrary env variables would prove quite useful. Sometimes, libraries are located in weird locations or need other special env configuration just like pkg-config.

[rgrinberg]

The issues in this ticket have been addressed. And I've made a dune port of ocaml-ssl (cc @toots)

[toots]

Want to merge it to https://github.com/savonet/ocaml-ssl @rgrinberg ? Would need @smimram's approval, tho.

[rgrinberg]

@toots I made the PR against savonet: https://github.com/savonet/ocaml-ssl/pull/39

Yeah, absolutely. I think it might be interesting to consider porting liquidsoap to dune as well. But I know hard can it be to chuck out so many carefully hand written makefiles :)

[toots]

Hehe. I wouldn't be against it necessarily but my main concern is that we've had a centralized build system that allows us to configure and build liquidsoap and all of its modules at once and it's been really helpful for a lot of tasks, including CI testing. I'll think about it, tho. I could start with packing away modules that haven't been updated in a while.