Dune: add rules for building C-based executables

[avsm]

I have a library (ctypes) that uses install_c_headers to install its headers along with the rest of the library.

Now I need to pass in a -I to a cc invocation to locate the directory with those headers, but I'm unable to figure out the right rule for this.

Trying to use:

• %{lib:ctypes} fails as it needs a third argument of a file (but I want the directory for -I
• %{lib:ctypes:} fails as it is unable to depend on the directory
• %{dep:../../src/ctypes} fails for the same reason as above and is unable to depend on the directory

I'd like to avoid the relative path solution if possible as that makes it harder to separate the libraries in a duniverse. Is there a way to retrieve the directory into which install_c_headers puts the header files in a rule? For reference my rule is:

(rule
  (targets generated_struct_stubs%{ext_exe})
  (deps generated_struct_stubs.c)
  (action (run %{cc} -o %{targets} %{deps})))

[emillon]

That would be related to #1546 (and possibly #1185 again if we want this to resolve to the local workspace first). (see also #1585 for what depending on a directory should mean)

[avsm]

That discussion makes sense, thanks for the pointer! My specific usecase is for whatever install_c_headers maps to, so that could be one of the sections referenced in #1546.

[jeremiedimino]

Note that allowing something like (run cc ... -I %{lib:ctypes}) is not enough: dune needs to know what files cc is going to read inside the included directory.

[avsm]

Right -- I can specify the exact C header files easily enough, but we still need a way to get just the directory name for the purposes of the -I rule.

[jeremiedimino]

Internally, we can represent something that is the path + the dependency on all the .X files inside this path, maybe we need something similar here. For instance: -I %{include-path:ctypes:*.h}. It's starting to be complicated, but at the same time proper compilation command do complicated things so they are hard to describe correctly.

Another approach would be to provide a dedicated stanza for C compilation that would hide all this low-level stuff.

[rgrinberg]

I'd like to avoid the low level approach if possible. Wouldn't this issue be fixed if we simply allowed to add stubs to executables? Then, adding ctypes as a normal dependency to the exe would include its directory.

[avsm]

Hm I’m not sure that’s enough. This executable is meant to be a C program (ie run via main() and not ocaml). So we would need cc-based executable rules too

[rgrinberg]

Would it be possible to call this main function on the OCaml side?

[avsm]

Unfortunately not :-/ The goal of this rule is to compile a standalone C-based executable so we can run the main() function without any external interference. Linking against asmrun introduces many more symbols into the executable.

I think that ctypes and the mirage linkers are somewhat special cases, as they're explicitly trying to link C code using well-controlled headers, and the results are intended to be used by OCaml programs. So it's hard to see how we can avoid exposing some details of C compilation -- or having explicit stanzas for invoking the C compiler (e.g. c_executable) for a higher level view.

[rgrinberg]

Having a c_executable stanza sounds fine to me. Something simple can easily be added if we use the same rules that we use for stubs (don't expect any proper dependency analysis).

[jeremiedimino]

Agreed. c_executable seems preferable to me to low-level %{} stuff. It's higher level, describes the intent better and gives less chances to write the wrong compilation commands. Properly specifying what a compiler does is a pretty tedious task that users should avoid doing on their own...

[rgrinberg]

Perfect, then let's go for c_executable. @avsm I'll try to have something that you can test by Friday.

[Niols]

Hello. I know this issue is getting old, but is it still planned to be dealt with? I experienced a bit with (foreign_library ...) and with C files including a main in an empty OCaml executable. It suggests that Dune already has all the tools to propose a stanza (c_executable ...) (or, I guess, (foreign_executable ...) would make more sense now) and I would like to have this feature for several of my projects.

I am of course aware that I'm just whining and that I cannot doing myself; and it's very fine if this feature is not planned to be added. I was just wondering. Thank you for your time!

[jeremiedimino]

We actually discussed this at the last dev meeting. For now we decided to not push in this direction. You can find a summary of our conversation here. If you have more to add to this subject, you are welcome to do so here :)

[Niols]

I was puzzled at first, wondering how you knew my name (and why on Earth you had decided to put an accent on the á). I then realised it must have been an other Nicolas. We are everywhere!

Anyway, I do not think I have much to add to the subject, and it is a very reasonable decision not to go in that direction. It is however very pleasant to know that this has been discussed, and to get an answer so quickly. Thank you very much!

[jeremiedimino]

No problem!

Regarding the name, that makes me think that I should probably add a list of participants to future meeting notes.