Dune: Renaming Jbuilder to Dune - Work Required

[rgrinberg]

There's a few different places where the word jbuilder is mentioned in the source. I'll list them out and we can decide what to do about each one.

• jbuilder opam package
• (jbuild_version ...) stanza
• jbuilder binary (also in error messages)
• jbuild files
• Jbuild/Jbuilder identifiers in the source
• jbuilder in the documentation

Obviously, we can't break anything for old users and I really don't want to change source code just for the rename. So it's just a question of where we introduce aliases to the old name I think.

[rdavison]

So will the renamed app allow for a transition period where existing jbuild files will continue to work until they are formally disallowed?

[avsm]

Yes, jbuild files will definitely continue to be supported for quite some time!

[jeremiedimino]

BTW, it's currently hard to identify that a source tree is a jbuilder project. We have jbuild-workspace files, but they are more for local development. When we rename jbuilder I think it would be good to add a toplevel configuration file to mark the root of the project.

One information I'd like to put in this file as well is the git urls of the dependencies of the project. It is often convenient to create a self-contained workspace for development, so we could have:

$ jbuilder get janestreet/core

that would create a workspace containing a clone of https://github.com/janestreet/core and all its transitive dependencies.

[jeremiedimino]

Thinking about it, this file could just be another jbuild file with a project stanza:

(project
 ((name foo)
  (deps (ocaml/ocaml-re ocsigen/lwt ...))))

[jeremiedimino]

One additional thing we need to do is provide an upgrade tool. Maybe just xxxx convert where xxxx is the new name of jbuilder.

It should:

• rename all the jbuild files
• convert the (jbuild_version ...) stanzas
• create the toplevel xxxx file with the project stanza
• s/"jbuilder"/"xxxx"/g the opam files
• s/jbuilder/xxxx/g the Makefile?
• other?

If there are historical mistake that we want to fix in the jbuild files, now is the time as well.

[jeremiedimino]

Do we actually need to have a compatibility period where we support both jbuild files and the new ones? If we have a tool that can automatically convert from jbuilder to dune without failing, the interest is limited. It's only useful when mixing projects in the same workspace, but one could just run the tool on older projects when preparing the workspace.

That would allows to focus on dune and keep the code simpler.

[dra27]

If we have an automatic upgrade, why not make it that dune automatically does it if it detects a legacy jbuilder project (and no dune files)? While you could work around it, it'd be nice for bisecting to remain simple.

[jeremiedimino]

That would mean that the source tree would be modified in place though, is that OK? The upgrade tool could be a no-op on dune projects, so that you could always call it first.

[dra27]

I was thinking that would be OK, yes. Although now I'm having doubts! Maybe either the separate command (or also being able to say, for example, dune build @install --auto-upgrade) is then enough.

[jeremiedimino]

Since it's a one time operation I think it should be a separate command.

[samoht]

If there are historical mistake that we want to fix in the jbuild files, now is the time as well.

Can the conversion tool also change the format from S-expression to JSON? (I am half-serious)

[jeremiedimino]

No

[jeremiedimino]

Another thing that I'd like to change while we do the migration: rename files_recursively_in to source_files_recursively_in, since files_recursively_in only considers source files.The upgrade tool can take care of this

[jeremiedimino]

We just had a developer meeting. The repository has now been moved to ocaml/dune. For now, we are still preparing the next minor release of jbuilder and then we'll start the renaming work.

[rgrinberg]

From the developer meeting: We also decided that we'll add versioning to workspace files as well as jbuild files for 1.0

[rgrinberg]

Let's all take this opportunity to remove the ppx_deriver kind. I believe it has no use.

Also @diml has mentioned another piece of driver cruft that will have to be eliminated and migrated https://github.com/ocaml/dune/pull/445#issuecomment-360223511

[jeremiedimino]

Let's all take this opportunity to remove the ppx_deriver kind. I believe it has no use.

It's used for the META generation, so that the plugin can be dynamically loaded by the ppx_deriving driver. If we remove it, we won't be able to do this for instance:

$ ocamlfind ocamlc -package ppx_sexp_conv -c foo.ml

[rgrinberg]

Another thing to keep in mind:

migrating (fallback) to (mode fallback)

[lpw25]

Whilst doing the renaming, how about changing (jbuild_version 1.1) to (lang dune 1.1). That would be in keeping with the plugin proposal and is generally more forward compatible. I also just prefer the idea of specifying the (versioned) name of the language of the file rather than the version of the tool that interprets it.

[jeremiedimino]

I created a new ticket to discuss the changes to configuration files: #461

[bluddy]

Can the conversion tool also change the format from S-expression to JSON? (I am half-serious)

JSON is a much more well-known format, making it much easier and less intimidating for newcomers to learn. It's also more readable, IMO, than S-expressions, particularly as the parentheses pile up. Other than Jane Street's preferences, I don't see a good reason to stick with the current format.

[mjambon]

I _personally_ slightly prefer S-expressions over json for writing jbuild files. And it's irrelevant. Interoperability with other tools and with other people of this world is more important. I think json5 would be good enough for old-timers and familiar to newcomers.

[rgrinberg]

JSON would be quite awkward for some of the features we'd like to add however. For example, we'd like a simple conditional langauge: (and foo (or bar baz)). How would you suggest this would be done in json?

[bluddy]

@rgrinberg, I essentially replied to your question here. It's too confusing to have both discussions, so maybe let's use that issue? I think this is more relevant there anyway.

[rgrinberg]

I'm closing this ticket as all the work described here has been done, cancelled, or WIP. Thanks to everyone for participation.