Dune: A bunch of warnings coming from a local opam switch with dune 2.0

[anton-trunov]

I have a project directory called /Users/anton/zilliqa/scilla/ and a local opam switch inside it (_opam subdirectory).

I ported the project to dune 2.0, but now I get a bunch of warnings for each local opam package like this one:

$ dune build --profile dev @install
File "/Users/anton/zilliqa/scilla/_opam/lib/core/core.dune", line 1, characters 0-0:
Warning: .dune files are ignored since 2.0. Reinstall the library with dune
>= 2.0 to get rid of this warning and enable support for the subsystem this
library provides.

I believe some of those libraries are not going to be updated anytime soon. Could you please suggest a fix or a workaround?

[jeremiedimino]

The library in question only needs to be reinstalled. opam reinstall core should fix this issue.

[anton-trunov]

I'm stuck with core v0.11.3 for a while, so opam reinstall core didn't work for me. Also, the project I'm working on has some more deps and some of those are using jbuilder or dune 1.x.

[jeremiedimino]

Ah, so just to be sure I get this right: the dune you are using in your shell is 2.0 but the one installed in the local opam switch is 1.x, is that right?

Indeed, that won't work. We hadn't considered this case. In theory, you should be able to install dune 2.0 in your local switch as well. Dune 2.0 is backward compatible and can build dune 1.x projects. Additionally, Dune 2.0 and jbuilder are co-installable, so having dependencies that use jbuilder shouldn't be an issue. That said, I realised yesterday that there is no version of the standalone jbuilder package that builds with OCaml 4.08 :( So if you are using OCaml 4.08, you are indeed stuck again.

I've been thinking of doing a minor release of the old jbuilder package so that we have at least one version of it compatible with 4.08.

[anton-trunov]

the dune you are using in your shell is 2.0 but the one installed in the local opam switch is 1.x, is that right?

It's the same dune 2.0.0 in both local switch and my shell:

# _opam is the local switch directory
$ _opam/bin/dune --version
11076342

$ dune --version
11076342

My guess is that jbuilder or dune in 1.x compatibility mode generate .dune files and opam gladly installs those into my local switch, which is traversed by dune, since it's in my project's directory:

.
├── _build
├── _opam
├── dune-project
├── src
├── tests
...

I've been thinking of doing a minor release of the old jbuilder package so that we have at least one version of it compatible with 4.08.

That would be indeed nice!

[jeremiedimino]

Here it is: https://github.com/ocaml/opam-repository/pull/15411

[jeremiedimino]

Actually, I'd like to do a bit more testing as I'm thinking of other ways this might fail. Could you give me some commands for me to reproduce your setup?

[anton-trunov]

Sure, anything.

Let me add some more info. Basically I have a bunch of opam switches

opam switch
#  switch                       compiler                    description
→  /Users/anton/zilliqa/scilla  ocaml-base-compiler.4.07.1
          /Users/anton/zilliqa/scilla
   4.07.1                       ocaml-base-compiler.4.07.1  4.07.1
   coq8.10                      ocaml-base-compiler.4.08.1  coq8.10
   coq8.8                       ocaml-base-compiler.4.07.1  coq8.8
   coq8.9                       ocaml-base-compiler.4.07.1  coq8.9
   coqdev4.08                   ocaml-base-compiler.4.08.1  coqdev4.08
   coqdev4.09                   ocaml-base-compiler.4.09.0  coqdev4.09

[NOTE] Current switch has been selected based on the current directory.
       The current global system switch is coqdev4.09.

The local switch can be reproduced with

git clone https://github.com/Zilliqa/scilla
cd scilla
opam switch create ./ --deps-only --with-test --yes ocaml-base-compiler.4.06.1

# if you execute `make test` you will see a bunch of warnings like the one I posted above

[jeremiedimino]

I just gave this a try but I had trouble installing secp256k1 on my system.

I'm away until next week, but here is a workaround I thought of in the meantime:

• look for the *.dune files that contains real info (most of them contains just the header). I counted only two: the one for ppx_expect and the one for ppx_inline_test
• grab the source of the corresponding package at the version they are installed, convert them to dune and pin them:
  $ opam source ppx_expect.v0.11.1` $ cd ppx_expect.v0.11.1 $ dune upgrade $ opam pin add ppx_expect .

Hopefully that should do the trick.

You can then delete all the *.dune files that contain nothing interesting to get rid of the warnings entirely

[jeremiedimino]

Second try: https://github.com/ocaml/opam-repository/pull/15427

This time I just used the original jbuilder package.

[anton-trunov]

I just learned that Dune 2.0 generates those warning even if opam packages are installed globally: https://travis-ci.com/Zilliqa/scilla/jobs/262472411

[jeremiedimino]

Yh, whether the opam switch is local or global doesn't make much difference for Dune. What matters is whether a given library is built by the dune invocation or not.

BTW, we stopped installing these files a while ago. So I'm guess these packages where installed by an old version of dune, i.e. older than 1.11.

[anton-trunov]

Could it be the case that the packages with those dune files are installed by dune 2.0, but in the 1.x (x < 11) compatibility mode?

[jeremiedimino]

Nope, the code to read or write these files is simply not there anymore. All that's left is the warning when the file is found.

[anton-trunov]

Oh, the dune files have to come from jbuilder then, I guess. Because neither reinstallation not fresh opam installation works (I checked it on my macOS machine and my coworker checked on his Ubuntu 18.04 machine)

[jeremiedimino]

Ok. In the end, most of these files are empty, so I guess we could do a minor release of jbuilder where we only generate these files if they are not empty. That would reduce the amout of warnings by a lot.

[anton-trunov]

That would be handy, thanks in advance!

[jeremiedimino]

Ok, I gave this a try. Could you try pinning jbuilder to b84f8a936308b6fdfedeedd536b2ab5eab29403b to see if it works? If yes, I'll do another release

[anton-trunov]

Worked like a charm! Thank you so much.

[jeremiedimino]

Great! I'm preparing a new minor release

[jeremiedimino]

here it is: https://github.com/ocaml/opam-repository/pull/15454

[anton-trunov]

Looks like this can be closed now. Thank you very much for your help!