Pkg.jl: "cannot merge projects" error

[KristofferC]

Testing the package "DisplayAs" gives the error

ERROR: can not merge projects
Stacktrace:
 [1] pkgerror(::String) at /workspace/srcdir/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:54
 [2] (::Pkg.Operations.var"#84#88"{Pkg.Operations.var"#93#94"{Bool,Cmd,Cmd,Nothing,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},Pkg.Types.Context,Pkg.Types.PackageSpec,String,Nothing,String})(::String) at /workspace/srcdir/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:1346
 [3] mktempdir(::Pkg.Operations.var"#84#88"{Pkg.Operations.var"#93#94"{Bool,Cmd,Cmd,Nothing,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},Pkg.Types.Context,Pkg.Types.PackageSpec,String,Nothing,String}, ::String; prefix::String) at ./file.jl:673
 [4] mktempdir(::Function, ::String) at ./file.jl:671 (repeats 2 times)

This type of error message seems similar to an assertion error so is it a bug in Pkg? If not, the error message should probably be expanded to say what is actually wrong and how to fix it. And this package passes on 1.3 so what's up with that?

[00vareladavid]

Yeah that is a pretty bad error message. It happens when the active manifest and the test manifest both have fixed nodes that disagree (for example the active manifest and the test manifest both are tracking the same UUID by path but are tracking it a different paths). Pkg has to preserve fixed nodes but it doesn't know which one to keep.

[tkf]

Most of my projects check-in test/Manifest.toml that is a full environment that includes the main environment simply via dev mode installation with relative path (path = ".."). This is possible as I am using a custom testing entry point https://github.com/tkf/Run.jl and not Pkg.test in CI. This has been working quite well especially for supporting Juila < 1.2 and using unregistered helper packages.

It would be great if I can keep using this workflow. Can Pkg just use test/Manifest.toml if all the required packages are in test/Manifest.toml?

[fredrikekre]

Pkg has to preserve fixed nodes but it doesn't know which one to keep.

IMO Manifest.toml should take precedent over test/Manifest.toml.

[00vareladavid]

Yeah, that would make sense. Probably better than erroring out

[tkf]

IMO Manifest.toml should take precedent over test/Manifest.toml.

Why not the other way around? Shouldn't project /test be a superset of project / (in terms of importable projects)? If so, shouldn't /test/Manifest.toml be used as-is, ignoring /Manifest.toml? The former is guaranteed to contain compatible dependencies while the latter may not, due to test-only dependencies.

[fredrikekre]

I mean, what we really want is https://github.com/JuliaLang/Pkg.jl/issues/1233.

[tkf]

I'm not so sure. I commented my concern in https://github.com/JuliaLang/Pkg.jl/issues/1233#issuecomment-572301060

[tkf]

FYI, I moved test/Manifest.toml to test/environments/main/Manifest.toml in most of my packages. Hopefully it works well in next NewPkgEval.

[laborg]

Is there a workaround for this? I'm getting the error in test, where the test project and the project under test depend on a local package that is tracked with the same UUID and an absolute path.
Why not bail out with this this kind of pseudo code?

# In Operations.jl line1348 do something like
if haskey(working_manifest, uuid) && ( uuid referenced entries are really not equal)  
....

[fredrikekre]

Is there a workaround for this?

Don't use thefunctionality of test/Project.tomls which is pretty beta (and documented as such) 🤷‍♂

[cscherrer]

Could this be made more clear? There's nothing I can find that says this is beta, only that functionality may change in the future. This seems to advocate this approach as the "modern" way of doing things, without any hint that it might be buggy:

https://julialang.github.io/Pkg.jl/v1/creating-packages/#Test-specific-dependencies-in-Julia-1.2-and-above-1

[rana]

Looking forward to a solution :grinning:

Using Test-specific dependencies in Julia 1.0 and 1.1 in the mean time.

[racinmat]

To me this happens even when I have completely identical Manifest.toml and test/Manifest.toml files, no paths there, everything from git or package server.

[dhanak]

To me this happens even when I have completely identical Manifest.toml and test/Manifest.toml files, no paths there, everything from git or package server.

Yes, this happens also when both the package and the test code tries to use the exact same dependency from a URL (i.e., not from a registry).

This used to work well with Julia 1.3, but it broke in 1.4, and has not been fixed since then. Pretty annoying.

[CameronBieganek]

Yes, this happens also when both the package and the test code tries to use the exact same dependency from a URL

Yup, I just ran into this again. My Project.toml and test/Project.toml are both tracking the same private github repo.

[dpinol]

I solved the issue removing the tested package from [deps] at my test/Project.toml