Pkg.jl: New JLLs as stdlibs make impossible to install old JLLs (and break BinaryBuilder)

[giordano]

This loosely mimics what we do in BinaryBuilderBase.jl/src/Prefix.jl to install JLLs using the resolver for a different version of Julia:

% julia-65382c708d/bin/julia --startup-file=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _  |  |
  | | |_| | | | (_| |  |  Version 1.6.0-DEV.1673 (2020-12-05)
 _/ |\__'_|_|_|\__'_|  |  Commit 65382c708d* (0 days old master)
|__/                   |

(@v1.6) pkg> activate --temp
  Activating new environment at `/tmp/jl_j1YfIG/Project.toml`

julia> using Pkg

julia> Pkg.add("GMP_jll"; julia_version=v"1.5")
    Updating registry at `~/.julia/registries/General`
    Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Resolving package versions...
Updating `/tmp/jl_j1YfIG/Project.toml`
  [781609d7] + GMP_jll
Updating `/tmp/jl_j1YfIG/Manifest.toml`
  [56f22d72] + Artifacts
  [8f399da3] + Libdl
  [781609d7] + GMP_jll
  Progress [========================================>]  1/1
1 dependency successfully precompiled in 2 seconds

(jl_j1YfIG) pkg> st -m GMP_jll
Status `/tmp/jl_j1YfIG/Manifest.toml`
  [781609d7] GMP_jll

julia> ctx = Pkg.Types.Context();

julia> installed_jlls = [
           Pkg.Types.PackageSpec(name=p.name, uuid=u, tree_hash=p.tree_hash) for (u, p) in ctx.env.manifest if endswith(p.name, "_jll")
       ]
1-element Vector{Pkg.Types.PackageSpec}:

PackageSpec(
  name = GMP_jll
  uuid = 781609d7-10c4-51f6-84f2-b8444358ff6d
  version = *
)

julia> Pkg.Operations.source_path(ctx, installed_jlls[1])
"/tmp/mozilla_mose0/julia-65382c708d/share/julia/stdlib/v1.6/GMP_jll"

The last path points to GMP_jll in Julia v1.6. This defeats the purpose of #2031 (and having JLLs as stdlibs in the first place). CC: @staticfloat

[KristofferC]

This defeats the purpose of #2031 (and having JLLs as stdlibs in the first place).

The point of having jlls as stdlibs was to not be able to install other versions of them, I thought.

[KristofferC]

https://github.com/JuliaLang/Pkg.jl/pull/2031#issuecomment-696536314

There's another thing that Pkg uses the current running Julia version for and that is to get the list of all the stdlibs and their dependencies.

In my case I don't care about stdlibs

Can deal with the stdlib stuff later if needed

I guess we are at the "later if needed" point now :D

[fingolfin]

Just to say it explicitly here, too: a reason why this is a problem is that it makes it impossible to build certain JLLs for Julia versions before 1.6: anything that links against GMP_jll currently is set to use GMP 6.1.2, to be compatible with older Julia versions; this setup is now broken -- so those JLL recipes don't build at all; to fix them, one would have to drop the version look for GMP (and similarly for MPFR), which then would lead to results that don't work in older Julia versions.

While I look forward to the day were we can drop support for Julia versions before 1.6, I think we are not yet there by a long shot (for some of us, we won't be there before there is a new Julia LTS release).

Anyway, I think you folks are already working on this, for which I am quite grateful. Thank you!

[staticfloat]

Good news! With the combination of this PR to Pkg: https://github.com/JuliaLang/Pkg.jl/pull/2318 and this PR to BBB: https://github.com/JuliaPackaging/BinaryBuilderBase.jl/pull/97 I think we will be able to properly address all the problems here.