Pkg.jl: Precompilation time when using Pkg3 as a package is dramatically increased

[KristofferC]

0.6.2 no Pkg3

julia> @time using JuliaDB
INFO: Precompiling module JuliaDB.
 58.453762 seconds (1.10 M allocations: 60.739 MiB, 0.46% gc time)

0.6.2 Pkg3

julia> @time using JuliaDB
INFO: Recompiling stale cache file /Users/kristoffer/.julia/lib/v0.6/JuliaDB.ji for module JuliaDB.
239.210437 seconds (7.44 M allocations: 376.640 MiB, 0.11% gc time)

This unfortunately makes using Pkg3 not very pleasant.

[AshtonSBradley]

another comparison (not quite so unpleasant):
0.6.2 Pkg3

julia> @time using JuliaDB
INFO: Precompiling module JuliaDB.
141.647968 seconds (2.25 M allocations: 105.205 MiB, 0.04% gc time)

0.6.2 no Pkg3

julia> @time using JuliaDB
INFO: Recompiling stale cache file /home/abradley/.julia/lib/v0.6/JuliaDB.ji for module JuliaDB.
 56.228227 seconds (1.21 M allocations: 66.301 MiB, 0.16% gc time)

on

julia> versioninfo()
Julia Version 0.6.2
Commit d386e40 (2017-12-13 18:08 UTC)
Platform Info:
  OS: Linux (x86_64-redhat-linux)
  CPU: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (DYNAMIC_ARCH Sandybridge)
  LAPACK: libopenblasp.so.0
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, ivybridge)

[KristofferC]

Yeah, this is not an inherent problem with Pkg3 but due to the fact that, right now, Pkg3 is a separate package that still has to hook into some core stuff like code loading. This causes compilation overhead for each precompilation process. Will be fixed.

[StefanKarpinski]

It may already be fixed since the code loading parts that Pkg3 was monkey patching are now built into Base instead.

[KristofferC]

It still puts a Base.require(:Pkg3) into every precompilation process which takes at least 3 seconds per process. Precompiling e.g. JSON spawns 3 such processes.

[KristofferC]

But perhaps that is not needed anymore?

[KristofferC]

It is not, https://github.com/JuliaLang/Pkg3.jl/pull/99 has no precompilation time regression on master. 🎉