Julia: precompile shouldn't try opening dylibs

[KristofferC]

Running the following:

❯ julia --project -J"/home/kc/Applications/julia-1.3.0/lib/julia/sys.so"  --output-o=sys.o \
 -e 'Base.reinit_stdio(); Base.init_depot_path(); Base.init_load_path(); using Plots'

error with

ERROR: LoadError: error compiling gr_draw_marker: could not load library "libGR.so"
libGR.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] precompile at ./essentials.jl:463 [inlined]
 [2] _precompile_() at /home/kc/.julia/packages/Plots/2KhB2/src/precompile.jl:3
 [3] top-level scope at /home/kc/.julia/packages/Plots/2KhB2/src/Plots.jl:226

when Plots tries to execute one of the precompile statements in https://github.com/JuliaPlots/Plots.jl/blob/master/src/precompile.jl. The reason it can't find it is because with --output-o the __init__ function doesn't run and GR makes its library availably to Base.DL_LOAD_PATH using that function.

It doesn't seem like precompile should go and try open dynamic libraries at least not in --output-o mode.

[JeffBezanson]

I fear we might need to start running __init__ functions during precompile. Otherwise code basically needs to work in two different contexts, which is always going to be brittle.

[StefanKarpinski]

Can you explain the thumbs down, @vtjnash?

[vtjnash]

It's only thought to be brittle because we don't test this part of codegen well. This bug nothing to do with __init__ or even packages. It's just another instance of this old bug:

bash$ ./julia --output-o=x.o -J usr/lib/julia/sys.dylib -e 'Base.reinit_stdio(); ccall(:does_not_exist, Cvoid, ())'

current result:

signal (11): Segmentation fault: 11
in expression starting at none:1
unknown function (ip: 0x0)
Allocations: 2491 (Pool: 2482; Big: 9); GC: 0
Segmentation fault: 11

expected result:

ERROR: could not load symbol "does_not_exist":
dlsym(RTLD_DEFAULT, does_not_exist): symbol not found
Stacktrace:
 [1] top-level scope at ./none:1

@vchuravy is fixing part of this aspect of codegen in https://github.com/JuliaLang/julia/pull/33820, I'll make a PR shortly to fix this bug.

(I think that reinit_stdio hack there is only needed _because_ we have been forcibly calling __init__ and then not resetting state properly. I'm hoping to get rid of that at some point too.)

[JeffBezanson]

Yes, it's fine to fix the library lookup part of this independently. The issue with __init__ is more general; people might try to run various top-level expressions that depend on init having run for various reasons. Of course, that might just trade one problem for another, which is that __init__ then needs to be written to handle re-initializing as well as initializing.