Julia: Fix doc about dlsym/dlopen

[tuckermcclure]

It's common to write some C code to be used from a higher level language, and the fact that Julia supports this so easily is wonderful. However, when developing the C code, there's a workflow problem. A ccall in Julia loads in the library, but then one can't unload or reload the library, so if one needs to continue development on the C side, one must restart Julia. This makes testing very slow, especially if testing the code involves a long setup (e.g., starting a big simulation to use the code).

I'm basing my conclusion that this can't currently be done on a Discourse topic about it:

https://discourse.julialang.org/t/unload-a-shared-library/5344

and this Google Groups thread from years ago:

https://groups.google.com/forum/#!topic/julia-dev/U6gCDbD4XiM

It would be a great feature to add and would help those integrating with C to work more fluidly.

[yuyichao]

I can't reproduce the problem.

julia> x = Libdl.dlopen("/usr/lib/libcairo.so")
Ptr{Void} @0x0000564cdf9e1910

julia> filter(x->contains(x, "cairo"), Libdl.dllist())
1-element Array{AbstractString,1}:
 "/usr/lib/libcairo.so"

julia> Libdl.dlclose(x)
true

julia> filter(x->contains(x, "cairo"), Libdl.dllist())
0-element Array{AbstractString,1}

This is only possible when you are only dealing with the library explicitly through Libdl.dlopen. If you use it implicitly through ccall or cglobal or having sth else depending on this then this can't be done.

[StefanKarpinski]

Note that you can still use ccall with libraries opened via dlopen by passing the pointer to the library where the name would normally go.

[yuyichao]

Can you? Last time I check it was never allowed.

[StefanKarpinski]

It was once upon a time, I didn't realize we'd removed that facility.

[yuyichao]

Just out of curiosity, do you remember when it was the case? I had that feeling a few months ago but didn't see it so I checked the blame all the way back and didn't find it at least as early as 0.2 IIRC.

[Keno]

Yeah, I don't think we ever allowed that. You can still pass the raw ptr you get from dlsym of course, which is what @StefanKarpinski may be remembering.

[StefanKarpinski]

You can still pass the raw ptr you get from dlsym of course

Yes, that's what I was thinking of.

[tuckermcclure]

Ok, thanks all.

Dear future readers, if you were calling

ccall((:my_fcn, "./my_lib.so"), ...)

but needed to close the library when done, then do this instead:

lib = Libdl.dlopen("./my_lib.so")
sym = Libdl.dlsym(lib, :my_fcn) # Note that this is called @dlsym in the doc, but that macro no longer exists!
ccall(sym, ...) # Remaining arguments are the same.
Libdl.dlclose(lib)

This took a long time for me to find for the following reasons:

1. The doc mentions dlsym and using it with ccall, but doesn't mention why that would be useful. I might try to update the doc.
2. The doc also mentions @dlsym, which doesn't exist now. Again, a doc thing.
3. I expected that one would be able to "find" the handle to the library after a ccall so as to close it. I'm gathering that something about this is hard to implement?

[pkofod]

Remember to quote names: @dlsym. You just pinged the user named dlsym :) (luckily he or she seems quite inactive)

[yuyichao]

I'm gathering that something about this is hard to implement?

Not so much as hard to implement as it's unclear what the API is. Do you want the function to crash or open it again if you closed it? The open is implicit so there doesn't seem to be a need for a explicit close.

[tuckermcclure]

Good point @yuyichao. I made a PR for the doc change (above). I realized that the macro that I had said was missing was actually created in the doc itself, so my PR just adds a few notes on closing libraries. I hope it's useful.

[yuyichao]

I'll also note that by writing cglobal((func, lib)) or ccall((func, lib), ...) you are basically asking for linking. You can't dlclose a library where there's still libraries depending on it so it's consistent that we also don't allow dlcloseing libraries in cglobal or ccall since we never free code (jit or sysimg).