Julia: Expose Base.find_package?

[KristofferC]

Base.find_package is similar to Pkg.dir except that it reports the actual path that would be loaded instead of just joinpathing things.

julia> Base.find_package("UUIDs")
"/Users/kristoffer/julia/usr/share/julia/stdlib/v0.7/UUIDs/src/UUIDs.jl"

julia> Base.find_package("SpecialFunctions")
"/Users/kristoffer/.julia/v0.6/SpecialFunctions/src/SpecialFunctions.jl"

julia> Base.find_package("Example")
"/Users/kristoffer/.julia/packages/Example/kH44/src/Example.jl"

Could be useful to expose.

[Datseris]

I plus one this, and to justify further:

I typically use the Pkg.dir() command to go to the test folder of a package quickly, and with this function this could be done easily with e.g.

cd(dirname(find_package("Example"))*"../test")

Other users (@iamed2) reported on slack that Pkg.dir() is useful in other situations as well.

[ararslan]

Could we just replace the current Pkg.dir behavior with that of Base.find_package?

[StefanKarpinski]

As it is, this is almost guaranteed to be used incorrectly. Yes, it works fine when called from Main and gives the right answer, but people will use from other packages, at which point it will be wrong since the same name there might load a different package entirely since the names that are visible from Main and their meanings are different from the names that are visible from packages. The correct usage is generally, Base.find_package(@__MODULE__, name), so the better API here would be a macro @find_package name which expands to that. We may as well bikeshed the name while we're at it.

[ararslan]

@packagedir? Or Pkg.@dir?

[StefanKarpinski]

It doesn't return a directory, so that would be a little confusing. But of course, we could have a macro that returns that instead.

[fredrikekre]

@which using Package?

[StefanKarpinski]

In https://github.com/JuliaLang/julia/issues/25720 I just suggested that the API for reload ought to be reload(ABC) where ABC is the package's module since the module has enough information in it to know exactly where to find the source. This is actually a similar situation and you could have something like this:

source_file(m::Module) = Base.locate_package(Base.PkgId(m))

julia> source_file(UUIDs)
"/Users/stefan/projects/julia/usr/share/julia/stdlib/v0.7/UUIDs/src/UUIDs.jl"

Does that do what people need? The tricky find_package interface is really only necessary when you don't already have the package. Of course, we may want to cache where packages were loaded from in a dict rather than do the locate_package lookup again since that can end up giving a different answer if LOAD_PATH or DEPOT_PATH have been changed since the code was loaded.

[stevengj]

On discourse I gave the examples of Documenter.jl and Coverage.jl, both of which need to find (or be given) the source directory of another package that may not have been loaded yet. But I think these could both be modified to ask the user to do import Foo first, and then pass the module Foo to Pkg.dir(Foo) or whatever.

[stevengj]

As another example, one of the tests for NLopt.jl uses the test problem defined in MathProgBase/test/nlp.jl. I'm not sure how to do that anymore, short of just copying the file into the NLopt.jl repo.

[KristofferC]

That specific use case seems kinda iffy though? What if MathProgBase changes the name of their files? Is the name of files in your test folder a part of the API?

Shouldn't MathProgBase provide a test_problem(...) or something if they want to provide test problems?

[Datseris]

What about non Julia files? I often want to go to the test directory of MIDI.jl because I know there is a midi file there I want to load.

[KristofferC]

From where do you want to load it?

Relying on absolute paths in other packages seems very brittle to me. Just have a function in MIDI?

[Datseris]

From where do you want to load it?

I am not sure I have understood the question: I typically want to load the midi file when developing. I know there is a midi file in the MIDI package test folders so that is an _awesome shortcut_.

Just have a function in MIDI?

Yes, I could (and seems like I will have to).

But why would you believe that the correct approach is that every single package should define single individual functions, instead of this veeeeery useful function Pkg.dir? Isn't it clear from this issue that plenty of users have plenty of uses for it? The existence of a "global" function that works for any package is just so useful.

I mean what about the suggestion

@which using Package?

? Julia has to know where this Package is coming from... Why not help the users in need by letting them know as well? I really don't see where you are coming from with not wanting such a feature, and this is why I am asking all these questions.

[KristofferC]

The whole reason I opened this issue is because there are cases where it is useful.

But @which using Package depends on your current project and in what module you execute it so it is not completely straightforward, which is why the old Pkg.dir style API doesn't really work anymore.

[fredrikekre]

If a particular file is part of the package API it makes sense to add an access function for it, or a function that return a directory containing things for example (like test problems or whatever).

[jekbradbury]

Once you've loaded a package, there's always a unique place on disk that it came from, so a Pkg.dir(::Module) API (rather than Pkg.dir(::AbstractString)) might work, similar to Python's module.__file__?

[StefanKarpinski]

Yes, that's the correct way to refactor all of these APIs. If someone wants to take a crack at that now for this and require, a PR would be welcomed. As long as the API operates on modules rather than names (which do not uniquely identify packages anymore) that would be highly welcomed.

[stevengj]

I'm putting together a PR now.