Pkg.jl: allow package to be disambiguated by `@user/package`

[StefanKarpinski]

One of the ideas that came out of https://discourse.julialang.org/t/decentralized-package-manager/20931 is that when we have multiple packages with the same name in registries, it would be nice to be able to disambiguate which one you want by writing pkg> add @user/package instead of having to do pkg> add package and then interactively selecting from different packages by that name. It's a nice qualifier since the info already exists and is well-known: people know that @KristofferC is the author of Crayons.jl so if someone else comes and creates a package that provides data about all the colors of Crayola Crayons™ and also calls it Crayons.jl then all you need to do to indicate that you want Kristoffer's package is write pkg> add @KristofferC/Crayons instead of just pkg> add Crayons.

[fredrikekre]

pkg> add @KristofferC/Crayons

Could it not just be pkg> add KristofferC/Crayons since / is not allowed in package names?

[00vareladavid]

But you can still add by path

[fredrikekre]

Yea, but how is that different from

pkg> add Crayons

that can also be a path but we fix that ambiguity by treating that as a package, which we can do here too.

[EricForgy]

Question...

It is not safe to assume that a package maps to a unique uuid.

Is it safe to assume that a @user/package maps to a unique uuid?

[00vareladavid]

Its unlikely that a single user will register two packages with the same name. What is the value in making that an explicit restriction?

[EricForgy]

I am not suggesting making it a restriction. It is a question.

I am taking a stab at implementing this and it would make life easier if @user/package mapped to a unique uuid. If that is not the case, there needs to be additional logic. That's all.

[00vareladavid]

I suppose you can make that assumption for now. That case will eventually need to be checked somewhere though: either the registry or here in the client.

[StefanKarpinski]

I'm not sure why there's any additional logic. Currently, when you do add package it selects a set of UUIDs as candidates to install. If there's only one UUID then it just installs that; if there are zero UUIDs then it's an error; if there are two or more UUIDs then you get a prompt to disambiguate. This is no different, add @user/package is just a more specific UUID selector—the rest of the logic remains identical. If, for example, the there are two packages registered as XYZ with these repos:

• https://github.com/SomeUser/XYZ1.jl
• https://github.com/SomeUser/XYZ2.jl

Then if someone writes add @SomeUser/XYZ they will get a prompt to pick which package they want.

Which raises the question: where does the user name come from? Is it just that the system understands GitHub URLs and knows that the SomeUser part of https://github.com/SomeUser/XYZ1.jl is a user/org name? So the @user part acts as an additional filter on candidate packages based on their URL. That seems reasonable enough. That does also mean that if you write add @SomeUser/YXZ and there exists an XYZ package but it doesn't have a repo URL that we can parse a user name out of or it does but the user name doesn't match, then there will simply be a "package not found" error.

[StefanKarpinski]

Yea, but how is that different from

pkg> add Crayons

that can also be a path but we fix that ambiguity by treating that as a package, which we can do here too.

It's not fundamentally, but it does mean that we'd be additionally relegating adding by relative path to more of a niche case. Perhaps that's ok, but I feel like add vendor/SomePkg is a pretty common use case—more common than having vendored packages in the top-level directory. We could go the other direction and say that people should always write the leading ./ for relative package adds. But I think that @user and @org are kind of nice since that is how you refer to them on GitHub and elsewhere.

My concern would be potential clash with using @1.2.3 for version numbers, but I kind of want to unify version number specification with branch selection in terms of syntax, i.e. maybe use #1.2.3 for version numbers the same way as we use #master to install from the master branch. Also, the @user/name pattern is fairly easy to syntactically distinguish from the trailing @version syntax.

[EricForgy]

I'm not sure why there's any additional logic.

Yeah. I'm not sure either. I should have said "might need to be some additional logic".

Which raises the question: where does the user name come from? Is it just that the system understands GitHub URLs and knows that the SomeUser part of https://github.com/SomeUser/XYZ1.jl is a user/org name? So the @user part acts as an additional filter on candidate packages based on their URL. That seems reasonable enough. That does also mean that if you write add @SomeUser/YXZ and there exists an XYZ package but it doesn't have a repo URL that we can parse a user name out of or it does but the user name doesn't match, then there will simply be a "package not found" error.

I never used Base.@kwdef and didn't even know about it until reading through some of the Pkg code. The cool thing about it is it seems you can add a user field to Project without breaking anything. Then user could even be an optional field in Project.toml.

If it isn't in Project.toml, then as a last resort you can maybe try to deduce it from the GitHub url, but that seems fragile and maybe not worth it. If someone expects some potential future ambiguity, e.g. in my case I am thinking about Countries.jl, Currencies.jl, etc, they can just put a user in their Project.toml.

Then it is a matter of including it in the filtering process like you say.

Would that be something to consider, i.e. adding an optional field to Project that can be picked up from Project.toml?

[StefanKarpinski]

Having to put a user in the project file eliminates most of what makes this idea appealing to me, i.e. that it cannot be done wrong and starts working immediately. I don't see what's so fragile about pattern matching repo URLs when filtering packages.

[tk3369]

Somewhat off topic. How do you using two different packages that have the same name?

[fredrikekre]

You don't.

[DilumAluthge]

If I understand correctly, @StefanKarpinski's proposal in this issue is to do pattern-matching on the package URL to try to figure out the username or organization name.

Here is a different proposal, which suggests explicitly adding namespaces to registries: https://github.com/JuliaLang/Pkg.jl/issues/1836