Pkg.jl: ]add checks github when package is already installed

[Orbots]

This is a real problem in an offline environment.

julia> LOAD_PATH
4-element Array{String,1}:
 "@"                                                
 "@v#.#"                                            
 "@stdlib/dd/dept/software/3ps/julia/julia_packages"
 "."                                                

julia> using UnicodePlots
[ Info: Recompiling stale cache file /dd/dept/software/3ps/julia/julia_packages/compiled/v1.0/UnicodePlots/Ctj9q.ji for UnicodePlots [b8865327-cd53-5732-bb35-84acbb429228]

julia> UnicodePlots.lineplot(sin,1:10)
           ┌────────────────────────────────────────┐
         1 │⠀⠀⣀⣀⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⠔⢣⠀⠀⠀⠀⠀⠀⠀⠀│ sin(x)
           │⠉⠉⠀⠀⠈⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡔⠁⠀⠀⢣⠀⠀⠀⠀⠀⠀⠀│
....

(Foo) pkg> add UnicodePlots
  Updating registry at `/dd/dept/software/3ps/julia/julia_packages/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`                  
_************* WAITS FOR A LONG TIME!!!! ****************_
┌ Warning: Some registries failed to update:
│     — /dd/dept/software/3ps/julia/julia_packages/registries/General — failed to fetch from repo
└ @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:157
 Resolving package versions...
  Updating `/dd/dept/software/users/mewert/swdevl/3ps/julia/1.0.2/Foo/Project.toml`
  [b8865327] + UnicodePlots v0.3.1
  Updating `/dd/dept/software/users/mewert/swdevl/3ps/julia/1.0.2/Foo/Manifest.toml`
  [864edb3b] + DataStructures v0.14.0
  [e1d29d7a] + Missings v0.3.1
....

julia> DEPOT_PATH
2-element Array{String,1}:
 "/dd/dept/software/3ps/julia/julia_packages"
 "/dd/home/mewert/.julia"

Possibly there is a configuration I can use to avoid this? I didn't have any luck on the forums.

Just tested in 1.0.2. Still a problem.

[Orbots]

rearranging depot order ( UnicodePlots is in the julia_packages "central" shared repo )

julia> DEPOT_PATH
2-element Array{String,1}:
"/dd/home/mewert/.julia"
"/dd/dept/software/3ps/julia/julia_packages"

(Bar) pkg> add UnicodePlots
Cloning default registries into /dd/home/mewert/.julia/registries
Cloning registry General from "https://github.com/JuliaRegistries/General.git"
ERROR: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:Net, curl error: Failed to connect to github.com port 443: Connection timed out
)

[Orbots]

I think this is in part due to my not understanding the relationships between projects, packages, environments and registries. I'm starting to understand that "installing" a package at the Repl: "(v1.0) pkg> add UnicodePlots" doesn't "install" it for in a central place that Bar can then use. Bar will go to the General registry to try to get it from github again being oblivious to what happed in environments/v1.0/Project.toml. Which is fine, I understand the rational. I just need a simple way to "install" packages as an "admin" and then let internet-challenged users "add" those "installed" packages in their dev packages.

[StefanKarpinski]

No, it's not your fault, if someone asks to add a package that already exists at some version—especially if it's already loaded—if that version is compatible, we should just use that version and not hit the internet at all.

[KristofferC]

We assume that the general registry will exist in the first DEPOT_PATH. The registry is needed because otherwise, we have no idea what UnicodePlots means and we cannot say if it is installed or not. Do you have the General registry in "/dd/dept/software/3ps/julia/julia_packages"?

[Orbots]

Yes the General registry has been cloned there when the admin "installed" packages. As a "user" I tried adding the specific version that was installed by the admin i.e. [email protected] but no joy. To my mind that should definitely work. (Bar) ] add UnicodePlots by the "user" could mean "check github to see if there is a newer version than installed locally" but (Bar) ] add [email protected] when that has been installed by admin should just use that.

[fredrikekre]

It did work though? UnicodePlots was added. The problem seems that it takes too long to detect that we can't update the registry.

[KristofferC]

We could relax not trying to clone the General registry into the first DEPOT_PATH is we have found it somewhere else.

[fredrikekre]

Yea. A user can always

pkg> registry add General

and use that if the system installed one is out of date.

[Orbots]

In the case where the DEPOT_PATH looks like:
2-element Array{String,1}:
"/dd/dept/software/3ps/julia/julia_packages"
"/dd/home/mewert/.julia"

It does work, just has to wait to timeout. If I switch those paths it will fail.

[fredrikekre]

Yea, we are only updating registries in the first entry of depot path.

[Orbots]

This works nicely now if I put the central depot after ~/.julia in JULIA_DEPOT_PATH. Needs to be done before loading julia. Thanks!