Julia: LibGit2 can not clone or fetch on Julia 1.3/master

[fredrikekre]

Opening this issue here since I am pretty sure Pkg has not changed anything, maybe this is related to the libgit2 upgrade (#32806)?

$ julia13 -e 'using Pkg; Pkg.Registry.update()'
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
┌ Warning: Some registries failed to update:
│     — /home/fredrik/.julia/registries/General — failed to fetch from repo
└ @ Pkg.Types ~/julia13/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1189

$ julia12 -e 'using Pkg; Pkg.Registry.update()'
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`

[KristofferC]

Works for me on a mac.

[fredrikekre]

Ok, I'm on Linux. Reverting #32806 fixes this issue for me.

[fredrikekre]

Reproducer without any Pkg code:

julia> import LibGit2

julia> repo = LibGit2.GitRepo("/home/fredrik/.julia/registries/General");

julia> remoteurl = "https://github.com/JuliaRegistries/General.git";

julia> LibGit2.fetch(repo; remoteurl=remoteurl)
ERROR: GitError(Code:ERROR, Class:Net, unrecognized URL prefix)
Stacktrace:
 [1] #fetch#62(::LibGit2.FetchOptions, ::String, ::typeof(LibGit2.fetch), ::LibGit2.GitRemote, ::Array{AbstractString,1}) at /home/fredrik/julia-master/usr/share/julia/stdlib/v1.4/LibGit2/src/error.jl:101
 [2] #fetch at /home/fredrik/julia-master/usr/share/julia/stdlib/v1.4/LibGit2/src/remote.jl:297 [inlined]
 [3] #fetch#115(::String, ::String, ::Array{AbstractString,1}, ::Nothing, ::Dict{Symbol,Tuple{Ptr{Nothing},Any}}, ::typeof(LibGit2.fetch), ::LibGit2.GitRepo) at /home/fredrik/julia-master/usr/share/julia/stdlib/v1.4/LibGit2/src/LibGit2.jl:290
 [4] (::LibGit2.var"#kw##fetch")(::NamedTuple{(:remoteurl,),Tuple{String}}, ::typeof(LibGit2.fetch), ::LibGit2.GitRepo) at /home/fredrik/julia-master/usr/share/julia/stdlib/v1.4/LibGit2/src/LibGit2.jl:272
 [5] top-level scope at REPL[4]:1

[omus]

Possibly unrelated: I saw the same error on Julia 1.0.4 today on macOS. I fixed it by going to the registry in my terminal and running git fetch. After that Pkg updates stopped producing the warning.

[fredrikekre]

Possibly unrelated: I saw the same error on Julia 1.0.4 today on macOS. I fixed it by going to the registry in my terminal and running git fetch. After that Pkg updates stopped producing the warning.

That did not fix it for me. Also, I tried from a different machine but could not reproduce. Any ideas on how to debug?

[fredrikekre]

Cloning from scratch also fails, with the same error message about unrecognized URL prefix.

[fredrikekre]

This also happens with the generic binaries btw.

[KristofferC]

Possible future way to debug this is to reduce it to just ccalls and then look into libgit2 where the error is generated.

[fredrikekre]

import LibGit2

const repo_url = "https://github.com/JuliaRegistries/General.git"
const repo_path = "/tmp/General"
const credentials = nothing
const callbacks = LibGit2.Callbacks()
const cred_payload = LibGit2.reset!(LibGit2.CredentialPayload(credentials))
callbacks[:credentials] = (LibGit2.credentials_cb(), cred_payload)
const lbranch = Base.cconvert(Cstring, "")
const remote_callbacks = LibGit2.RemoteCallbacks(callbacks)
const fetch_opts = LibGit2.FetchOptions(callbacks=remote_callbacks)
const clone_opts = LibGit2.CloneOptions(
            bare = Cint(false),
            checkout_branch = Cstring(C_NULL),
            fetch_opts = fetch_opts,
            remote_cb = C_NULL
        )
LibGit2.ensure_initialized()
const clone_opts_ref = Ref(clone_opts)
const repo_ptr_ptr = Ref{Ptr{Cvoid}}(C_NULL)

ccall((:git_clone, :libgit2), Cint,
            (Ptr{Ptr{Cvoid}}, Cstring, Cstring, Ref{LibGit2.CloneOptions}),
            repo_ptr_ptr, repo_url, repo_path, clone_opts_ref)

returns -1.

[nalimilan]

FWIW both examples work here on latest master. Can anybody else reproduce?

[omus]

@fredrikekre can you post your git config?

[fredrikekre]

Interesting, I reduced the .gitconfig file to

[http]
    proxy =

I am not sure why it is there, I don't think I have added it myself. Seems like [email protected] fails for this, while [email protected] handles it (as does regular git)

[fredrikekre]

Maybe related to this point from libgit2 0.28 release notes:

HTTP proxy support is now builtin; libcurl is no longer used to support
proxies and is removed as a dependency.

[nalimilan]

Maybe file an issue against libgit2? You can also tried reverting the changes I made to gitcredential.jl in https://github.com/JuliaLang/julia/pull/32806, just in case (they are required for some specific tests, but not for most operations).

[fredrikekre]

See https://github.com/libgit2/libgit2/issues/5220.

[fredrikekre]

Removing from milestone since it is kinda weird to have an empty http.proxy, and we can upgrade libgit2 in 1.3.1 or whatever.

[kcajf]

I can reproduce something similar, but the snippet you sent above returns -12. It does appear to have been introduced in a recent commit - julia 1.2.0 runs fine but 1.3.0-rc1 and master / latest nightlies (47f2800747) fail.

[tkluck]

This issue also appeared for me with http(s)_proxy set as an environment variable. It needs to be unset (not set to empty string) to avoid it.

[kcajf]

Thanks for the suggestion @tkluck - unfortunately unsetting http_proxy and http_proxy doesn't make a difference more me - still getting -12 return code :(

[wsshin]

I had this issue just now and would like to report the test results with three fresh-built Julia binaries in release-1.0, release-1.3, and master branches. (I was not able to test release-1.2 due to build errors.) All three tests were conducted with empty ~/.julia. The environment variables http_proxy and https_proxy are defined in ~/.bashrc.

release-1.0 (no error):

julia> VERSION
v"1.0.6-pre.0"

(v1.0) pkg> update
 Resolving package versions...
   Cloning default registries into /Users/MY_USER_ID/.julia/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
    Fetching: [========================================>]  99.9 %

release-1.3 (error):

julia> VERSION
v"1.3.0-rc3.0"

(v1.3) pkg> update
   Cloning default registries into `~/.julia`
   Cloning registry from "https://github.com/JuliaRegistries/General.git"
ERROR: Git repository not found at 'https://github.com/JuliaRegistries/General.git'

master (error):

julia> VERSION
v"1.4.0-DEV.298"

(v1.4) pkg> update
   Cloning default registries into `~/.julia`
   Cloning registry from "https://github.com/JuliaRegistries/General.git"
ERROR: Git repository not found at 'https://github.com/JuliaRegistries/General.git'

Hope this issue to be resolved soon!

[kcajf]

Is your http_proxymissing a trailing slash? See https://github.com/libgit2/libgit2/issues/5255

[wsshin]

@kcajf, your solution works! Thank you so much!!!

For others, the solution here is to put / at the end of the proxy port number. For example, in ~/.bashrc, use

export http_proxy=http://your.proxy.address:8080/

instead of

export http_proxy=http://your.proxy.address:8080

[StefanKarpinski]

Sigh. It's annoyingly fussy of libgit2 to care if there's a trailing slash or not.

[kcajf]

Yeah. I can't imagine it will be hard to get it fixed upstream, but the question is whether it can happen in a reasonable timeframe for 1.3.0. I expect this will cause trouble for a lot of corporate Julia users, since I'd guess rhat no-trailing-slash is the more common format.

[StefanKarpinski]

We could fix it on our end by looking for the environment variable and detecting the lack of trailing slash and fixing it. It's a bit weird to be messing with user environment variables that are not even Julia-specific though. That feels kind of dangerous even if it saves some trouble...

[kcajf]

It can also be picked up from ~/.gitconfig and a few other places, so there's no overall solution

[nalimilan]

libgit2 devs haven't commented on that yet, maybe that's just an oversight. If not, we can try to convince them to change this...

[StefanKarpinski]

They've commented since and it seems like this will be fixed in the next libgit2 release.

[antoine-levitt]

With 1.3, upgrading from 1.1, I get

(v1.3) pkg> up
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
┌ Warning: Some registries failed to update:
│     — /home/levitta/.julia/registries/General — failed to fetch from repo
└ @ Pkg.Types /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1199
 Resolving package versions...

After rm -rf .julia, I get

(v1.3) pkg> up
   Cloning default registries into `~/.julia`
   Cloning registry from "https://github.com/JuliaRegistries/General.git"
ERROR: Git repository not found at 'https://github.com/JuliaRegistries/General.git'

I'm behind a university proxy, and my http_proxy, https_proxy and their caps versions are all set correctly (with a trailing slash).

edit: I should mention that worked in 1.1 and 1.2
edit2: oh, and the above script returns -12.

[kcajf]

see code here for how to extract the actual error message: https://github.com/libgit2/libgit2/issues/5255#issuecomment-540705608

the proxy urls can be sourced from the env vars, but can also come from other places e.g. ~/.gitconfig. Maybe they are being set elsewhere? It would be nice if we could get libgit2 to print out the final proxy url it is detecting.

I fear this issue is going to cause a lot of trouble for corporate & university users.. Hopefully libgit2 release an update soon that can be included in a minor julia version.

[antoine-levitt]

Ooooh, very good catch, it was indeed set in .gitconfig, which has precedence on the env variables. With that, it works!

[jdtuck]

Trailing slash is the fix, but man that is really stupid.....

[StefanKarpinski]

Agree. It's a libgit2 bug, not a Julia bug. Should be fixed in a newer libgit2.

[jekriske-lilly]

Still an issue in Julia 1.3.1

[StefanKarpinski]

@staticfloat: did we not upgrade libgit2?

[staticfloat]

Last time we rebuilt libgit2 (e.g. rebuilt with new BB shards) was https://github.com/JuliaLang/julia/commit/e44652ab03ce384444bddfaca270c04b42d968a8

Last time we upgraded libgit2 version was https://github.com/JuliaLang/julia/commit/d0b5d9850fb7b51c7831d3897ad46e4d3478d322

If we need a specific version that contains a fix, let me know and I'll build new versions.

[KristofferC]

0.28.4 has a bunch of security fixes so should probably upgrade?

[fredrikekre]

0.24.4 doesn't seem to include https://github.com/libgit2/libgit2/commit/c6ab183e9c960b74471a7d106a4deb0c8b28a8ec though which supposedly fixed this issue (https://github.com/libgit2/libgit2/issues/5255#issuecomment-541589326).

[StefanKarpinski]

So I guess the question is whether we want to patch libgit2 or just wait until they make a release that includes this fix.

[gwatcha]

I had no proxy and I had this issue, I fixed it by downgrading libgit2 from 0.99.0 to 0.27.8

[n8xm]

I had no proxy and I had this issue, I fixed it by downgrading libgit2 from 0.99.0 to 0.27.8

Can confirm. I also had this issue, without any proxy set up (at least not in my .gitconfig). I fixed it the exact same way - by downgrading libgit2 from 0.99.0 to 0.27.8.

[znmeb]

I had no proxy and I had this issue, I fixed it by downgrading libgit2 from 0.99.0 to 0.27.8

Can confirm. I also had this issue, without any proxy set up (at least not in my .gitconfig). I fixed it the exact same way - by downgrading libgit2 from 0.99.0 to 0.27.8.

I have this too on Arch Linux - libgit2 is 1:0.99.0-2

Downgrading libgit2 is not an option. I either need something to put in my .gitconfig that will make this go away without breaking anything else, or a fix in Julia.

$ julia --version
julia version 1.3.1

...

julia> using Pkg

julia> Pkg.add("IJulia")
   Cloning default registries into `~/.julia`
   Cloning registry from "https://github.com/JuliaRegistries/General.git"
ERROR: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)
Stacktrace:
 [1] top-level scope at REPL[2]:1
caused by [exception 1]
GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)
Stacktrace:
 [1] top-level scope at REPL[2]:1

[StefanKarpinski]

You can use the soon-to-be-released Julia 1.4 with JULIA_PKG_SERVER=pkg.julialang.org and it won't use libgit2 at all for installing registered packages.

[toktay]

I concur this bug. I am using manjaro.. libgit2 is upgraded to 0.99 and my julia environment (1.3.1) is broken right now.... I have installed julia to a new machine. It is also broken as well. I noticed that bug only impacts updating of registry from github. But packages still can be installed if registery is manually cloned under ~/.julia/registries/General... I presume update can be handled manually as well...

[toktay]

When will this happen?

You can use the soon-to-be-released Julia 1.4 with JULIA_PKG_SERVER=pkg.julialang.org and it won't use libgit2 at all for installing registered packages.

[StefanKarpinski]

You can do it right now if you use a release candidate or JuliaPro 1.3.

[jxy]

I have libgit2 upgraded to 0.99 on freebsd, and got the same problem. The workaround for pkg is manually updating ~/.julia/registries/General.

[fredrikekre]

Please report to the libgit2 repo instead (unless it is a duplicate of https://github.com/libgit2/libgit2/issues/5220, in which you can maybe bump that issue.)

[diabonas]

The issues with LibGit2 0.99.0 reported in the comments are unrelated to the original bug report, see issue #35043 instead and PR #35232 for a suggested fix.

[getzze]

You can use the soon-to-be-released Julia 1.4 with JULIA_PKG_SERVER=pkg.julialang.org and it won't use libgit2 at all for installing registered packages.

This workaround doesn't work unfortunately, I am using julia 1.4.

[tuhtah]

@getzze How are you invoking julia? Maybe you forgot to export the JULIA_PKG_SERVER environment variable in your shell before invoking Julia in a separate shell prompt?

The workaround works on _Arch Linux_ using _Julia_ 1.4.0:

[tamer@zinc ~]$ pacman -Q libgit2 julia
libgit2 1:0.99.0-2
julia 2:1.4.0-1

[tamer@zinc ~]$ rm -r ~/.julia/registries/General/

[tamer@zinc ~]$ julia -e 'using Pkg; Pkg.update()'
    Cloning default registries into `~/.julia`
    Cloning registry from "https://github.com/JuliaRegistries/General.git"
ERROR: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)
Stacktrace:
 [1] up(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; level::Pkg.Types.UpgradeLevel, mode::Pkg.Types.PackageMode, update_registry::Bool, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:206
 [2] up at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:201 [inlined]
 [3] #up#47 at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:196 [inlined]
 [4] up at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:196 [inlined]
 [5] #up#44 at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:193 [inlined]
 [6] up() at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:193
 [7] top-level scope at none:1
caused by [exception 1]
GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)
Stacktrace:
 [1] up(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; level::Pkg.Types.UpgradeLevel, mode::Pkg.Types.PackageMode, update_registry::Bool, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:206
 [2] up at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:201 [inlined]
 [3] #up#47 at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:196 [inlined]
 [4] up at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:196 [inlined]
 [5] #up#44 at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:193 [inlined]
 [6] up() at /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:193
 [7] top-level scope at none:1

[tamer@zinc ~]$ JULIA_PKG_SERVER=pkg.julialang.org julia -e 'using Pkg; Pkg.update()'
    Cloning default registries into `~/.julia`
######################################################################## 100.0%
      Added registry `General` to `~/.julia/registries/General`
   Updating registry at `~/.julia/registries/General`
   Updating `~/.julia/environments/v1.4/Project.toml`
 [no changes]
   Updating `~/.julia/environments/v1.4/Manifest.toml`
 [no changes]

[fredrikekre]

If you already have the registry as a git clone it will keep using it as a git clone instead of using the Pkg server, thats why you have to delete it.

[getzze]

Thanks, I didn't get that you have to remove the registries before.
However, I am using the development version of some packages so I need to fetch from git and one package failing to be fetched makes all the update fail.

[tastyminerals]

Running Julia 1.4.0. Not using any proxy. Windows 10 Pkg.add works, on Linux Manjaro for the same package it throws the error above :(

@tuhtah solution works though

[tbhaxor]

Hi, I am on manjaro Linux. In my case, this helped

mkdir -p /home/terabyte/.julia/registries/General && git clone https://github.com/JuliaRegistries/General.git ~/.julia/registries/General

Then in julia

julia> using Pkg

julia> Pkg.add("IJulia")

[ghost]

There is no need to delete any folder. With...

X@alienware: ~/ pacman -Q libgit2 julia
libgit2 1:1.0.0-1
julia 2:1.4.0-1

I get

X@alienware: ~/ julia -e 'using Pkg; Pkg.update()'
   Updating registry at '/usr/applications/academia/julia/registries/General'
   Updating git-repo 'https://github.com/JuliaRegistries/General.git'
┌ Warning: Some registries failed to update:
│     — /usr/applications/academia/julia/registries/General — failed to fetch from repo
└ @ Pkg.Types /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:1122
   Updating '/usr/applications/academia/julia/environments/v1.4/Project.toml'
 [no changes]
   Updating '/usr/applications/academia/julia/environments/v1.4/Manifest.toml'
 [no changes]

I originally applied @gwatcha workaround, however using

X@alienware: General/ pwd
/usr/applications/academia/julia/registries/General
                              --   --   (master  origin)     
X@alienware: General/ git pull

Does the same job. When I went back to julia and tried to upgrade the packages... they did upgrade:

(@v1.4) pkg> update
   Updating registry at `/usr/applications/academia/julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
┌ Warning: Some registries failed to update:
│     — /usr/applications/academia/julia/registries/General — failed to fetch from repo
└ @ Pkg.Types /build/julia/src/julia-1.4.0/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:1122
  Installed RecipesPipeline ─ v0.1.2
  Installed Plots ─────────── v1.0.8
   Updating `/usr/applications/academia/julia/environments/v1.4/Project.toml`
  [91a5bcdd] ↑ Plots v1.0.4 ⇒ v1.0.8
   Updating `/usr/applications/academia/julia/environments/v1.4/Manifest.toml`
  [91a5bcdd] ↑ Plots v1.0.4 ⇒ v1.0.8
  [01d81517] + RecipesPipeline v0.1.2
   Building Plots → `/usr/applications/academia/julia/packages/Plots/7U0ob/deps/build.log`

All this because I couldn't get Plots 1.0 from JULIA_PKG_SERVER=pkg.julialang.org. So, if you fancy/need the latest version of the packages, updating the registries by hand can do the trick until julia and libgit2 talk to each other again.

Edit: apparently it is now in pkg.julialang.org (only up to v0.28 when I tried). It's worth a shot if you don't want to be messing with folders and downgrading packages.

[hzhangxyz]

git issue same here

ERROR: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)

---

env: Arch, julia1.4.0, libgit2 1.0.0

[StefanKarpinski]

If you're on Julia 1.4, you can do export JULIA_PKG_SERVER=pkg.julialang.org and you won't need to use git for Pkg operations. You'll need to do rm -rf ~/.julia/registries/General first to upgrade the General registry to using the Pkg protocol instead of git though.

[nalimilan]

That issue should be reported to Arch, they need a patch like https://github.com/JuliaLang/julia/pull/35233 if they use libgit2 1.0.

[barche]

For the record, I got hit by this again trying Julia 1.5 for the first time at work again, trailing slash needed on the proxy in ~/.gitconfig.

[StefanKarpinski]

Someone who cares about this is going to have to harass the libgit2 project and make sure they actually make a release that includes a bug fix for this. Clearly waiting for them to fix it on their own isn't working out.

[TLDay]

For me, this error happend because I'm using a socks5 proxy(.gitconfig, and git clone mannully works with socks5 proxy). Change to http proxy fixed it.

[yalwan-iqvia]

Jan 21

So I guess the question is whether we want to patch libgit2 or just wait until they make a release that includes this fix.

Aug 21

Clearly waiting for them to fix it on their own isn't working out.

So go with the former? (patch libgit2) You might even be able to contribute the patch to them.

Asking users to file issues/deal with upstream libraries is a little unfair, IMO (its the users who will care in the end, after all)

[StefanKarpinski]

Fair or not, I don't encounter this issue and I don't feel like mucking about in libgit2 to fix this. If you want to pay for someone to work on this, contact [email protected] and we can figure out a consulting contract. You can also purchase JuliaTeam and then you won't have this issue because you can connect to a package server that's inside of your firewall, so you don't need to go through a proxy at all (there are lots of other nice benefits too). At the very least, you could make some noise on https://github.com/libgit2/libgit2/issues/5220 instead of here. @fredrikekre is literally the only person who has posted there. Instead everyone bugs us and I guess expects us to harass the the libgit2 developers for them? If everyone who has posted here also posted a complaint on the libgit2 repo they might have fixed it by now.

[StefanKarpinski]

I have now pinged everyone who has posted here and in related issues where this libgit2 bug was the underlying cause of their problem on the relevant libgit2 issue. Please chime in there with your bug reports regarding this.

[KristofferC]

Maybe https://github.com/JuliaLang/julia/pull/35233 would help? I haven't looked into the patch notes.