Julia: BUG: Compatibility problems with libgit2 0.25.0

[musically-ut]

I recently upgraded to Julia 0.5.0 and ran into the following problem while trying to do a Pkg.add:

~ via ⬢ v6.2.1
➔  julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.0 (2016-09-19 18:14 UTC)
 _/ |\__'_|_|_|\__'_|  |
|__/                   |  x86_64-apple-darwin16.0.0

julia> Pkg.add("IJulia")
INFO: Initializing package repository /Users/musically_ut/.julia/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:Invalid, Invalid version 0 on git_proxy_options)
 in macro expansion at ./libgit2/error.jl:99 [inlined]
 in clone(::String, ::String, ::Base.LibGit2.CloneOptions) at ./libgit2/repository.jl:191
 in #clone#109(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredentials}, ::Function, ::String, ::String) at ./libgit2/libgit2.jl:327
 in (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::String) at ./<missing>:0
 in (::Base.Pkg.Dir.##4#6{String,String})() at ./pkg/dir.jl:49
 in cd(::Base.Pkg.Dir.##4#6{String,String}, ::String) at ./file.jl:59
 in init(::String, ::String) at ./pkg/dir.jl:47
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{Any,N}) at ./pkg/dir.jl:28
 in add(::String) at ./pkg/pkg.jl:100

julia>

Downgrading from libgit2 version 0.25.0 (which was released 11 days ago at the time of writing) to 0.24.1 solves the problem for me:

~ via ⬢ v6.2.1
➔  brew info libgit2
libgit2: stable 0.25.0 (bottled), HEAD
C library of Git core methods that is re-entrant and linkable
https://libgit2.github.com/
/usr/local/Cellar/libgit2/0.23.4 (100 files, 1.6M)
  Poured from bottle on 2016-01-07 at 18:02:04
/usr/local/Cellar/libgit2/0.24.1 (100 files, 1.7M)
  Poured from bottle on 2016-07-20 at 02:21:38
/usr/local/Cellar/libgit2/0.25.0 (105 files, 1.8M) *
  Poured from bottle on 2016-12-30 at 21:37:26
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/libgit2.rb
==> Dependencies
Build: pkg-config ✔, cmake ✘
Recommended: libssh2 ✔
==> Options
--universal
        Build a universal binary
--without-libssh2
        Build without libssh2 support
--HEAD
        Install HEAD version

~ via ⬢ v6.2.1
➔  brew switch libgit2 0.24.1
Cleaning /usr/local/Cellar/libgit2/0.23.4
Cleaning /usr/local/Cellar/libgit2/0.24.1
Cleaning /usr/local/Cellar/libgit2/0.25.0
7 links created for /usr/local/Cellar/libgit2/0.24.1

~ via ⬢ v6.2.1
➔  julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.0 (2016-09-19 18:14 UTC)
 _/ |\__'_|_|_|\__'_|  |
|__/                   |  x86_64-apple-darwin16.0.0

julia> Pkg.add("IJulia")
INFO: Initializing package repository /Users/musically_ut/.julia/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
INFO: Updating cache of BinDeps...
INFO: Updating cache of Compat...
INFO: Updating cache of Conda...
INFO: Updating cache of Homebrew...
INFO: Updating cache of IJulia...
INFO: Updating cache of JSON...
...

My version info:

julia> versioninfo()
Julia Version 0.5.0
Commit 3c9d753* (2016-09-19 18:14 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin16.0.0)
  CPU: Intel(R) Core(TM) i5-4260U CPU @ 1.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)

This problem was not limited to only IJulia package.

[tkelman]

How did you install julia? We haven't updated the bindings to be compatible with 0.25.0 yet. Would welcome a PR that did so, as long as it also kept compatibility with the existing working versions via appropriate if statements.

[musically-ut]

I used homebrew to update Julia.

musically_ut at Schiaparelli in ~ via ⬢ v6.2.1
➔  brew info julia
staticfloat/julia/julia: stable 0.5.0 (bottled), HEAD
http://julialang.org
/usr/local/Cellar/julia/0.3.8 (500 files, 80.4M)
  Poured from bottle on 2015-05-22 at 00:33:41
/usr/local/Cellar/julia/0.3.11 (499 files, 81.8M)
  Built from source on 2015-11-11 at 23:01:16
/usr/local/Cellar/julia/0.4.1 (678 files, 122.4M)
  Poured from bottle on 2015-11-06 at 17:53:33
/usr/local/Cellar/julia/0.4.2 (679 files, 122.7M)
  Poured from bottle on 2016-03-28 at 00:05:47
/usr/local/Cellar/julia/0.4.5 (676 files, 101.5M)
  Poured from bottle on 2016-04-23 at 22:33:02
/usr/local/Cellar/julia/0.4.6_1 (681 files, 122.2M)
  Poured from bottle on 2016-07-20 at 02:21:59
/usr/local/Cellar/julia/0.5.0 (755 files, 143.3M) *
  Poured from bottle on 2016-12-30 at 21:38:59
From: https://github.com/staticfloat/homebrew-julia/blob/master/julia.rb
==> Dependencies
Build: cmake ✘
Required: staticfloat/julia/llvm37-julia ✔, pcre2 ✔, gmp ✔, fftw ✔, mpfr ✔, libgit2 ✔, mbedtls ✔, staticfloat/julia/arpack-julia ✔, staticfloat/julia/openblas-julia ✔, staticfloat/julia/suite-sparse-julia ✔
==> Requirements
Build: git ✔
Required: fortran ✔
==> Options
--system-libm
    Use system's libm instead of openlibm
--HEAD
    Install HEAD version
==> Caveats
Documentation and Examples have been installed into:
/usr/local/Cellar/julia/0.5.0/share/julia

Test suite has been installed into:
/usr/local/Cellar/julia/0.5.0/share/julia/test

To perform a quick sanity check, run the command:
brew test -v julia

To crunch through the full test suite, run the command:
/usr/local/Cellar/julia/0.5.0/bin/julia -e "Base.runtests()"

---

I see what you are getting at for the fix; something like this:

    @static if LibGit2.VERSION >= v"0.24.0"
        custom_headers::StrArrayStruct
    end

in CloneOptions.

If I get sufficient free time over this weekend, I might take a stab at it.

[tkelman]

Ah right, then this is a bug with https://github.com/staticfloat/homebrew-julia (specifically this line https://github.com/staticfloat/homebrew-julia/blob/409e324c5a15d5a49af2d5eadd81d28c77054cb7/julia.rb#L44 needs to be more specific about acceptable versions, if homebrew is capable of that nowadays). Would still be good to unblock ourselves from upgrading.

I guess if julia were ever added to the main homebrew-core then this kind of thing might be found by their buildbots first. Ref https://github.com/Homebrew/homebrew-core/pull/8221 and https://github.com/Homebrew/homebrew-core/pull/8275.

[simonbyrne]

See https://github.com/libgit2/libgit2/blob/master/CHANGELOG.md#breaking-api-changes-1 for the necessary changes.

[simonbyrne]

I've made what I think are the necessary changes in this branch https://github.com/JuliaLang/julia/tree/sb/libgit2/25

However I haven't tried it as the patches no longer apply cleanly: would someone else be able to update these?

[tkelman]

Sure, I'll take a look.

[tkelman]

Rebased the patches and seems to work for me locally on Linux, should I push to your branch and open it as a PR?

[simonbyrne]

Yes, that's a good idea.

[RatanRSur]

So is there no stable package managed way to have this fix on macOS? I have to get the 0.6 pre alpha or build from source?

[simonbyrne]

The official .dmg should still be okay: http://julialang.org/downloads/

[simonbyrne]

Further discussion of the homebrew issue should continue at https://github.com/staticfloat/homebrew-julia/issues/231

[tkelman]

use homebrew-cask (which wraps the official dmg installer) if you want something "package managed"

[sivark]

I've hit the same problem after upgrading to libgit2 0.25.1-1 on Arch Linux.

Is there any solution other than downgrading my libgit2 installation? I'm afraid downgrading is hard because I have other packages which depend on the updated libgit2.

[nalimilan]

@sivark At this point, the best solution is to upgrade Julia to 0.6 beta if you can.

[sivark]

I see. Thanks.

[simonbyrne]

The prebuilt tarball from http://julialang.org/downloads/ should also still be okay.