Julia: Update problems with corporate proxy with user and pass

Created on 8 Mar 2017 · 47Comments · Source: JuliaLang/julia

Hello,

I know this question was asked a hundred times in the web, but I did'nt find any solution. So I ask again for my special situation:

I istalled a fresh Julia 0.5.1 64-bit on my windows 7 computer. There is no program "git" or "git.exe" on the computer (which is part of several soutions) and it did not come with Julia.

If I call e.g. Pkg.status() (or any other Pkg related command) then I get:

INFO: Initializing package repository C:\Users\di29394\.julia\v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:OS, Failed to set proxy: Falscher Parameter.
)
 in macro expansion at .\libgit2\error.jl:99 [inlined]
 in clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\repository.jl
:191
 in #clone#112(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredent
ials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:327
 in (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::St
ring) at .\<missing>:0
 in (::Base.Pkg.Dir.##4#6{String,String})() at .\pkg\dir.jl:49
 in cd(::Base.Pkg.Dir.##4#6{String,String}, ::String) at .\file.jl:48
 in init(::String, ::String) at .\pkg\dir.jl:47
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::Base.TTY, ::Vararg{Base.TTY,N}) a
t .\pkg\dir.jl:28
 in status(::Base.TTY) at .\pkg\pkg.jl:136 (repeats 2 times)

The environment variables HTTP_PROXY, HTTPS_PROXY, FTP_PROXY,http_proxy,https_proxy and ftp_proxy are set in a proper way, i.e. http://user:pass@proxy:port and work well e.g. for all of my Python installations.
I also set C:/users/user/.gitconfig to:

[http]
        proxy = proxy:port
[https]
        proxy = proxy:port
[url "https://"]
        insteadof = git://

and also tried to set

[http]
        proxy = user:pass@proxy:port
[https]
        proxy = user:pass@proxy:port
[url "https://"]
        insteadof = git://

The error is always the same.

What to do now?

P.S.: user, pass, proxy and port were set to the actual values :-)

libgit2

Source

Milou314

All 47 comments

Hi @Milou314! We use the GitHub issue tracker for bug reports and feature request. Since this is more of a question, it should be posted to Discourse. Thanks!

ararslan on 8 Mar 2017

@ararslan, i can clearly see a bug report here. You don't?

lobingera on 9 Mar 2017

Well, the ending question makes it sound more like a request for help. The OP does not say explicitly whether the steps he followed are those recommended in the manual, or whether this issue is precisely about the fact that no documentation exists about proxies.

Anyway somebody familiar with the libgit2 code will tell whether it's a bug or not.

nalimilan on 9 Mar 2017

@nalimilan, maybe re-opening increases the visibility for libgit2 experts?

lobingera on 9 Mar 2017

Ref: https://discourse.julialang.org/t/update-problems-behind-a-corporate-proxy-with-user-and-pass/2560

fredrikekre on 9 Mar 2017

@fredrikekre If i'm not mistaken, some of the experts regularly write, they are better reachable on github.
And second, this thread on discourse didn't acquire many answers until now.
Do you have one?

lobingera on 9 Mar 2017

😕2

No, I just cross-referenced.

fredrikekre on 9 Mar 2017

I suspect it may have something to do with this issue: https://github.com/libgit2/libgit2/issues/2106. Could you try the latest nightly build (which has a newer version of libgit2) and see if that fixes it?

simonbyrne on 13 Mar 2017

Hi, I downloaded the nightly build. This is what happened:

C:\Julia\bin>set HTTPS_PROXY=http://...:...@...:...

C:\Julia\bin>set HTTP_PROXY=http://...:...@...:...

C:\Julia\bin>set FTP_PROXY=http://...:...@...:...

C:\Julia\bin>julia.exe
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-pre.alpha.140 (2017-03-14 00:06 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 58ed252* (0 days old master)
|__/                   |  x86_64-w64-mingw32

julia> Pkg.status()
INFO: Initializing package repository C:\Users\di29394\.julia\v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:OS, failed to send request: Das Zeitlimit für den Vorgang wurde erreicht.)
Stacktrace:
 [1] macro expansion at .\libgit2\error.jl:99 [inlined]
 [2] clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\repository.j
l:287
 [3] #clone#98(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredent
ials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:474
 [4] (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::S
tring) at .\<missing>:0
 [5] (::Base.Pkg.Dir.##8#10{String,String})() at .\pkg\dir.jl:55
 [6] cd(::Base.Pkg.Dir.##8#10{String,String}, ::String) at .\file.jl:58
 [7] init(::String, ::String) at .\pkg\dir.jl:53
 [8] #cd#1(::Array{Any,1}, ::Function, ::Function, ::Base.TTY, ::Vararg{Base.TTY,N} w
here N) at .\pkg\dir.jl:28
 [9] status(::Base.TTY) at .\pkg\pkg.jl:143 (repeats 2 times)

julia>

Das Zeitlimit für den Vorgang wurde erreicht means, that the time limit for the current task was exeeded. The same proxy settings for Python work well.

Well, at least the error is now different than before.

Gretings from Germany!

Milou314 on 14 Mar 2017

@Milou314, afaiu even the 'new' libgit2 would read configuration form a .gitconfig or from ENV?

lobingera on 14 Mar 2017

Hmm, it seems that 0.6 is completely ignoring the proxy settings (i.e. giving an invalid proxy doesn't throw any error on my machine).

Unfortunately this is too far out of my experience. If anyone else wants to take it up you can use poxyproxy, which is what libgit2 use for testing.

simonbyrne on 14 Mar 2017

@simonbyrne; looks like this is a OS-specific problem. I just installed (actually only unpacked) the linux nightly 0.6 and Pkg.init() could correctly fetch METADATA via http proxy set via export in shell.

lobingera on 14 Mar 2017

Are you behind a proxy? My concern is that it's completely ignoring the proxy settings.

simonbyrne on 14 Mar 2017

@simonbyrne; i'm currently at my workplace, which is firewalled and proxied and the linux i tested is even further away from the internet connection (cloud-instance far away...).

xxxxxxxx@euca-10-211-211-211:~$ ./julia-58ed252f6c/bin/julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-pre.alpha.140 (2017-03-14 00:06 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 58ed252 (0 days old master)
|__/                   |  x86_64-pc-linux-gnu

julia> ENV
Base.EnvHash with 24 entries:
  "XDG_SESSION_ID"     => "2052"
  "TERM"               => "xterm"
  "SHELL"              => "/bin/bash"
  "SSH_CLIENT"         => "10.948.199.964 55847 22"
  "SSH_TTY"            => "/dev/pts/0"
  "http_proxy"         => "http://10.944.1.90:8080/"
  "USER"               => "xxxxxxxx"
  "LS_COLORS"          => "rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01…
  "ftp_proxy"          => "http://10.944.1.90:8080/"
  "MAIL"               => "/var/mail/xxxxxxxx"
  "PATH"               => "/home/xxxxxxxx/bin:/home/xxxxxxxx/.local/bin:/usr/lo…
  "PWD"                => "/home/xxxxxxxxx"
  "LANG"               => "en_US.UTF-8"
  "https_proxy"        => "http://10.944.1.90:8080/"
  "SHLVL"              => "1"
  "HOME"               => "/home/xxxxxxxx"
  "no_proxy"           => "localhost,127.0.0.1,instance-data,169.254.169.254,.n…
  "LOGNAME"            => "xxxxxxxx"
  "SSH_CONNECTION"     => "10.948.199.964 55847 10.211.211.211 22"
  "LESSOPEN"           => "| /usr/bin/lesspipe %s"
  "XDG_RUNTIME_DIR"    => "/run/user/1000"
  "LESSCLOSE"          => "/usr/bin/lesspipe %s %s"
  "_"                  => "./julia-58ed252f6c/bin/julia"
  "OPENBLAS_MAIN_FREE" => "1"

julia> Pkg.init()
INFO: Initializing package repository /home/xxxxxxxx/.julia/v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
julia> Pkg.status()
No packages installed

lobingera on 14 Mar 2017

Now on a windows box in the same network i get:

julia> Pkg.init()
INFO: Initializing package repository C:\Users\xxxxxxxx\.julia\v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:OS, failed to send request: The operation time
d out
)
Stacktrace:
 [1] macro expansion at .\libgit2\error.jl:99 [inlined]
 [2] clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\reposit
ory.jl:287
 [3] #clone#98(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCr
edentials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:474
 [4] (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String
, ::String) at .\<missing>:0
 [5] (::Base.Pkg.Dir.##8#10{String,String})() at .\pkg\dir.jl:55
 [6] cd(::Base.Pkg.Dir.##8#10{String,String}, ::String) at .\file.jl:58
 [7] init(::String, ::String) at .\pkg\dir.jl:53
 [8] init() at .\pkg\pkg.jl:74

(inlc. setting of HTTPS_PROXY)

lobingera on 14 Mar 2017

My concern is that it's completely ignoring the proxy settings.

should they be visible in Libgit2 like ProxyOptions or getconfig?

lobingera on 14 Mar 2017

I have the same issue. This also affects proxies without user name and password and make 0.6 unusable for me at work. I see this as something that is critical to fix before 0.6 release.

mpastell on 15 Mar 2017

@mpastell What OS are you using? And what errors does, say,

LibGit2.clone("https://github.com/JuliaLang/Example.jl","example")

give in both 0.5 and 0.6 nightly?

simonbyrne on 15 Mar 2017

I can confirm that passwordless proxy settings (both set via ENV and .gitconfig) are ignored on the Windows 0.6 build. This is indeed a regression.

simonbyrne on 15 Mar 2017

@simonbyrne, so which windows build (post 0.5) should be able to read proxy setting correctly?

lobingera on 15 Mar 2017

I'm not sure I understand your question: from what I can tell, the 0.6 Windows nightlies do not read the proxy settings at all.

simonbyrne on 15 Mar 2017

If you call this a regression, then there should be windows version in the past that does Pkg via LibGit2 that worked with proxy settings.

lobingera on 15 Mar 2017

Well, 0.5 does. I don't know which commit broke it, but I would suspect it was the upgrade of LibGit2 from 0.24 to 0.25.

simonbyrne on 15 Mar 2017

I'm confused. But why then is it working with a 0.6 linux?

lobingera on 15 Mar 2017

I think libgit2 uses different libraries for the http stuff on Windows and Linux.

simonbyrne on 15 Mar 2017

Great.

lobingera on 15 Mar 2017

👎1

The release notes of libgit2 0.25 say:

Add git_proxy_options which is used by the different networking
implementations to let the caller specify the proxy settings instead of
relying on the environment variables.

mpastell on 15 Mar 2017

Ah, so that's what's going on (I should have remembered, I made the changes to the structs). This might explain the linux behaviour:

 * Do not attempt to connect through a proxy
 *
 * If built against libcurl, it itself may attempt to connect
 * to a proxy if the environment variables specify it.

simonbyrne on 15 Mar 2017

maybe enough to change the default value of that parameter then?

tkelman on 16 Mar 2017

@ararslan Has this been tested sucessfully?

lobingera on 17 Mar 2017

GitHub auto-closed this since I merged Simon's PR that had the text "fix #20948" in it. This shouldn't be closed until we can verify it's working as expected.

ararslan on 17 Mar 2017

For me the issue is fixed with the current nightly (8f97e6e, Windows 7 x64 ). I just tested and it works behind proxy with the same settings as in 0.5.1 (no authentication).

mpastell on 17 Mar 2017

I changed all as proposed by #21044 - well this results in:

C:\Julia\bin>Julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-pre.alpha.140 (2017-03-14 00:06 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 58ed252* (3 days old master)
|__/                   |  x86_64-w64-mingw32

julia> Pkg.status()
INFO: Initializing package repository C:\Users\di29394\.julia\v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:OS, failed to send request: Das Zeitlimit für den V
organg wurde erreicht.
)
Stacktrace:
 [1] macro expansion at .\libgit2\error.jl:99 [inlined]
 [2] clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\repository.j
l:287
 [3] #clone#98(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredent
ials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:474
 [4] (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::S
tring) at .\<missing>:0
 [5] (::Base.Pkg.Dir.##8#10{String,String})() at .\pkg\dir.jl:55
 [6] cd(::Base.Pkg.Dir.##8#10{String,String}, ::String) at .\file.jl:58
 [7] init(::String, ::String) at .\pkg\dir.jl:53
 [8] #cd#1(::Array{Any,1}, ::Function, ::Function, ::Base.TTY, ::Vararg{Base.TTY,N} w
here N) at .\pkg\dir.jl:28
 [9] status(::Base.TTY) at .\pkg\pkg.jl:143 (repeats 2 times)

julia>

AnacondaPython conda list still works well, i.e. no issue with proxy settings in environment variables.
Any suggestions?

Milou314 on 17 Mar 2017

@Milou314 The version you tested is too old: Version 0.6.0-pre.alpha.140 (2017-03-14 00:06 UTC). The possible fix was only merged yesterday.

nalimilan on 17 Mar 2017

Oh, sorry. I just changed the lines myself. So there are other changes not reported in #21044 ??
I'll try the new release asap.

Milou314 on 17 Mar 2017

C:\Julia\bin>julia.exe
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-pre.alpha.161 (2017-03-17 02:27 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 8f97e6e* (0 days old master)
|__/                   |  x86_64-w64-mingw32

julia> Pkg.status()
INFO: Initializing package repository C:\Users\di29394\.julia\v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:Net, invalid URL: 'myname:mypass@myproxy:myport'
)
Stacktrace:
 [1] macro expansion at .\libgit2\error.jl:99 [inlined]
 [2] clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\repository.j
l:287
 [3] #clone#98(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredent
ials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:474
 [4] (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::S
tring) at .\<missing>:0
 [5] (::Base.Pkg.Dir.##8#10{String,String})() at .\pkg\dir.jl:55
 [6] cd(::Base.Pkg.Dir.##8#10{String,String}, ::String) at .\file.jl:58
 [7] init(::String, ::String) at .\pkg\dir.jl:53
 [8] #cd#1(::Array{Any,1}, ::Function, ::Function, ::Base.TTY, ::Vararg{Base.TTY,N} w
here N) at .\pkg\dir.jl:28
 [9] status(::Base.TTY) at .\pkg\pkg.jl:143 (repeats 2 times)

julia>

myname:mypass@myproxy:myport I just changed for the post from the right values to dummies.

Milou314 on 17 Mar 2017

I would check the format for proxy url in .gitconfig as you now get an error from invalid URL instead of a timeout.

mpastell on 17 Mar 2017

@Milou314 Are you using environment variables or .gitconfig?

simonbyrne on 17 Mar 2017

Well, this was (finally) the right question! For Python I use environment variables. But I found somewehre in the web the proposal to generate .gitconfig. I did this, but obviously with wrong parameters (missing leading http://). I never got hints with preceeding Julia versions, that this is a problem. Now I corrected the settings and all runs well (just successfully called Pkg.add("IJulia")).

I also checked, wether the new version works without .gitconfig and just with environment variables. This works now well too.

Thank you all for fixing the mixed software and human bug ;)

Milou314 on 17 Mar 2017

Fantastic. I was also successful in emulating it in a VM via both a .gitconfig with:

[http]
proxy = http://user:pass@proxyhost:port

[https]
proxy = http://user:pass@proxyhost:port

and with the environment variables

ENV["HTTP_PROXY"] = "http://user:pass@proxyhost:port"
ENV["HTTPS_PROXY"] = "http://user:pass@proxyhost:port"

simonbyrne on 17 Mar 2017

@simonbyrne; you ran windows in VM? Or linux?

lobingera on 17 Mar 2017

yay, an easy fix for once. could use a doc note too now that we have a known good "how it should work" confirmation

(@Milou314 just editing .jl files in base isn't quite enough to change behavior since the standard library is precompiled, see the devdocs on building a system image for the step you were missing)

tkelman on 17 Mar 2017

Windows VM only.

simonbyrne on 17 Mar 2017

I am experiencing building error in Julia v1.0.2 on both packages ZipFile and LibCURL. I am in a corporate proxy that requires user and pass.

When I move my connection behind a VPN, I can build both packages without error. It seems the problem I get is similar to this old problem. Is there a regression?

jzazo on 3 Dec 2018

I am experiencing this issue on Julia version ≥ 1.0. It seems that my proxy settings are correct because I can git clone the packages directly from the terminal, but for some reason Julia does not recognize the proxy settings.

See the report in the discourse at https://discourse.julialang.org/t/cannot-add-packages-maybe-due-to-corporate-firewall/19352.

wsshin on 7 Jan 2019

@wsshin Does it work if you apply the configuration changes described at https://github.com/JuliaLang/julia/issues/20948#issuecomment-287306113? If not, please file a new issue.

nalimilan on 7 Jan 2019

No, the configuration does not work; our proxy server does not require login.

I will file a new issue.

wsshin on 7 Jan 2019

Was this page helpful?

0 / 5 - 0 ratings