Pkg.jl: Fail to download registries from Pkg Server on Windows

[KristofferC]

In the logs (e.g. https://ci.appveyor.com/project/JuliaLang/pkg-jl/builds/30415010)

    Cloning default registries into `C:\Users\appveyor\.julia`
┌ Warning: could not download https://pkg.julialang.org/registries
└ @ Pkg.Types C:\projects\pkg-jl\src\Types.jl:870
    Cloning registry from "https://github.com/JuliaRegistries/General.git"
      Added registry `General` to `C:\Users\appveyor\.julia\registries\General`

[StefanKarpinski]

This has been working though, right? I wonder what changed. It's possible pkg.julialang.org was down for a bit. @vdayanand, did you happen to take that down for a bit?

[KristofferC]

This has been working though, right?

I'm not sure. I feel to not miss stuff like this we need to run some tests where the normal git fallbacks are disabled. Otherwise, we make a mistake but CI keeps working.

[shihaoyan2018]

So,it is a bug? How can i use it on Windows,

[johnnychen94]

FYI, I can successfully fix this issue locally with the following changes

[System.Net.ServicePointManager]::SecurityProtocol =
-                    [System.Net.SecurityProtocolType]::Tls12;
+                    [System.Net.SecurityProtocolType]::SystemDefault;

in https://github.com/JuliaLang/Pkg.jl/blob/master/src/PlatformEngines.jl#L372-L373

This issue might be caused because supported protocols are limited to Ssl3 and Tls.

But looks like it would cause some other CI failures. Here's the log

Update:

Okay, it seems that using an older version Tls (v1.0) doesn't break other tests. Here's the log

[KristofferC]

@staticfloat is there a problem using Tls over Tls12?

[staticfloat]

Yes; the majority of the internet won't talk to you if you use Tls or Ssl3; you _must_ use TLS 1.2+ if you want to, for instance talk to GitHub. In the log that @johnnychen94 posted, note all the errors towards the bottom that say Could not create SSL/TLS secure channel."; that's the symptom of Powershell attempting to use an older protocol and the server failing to set things up.

I'm not sure what the issue with https://pkg.julialang.org would be in the log you posted; the server itself will respond with TLS v1.2 as you can see by running curl -v --tlsv1.2 --HEAD 'https://pkg.julialang.org'; I'm thinking that the issue in your logs is something separate from @johnnychen94's issues. I think @johnnychen94 needs to install the Windows WMF3 and TLS easy fixes, as described here: https://julialang.org/downloads/platform/

[johnnychen94]

I'm thinking that the issue in your logs is something separate from @johnnychen94's issues.

I've asked @findmyway to test PkgServer in windows for several Pkg versions:

| branch | Tls12 | remove Tls |
| --- | --- | --- |
| julia 1.4.0-rc1 | ❌ | no test |
| release-1.4(462eaf8)| ✅( ✅in CI) | ✅ |
| master(5345ec0) | ✅(❌ in CI) | ✅ |

To make sure it's not due to the unstable network connection, the test used a pkg server in LAN. He also tested the cases with Julia starting from cmd and PowerShell, and the results are the same.

As a conclusion, the issue I met is already fixed (but not in rc1), while the one @KristofferC met seems more related to a CI issue. Would it be reasonable to release an rc2 and ask users to test?

logs and screenshots

julia> versioninfo()
Julia Version 1.4.0-rc1.0
Commit b0c33b0cf5 (2020-01-23 17:23 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i5-7300U CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

(@v1.4) pkg> st Pkg
Status `C:\Users\juti\.julia\environments\v1.4\Project.toml`
  [54cfe95a] Pkg v1.4.0 [`..\..\..\workspace\github\Pkg.jl`]

Here are two screenshots:

julia 1.4.0-rc1.0 ❌

release-1.4 and master ✅

[shihaoyan2018]

Julia version : Julia-1.4-rc2

I still can't use PkgServer made by myself on Julia-1.4-rc2.But i can use kr.pkg.julialang.org and pkg.julialang.org on it.

julia> versioninfo()
Julia Version 1.4.0-rc2.0
Commit b99ed72c95 (2020-02-24 16:51 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: AMD Ryzen 3 3200G with Radeon Vega Graphics
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-8.0.1 (ORCJIT, znver1)
Environment:
JULIA_PKG_SERVER = http://45.77.28.188:8000

(@v1.4) pkg> add Images
Cloning default registries into C:\Users\Shihaoyan\.julia\
┌ Warning: could not download http://45.77.28.188:8000/registries
└ @ Pkg.Types D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Types.jl:880
Cloning registry from "https://github.com/JuliaRegistries/General.git"

[johnnychen94]

I guess the reason is that the downloader engine used by Pkg in Windows is hardcoded to use https TLS1.2 (if I understand it right), and you're using pkgserver without SSL enabled. If you can set up SSL for your pkgserver it might be good to go.

[shihaoyan2018]

ok,I will try it.Thanks.

[shihaoyan2018]

PkgServer made by myself with SSL can be used on julia-1.4-rc2.
So,the problem maybe is windows tls setting.

julia> versioninfo()
Julia Version 1.4.0-rc2.0
Commit b99ed72c95 (2020-02-24 16:51 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: AMD Ryzen 3 3200G with Radeon Vega Graphics
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-8.0.1 (ORCJIT, znver1)
Environment:
JULIA_PKG_SERVER = https://yanshihao.wang/

(@v1.4) pkg> add HTTP
Cloning default registries into C:\Users\Shihaoyan\.julia
Added registry General to C:\Users\Shihaoyan\.julia\registries\General
Resolving package versions...
Installed IniFile ───── v0.5.0
Installed MbedTLS_jll ─ v2.16.0+1
Installed HTTP ──────── v0.8.12
Installed MbedTLS ───── v1.0.0

[StefanKarpinski]

The Pkg code does not allow insecure connections to non-local servers. If you want to connect to a remote server you may want to use an SSH tunnel or something like that.

[KristofferC]

Well, we'll see if this becomes a problem in practice...