Temurin-build: Download extremely slow. How about sharing torrents to spare bandwidth?

[LigH-de]

Trying to download OpenJDK 11, 174 MB at 20 KB/s, 2 hours estimated. Slows down even further (10 KB/s) during the attempt.

LibreOffice is provided via torrents. Might be useful for this project as well if bandwidth is an issue for you.

[aahlenst]

Releases are hosted on GitHub (e.g. https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.5%2B10/OpenJDK11U-jdk_x64_linux_hotspot_11.0.5_10.tar.gz). GitHub stores and distributes them with Amazon S3. So it's unlikely that it is an upstream problem. I just downloaded the aforementioned tarball with 5,07 MB/s.

[LigH-de]

I'm still downloading a precompiled Windows x64 MSI installer, almost 30 of 174 MB so far... at 5 KB/s now.

[LigH-de]

34.3 MB ... download stalled. The browser displays less than 1 KB/s.

[karianna]

34.3 MB ... download stalled. The browser displays less than 1 KB/s.

Mine comes down in about 65 seconds, I suspect you've got a localized issue.

[LigH-de]

In this case, I would have to blame Deutsche Telekom... pseudo wanna-be broadband connection in rural areas.

[aahlenst]

@LigH-de I'm sorry that your downloads are slow, but so far there's no indication that the problem is on our side. GitHub Status and AWS Status are all green, too. If you want help to diagnose your problem, please provide at least a traceroute.

Such problems are usually caused by switch ports that are over capacity at some transit point, either because too much traffic was routed in the wrong direction (fixable by changing the routing table) or because some ISP is too cheap to upgrade the peering ports or buy more transit capacity. In any case, your best bet is to complain to your ISP and GitHub (in that order). They are in the position to help. Provide them at least a traceroute.

[LigH-de]

Thank you. It is certainly a temporal issue. Always around the same times, some services get unreliable (e.g. Twitch streams keep stalling, desyncing, disconnecting), other services are not harmed.

Right now I just downloaded it with 2 MB/s in 1 minute.

[jpstotz]

AdoptOpenJDK downloads are also very very slow via my 50MBit DSL connection (Germany - Deutsche Telekom VDSL, perfectly working usually more than 5MB/sec dl speed). I usually get 30-50 KB/sec - therefore a typical OpenJDK download takes more than 1 hour.
I assume that this is some problem between Deutsche Telekom and Amazon (and "telia.net" according to tracert the network system that connects them both).

The used download server is
52.216.161.35 (github-production-release-asset-2e65be.s3.amazonaws.com) according to some IP location tracker this IP is located in Virginia (USA). The data is transmitted via IPv4.

[aahlenst]

The problem with Deutsche Telekom is that they want other providers to buy transit. That’s why all their other links are saturated and you get abysmal download speeds. The best thing you can do is complain to both DTAG and GitHub in the hope that they start to exchange traffic via a direct link.

[LigH-de]

I just got a download of about 8 MB/s. Maybe they negotiated something?

[LigH-de]

Similar reports in the doom9 forum regarding AviSynth+; I do not know which ISP this reporter uses, but I believe he resides in Poland, so might have to pass through a German backbone.

Trying to contact github support about it.

[Ianmcmill]

Similar reports in the doom9 forum regarding AviSynth+; I do not know which ISP this reporter uses, but I believe he resides in Poland, so might have to pass through a German backbone.

Trying to contact github support about it.

I do have the same. Same provider as you.

[RouHim]

Same problems here, also germany. Provider is 1&1.

Edit: fixed the issue with a non european ip.

[LigH-de]

Today I tried to notify Deutsche Telekom (with our company's priority support) as well as Telia. I suppose the throttling is somewhere between their backbone networks...

[Nuc1eoN]

Same for me, I am using Deutsche Telekom too. This happens with all github downloads.

Using my VPN client (also using a german IP) I can downlaod at full speed.

So this is an issue with our ISP? Is there anything we can do about it?

[samba2]

Same here (also Deutsche Telekom). I worked around it by using Opera and its build-in VPN.

[LigH-de]

I don't know if it helps when many people complain about the same issue (in the Telekom Kundencenter web portal, not the phone hotline). But better try than shrug.

[reinhardjurk]

Same issue here (Deutsche Telekom), thanks for the hint with Opera. Will complain at Deutsche Telekom. Interestingly also over T-Mobile (same company).

[tpavesi]

Dowloading at 44kb/s

|------------------------------------------------------------------------------------------|
| Statistics |
| Host - % | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
| 10.0.0.1 - 0 | 14 | 14 | 0 | 0 | 1 | 1 |
| 10.73.200.1 - 0 | 14 | 14 | 8 | 13 | 22 | 15 |
|nrth-core-2a-xe-10010-0.network.virginmedia.net - 0 | 14 | 14 | 10 | 14 | 24 | 16 |
| No response from host - 100 | 2 | 0 | 0 | 0 | 0 | 0 |
| No response from host - 100 | 2 | 0 | 0 | 0 | 0 | 0 |
| m686-mp2.cvx1-b.lis.dial.ntli.net - 10 | 11 | 10 | 0 | 21 | 28 | 28 |
| No response from host - 100 | 2 | 0 | 0 | 0 | 0 | 0 |
| uk-lon01b-ri1-ae-25-0.aorta.net - 0 | 14 | 14 | 18 | 21 | 30 | 18 |
| ldn-b1-link.telia.net - 0 | 14 | 14 | 17 | 21 | 31 | 19 |
| No response from host - 100 | 2 | 0 | 0 | 0 | 0 | 0 |
| No response from host - 100 | 2 | 0 | 0 | 0 | 0 | 0 |
| ffm-bb2-link.telia.net - 10 | 11 | 10 | 0 | 37 | 41 | 37 |
| ffm-b1-link.telia.net - 0 | 14 | 14 | 34 | 40 | 59 | 35 |
| github-ic-350972-ffm-b1.c.telia.net - 0 | 14 | 14 | 31 | 37 | 48 | 34 |
| No response from host - 100 | 2 | 0 | 0 | 0 | 0 | 0 |
| No response from host - 100 | 2 | 0 | 0 | 0 | 0 | 0 |
| lb-140-82-118-4-ams.github.com - 10 | 11 | 10 | 0 | 41 | 44 | 41 |
|________________________________________________|______|______|______|______|______|______|

[LigH-de]

So there is Telia involved again... but hard to know who to blame in this battle between big backbone ISP's.

[aahlenst]

@tpavesi Is Virgin Media owned by Liberty Global? Traceroute looks like that (aorta.net). Liberty Global is a notorious offender in that regard. Complain to their customer support.

@LigH-de Telia is one of the biggest providers of transit connectivity. Not their fault if the parties involved don't want to upgrade the link.

[tpavesi]

Virgin media is owned by Liberty Global. There was a huge outage in UK in
the past few days exactly where you mentioned.

On Mon, 27 Apr 2020 at 07:35, Andreas Ahlenstorf [e-mail]
wrote:

@tpavesi https://github.com/tpavesi Is Virgin Media owned by Liberty
Global? Traceroute looks like that (aorta.net). Liberty Global is a
notorious offender in that regard. Complain to their customer support.

@LigH-de https://github.com/LigH-de Telia is one of the biggest
providers of transit connectivity. Not their fault if the parties involved
don't want to upgrade the link.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/AdoptOpenJDK/openjdk-build/issues/1349#issuecomment-619759818,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAXA2GBZAA52NG53YZMUSZ3ROURR5ANCNFSM4JFRUSVA
.

[LigH-de]

Indeed. Twitch broke as well, so the audience of that case was large.

[2wing]

An observation from rural France on copper and 100kbs max (oh yes).
Same problem trying to get "AdoptOpenJDK" ... but have found an 'agricultural' workround.
If I leave browser open and machine undisturbed the download does as others have found ... a few seconds then breaks ... repeat a few times until a few percent then crashes/closes/ends.
But .. if I continue to use the browser and move between pages this seems to prod the beast back to life.
Had to try this a few times but seems to be repeatable ... in that the download falls asleep.
Dont know if this is useful but it might help in the meantime.
Win7pro64+Opera

[Eskibear]

@karianna
FYI that I'm working on VS Code Coding pack for Java, which will download AdoptOpenJDK for users if they don't have one. For this moment from the statistics, I find about 6% users failed to download the JDK after 3 retries, which is not a small number. Is there any idea to improve it?

BTW, besides those ISPs mentioned in above comments, with almost all ISPs in China, we have unstable access to GitHub/AWS.

[aahlenst]

The problem is not a lack of bandwidth at the source. The problem are saturated interconnects between ISPs. There's not much we can do as long as we delegate the handling of the downloads to GitHub. We're in the passenger's seat so to speak.

There are ways to improve the situation, but they make things more complicated and require significant work and maybe money. Money is less of a problem, work is because we're short-handed.

Options I see:

• Piping downloads through the API instead of redirecting to GitHub. Wastes resources on streaming binaries, but they can be cached by Cloudflare. We'd need an ICP license from China to cache content there and enable Cloudflare China CDN. Combining the API with downloads is not something that I want, however.
• Develop a program that can sync a local directory on a machine/Azure Blob Storage with our API/GitHub (releases only). This would enable us to host our downloads ourselves (probably again via Cloudflare) and give the option to host mirrors (no ICP license required because someone else would host it). Requires the introduction of GPG signing of binary checksums (should be done anyway). For China, see above. Problem: Delay when synchronizing API, GitHub and local machine/Azure Blob Storage.
• Do something smart with Cloudflare Workers. For China, see above.

If somebody wants to work on it, please get in touch.

[Eskibear]

Thanks for the information. It looks that an ideal solution would be GitHub/AWS to work with different ISPs, then we benefit for free 🤷 .

About mirrors, below is a finding, just FYI in case it's useful.
A mirror hosted by Tsinghua University in China: https://mirrors.tuna.tsinghua.edu.cn/AdoptOpenJDK/
and the original ticket for it. (It's in Chinese only and you might translate the page.)

[aahlenst]

Thanks for the link to that Chinese mirror. https://github.com/tuna/tunasync-scripts/blob/master/adoptopenjdk.py is the script they are using. Doing something like that is one potential way forward, but as I said, and I cannot stress it enough, this requires verification of checksums which is a lot easier with GPG.

[Dom1n1c]

Same here (also Deutsche Telekom). I worked around it by using Opera and its build-in VPN.

Same for me.. slow in Chrome/Opera/Edge without VPN
(Yes, my download is a mqtt-client, but expierienced the same for JDK)

When using VPN (for instance Operas inbuilt client)
Download raises to at least 500KB/s

[Siedlerchr]

Thanks for the hint with the chinese mirror. Much faster than using the Github download directly

[getashraf]

I am using 1&1 (Deutsche Telekom) and still facing the same issue.
I am getting 1KB/s sometimes :-(

Any solution for this?

[bmarwell]

@getashraf I opened a ticket at Deutsche Telekom Kundenportal. Once this is resolved, I will demand some money back.
I added a link to this article: https://www.golem.de/news/deutsche-telekom-peering-zu-us-servern-scheint-langsam-zu-sein-2012-152982.html

But expect calls from Telekom. They are forced to randomly call you whether the problem persists or whether you tempered with your router. They also insist that you have a working connection as they see only your DSL sync speed at the help desk.

[varmandra]

There is an official statement from German Telekom
https://telekomhilft.telekom.de/t5/Telefonie-Internet/Downloadgeschwindigkeiten-von-GitHub-mal-wieder-unterirdisch/td-p/3873484/page/11

I don't know how true this is, because it is alway easier to say it is the failure of an other service.

BdT
Varmandra

[JanFeld]

@getashraf I opened a ticket at Deutsche Telekom Kundenportal. Once this is resolved, I will demand some money back.
I added a link to this article: https://www.golem.de/news/deutsche-telekom-peering-zu-us-servern-scheint-langsam-zu-sein-2012-152982.html

But expect calls from Telekom. They are forced to randomly call you whether the problem persists or whether you tempered with your router. They also insist that you have a working connection as they see only your DSL sync speed at the help desk.

This thread has been marked as resolved but actually isn't. I'm still downloading with about 4-5 kbit/s and shortly after everything stops downloading. As stated in the official response it's not their fault. I'm not sure about this. Nevertheless it has been marked as resolved...

[aahlenst]

I'm not sure about this.

Why?

[JanFeld]

They've had problems every now and then when every other provider didn't had these problems. Additionally it's a kind of a general mistrust in Telekom support. Problems are either unresolved and marked as solved due to "wrong departement" and/or it takes a long way through support levels with waiting on phone for hours. I've had the same problem a few years ago.

[aahlenst]

They've had problems every now and then when every other provider didn't had these problems. Additionally it's a kind of a general mistrust in Telekom support. Problems are either unresolved and marked as solved due to "wrong departement" and/or it takes a long way through support levels with waiting on phone for hours. I've had the same problem a few years ago.

Ah, I see I misread. I thought you implied the problem is at Adopt.

[msugakov]

Downloading with lftp using multiple concurrent connections helps ease the pain a bit. E.g.

lftp -c "pget -c -n 20 https://github.com/mikefarah/yq/releases/download/3.4.1/yq_linux_amd64"

Yet download times are still far from normal.

[rhuss]

@getashraf I opened a ticket at Deutsche Telekom Kundenportal. Once this is resolved, I will demand some money back.
I added a link to this article: https://www.golem.de/news/deutsche-telekom-peering-zu-us-servern-scheint-langsam-zu-sein-2012-152982.html

@bmarwell have you ever heard back from them ? I did the same with a "Stoerungsmeldung" but that was a complete disaster. They wanted to send a technician to my house and/or the DSLAM to "fix" this" and when I called back to cancel that the hotline was completely clueless when I told about issues in the backbone which only can be circumvented to not use the standard Telekom routing (e.g. via VPN). She then wanted to sell me their "Computerhilfe" service. Seriously.

[bmarwell]

@getashraf I opened a ticket at Deutsche Telekom Kundenportal. Once this is resolved, I will demand some money back.
I added a link to this article: https://www.golem.de/news/deutsche-telekom-peering-zu-us-servern-scheint-langsam-zu-sein-2012-152982.html

@bmarwell have you ever heard back from them ? I did the same with a "Stoerungsmeldung" but that was a complete disaster. They wanted to send a technician to my house and/or the DSLAM to "fix" this" and when I called back to cancel that the hotline was completely clueless when I told about issues in the backbone which only can be circumvented to not use the standard Telekom routing (e.g. via VPN). She then wanted to sell me their "Computerhilfe" service. Seriously.

Deutsche Telekom wanted to provide a solution by last week. 🙄 I'd say, better ask the Twitter team to call you back. They do know more about what's going on. I also told the supporter to teach the 1st level team about this issue. Didn't happen, as it seems.

So, nag the Twitter team. I think it's @telekom_hilft or similar.

[bottkars]

worked for me with NordVPN . provider Deutsche telekom Sucks with Amazon

[nurgasemetey]

Downloading with lftp using multiple concurrent connections helps ease the pain a bit. E.g.

lftp -c "pget -c -n 20 https://github.com/mikefarah/yq/releases/download/3.4.1/yq_linux_amd64"

Yet download times are still far from normal.

Your solution is awesome, it worked.