Sdk: APT repository inaccessible

[zmeggyesi]

For a while now, the APT repository for dart (https://storage.googleapis.com/download.dartlang.org/linux/debian) has been returning an HTTP403 FORBIDDEN when queried by apt-get update orapt-get install. Directly opening the URL results in a GCS error message with the detail Anonymous users does not have storage.objects.get access to download.dartlang.org/linux/debian.. As such, the APT-method of installing the SDK is unusable right now.

This has apparently been going on for several weeks now - I've found the error going back two weeks, possibly earlier, but APT logs have been rotated out from before that.

[whesse]

This seems to be about the URL and cloud storage not supporting directory listing, only fetching files directly. I thought apt-get was working fine with this constraint, and only looked for specific files, and did not try and list directories. Which version of apt-get are you running, on which platform?

[zmeggyesi]

Various, actually:

A git-/build-server:

root@singularity:/# cat /etc/issue.net
Debian GNU/Linux buster/sid
root@singularity:/# apt-get --version
apt 1.5~beta1 (amd64)

A build slave:

root@serenity:/# cat /etc/issue.net
Debian GNU/Linux 9
root@serenity:/# apt-get --version
apt 1.4.6 (amd64)

Most interestingly, this combination works!

root@duality:~# cat /etc/issue.net
Ubuntu 16.04.2 LTS
root@duality:~# apt-get --version
apt 1.2.20 (amd64)

[malkia]

I've been affected by this too:

(zesty)malkia@localhost:~/p/bazel-tools$ sudo apt-get update
Ign:1 http://dl.google.com/linux/chrome/deb stable InRelease
Hit:2 http://storage.googleapis.com/bazel-apt stable InRelease
Hit:3 http://packages.microsoft.com/repos/vscode stable InRelease
Hit:4 http://dl.google.com/linux/chrome/deb stable Release
Err:5 https://storage.googleapis.com/download.dartlang.org/linux/debian stable InRelease
403 Forbidden [IP: 172.217.5.208 443]
Get:6 http://archive.ubuntu.com/ubuntu artful InRelease [237 kB]
Hit:8 http://archive.ubuntu.com/ubuntu artful-updates InRelease
Hit:9 http://archive.ubuntu.com/ubuntu artful-security InRelease

[alescdb]

Same error here on Ubuntu 17.10 with apt 1.5~rc4 (amd64)

Err:14 http://storage.googleapis.com/download.dartlang.org/linux/debian unstable InRelease                                        
  403  Forbidden

and/or

Err:15 https://storage.googleapis.com/download.dartlang.org/linux/debian stable InRelease
  403  Forbidden

[whesse]

Note that the line above the failing one in comment 4 shows a download from dl.google.com of Release from a directory succeeding, but clicking on the link gives a 404, just like from download.dartlang.org. So the problem seems not to be as simple as directory surfing disabled. Also, it is looking for an InRelease file, when all we provide is a Release file. The spec for repositories says both are supported. But that servers "SHALL" provide an InRelease file. We don't support this newer spec. The repository_creator tool we are using to create the signed repositories doesn't create it. I will file an issue with the makers of that tool to add an InRelease file.

[602p]

Same issue here on Debian 3.16:

Err:9 https://storage.googleapis.com/download.dartlang.org/linux/debian stable InRelease                                
  403  Forbidden [IP: 2607:f8b0:4009:808::2010 443]

[whesse]

The problem may be that it is returning a 403, not a 404. A message from the repository-creator author says:

. Are there plans to update repository_creator to create a new-style linux repository with an InRelease file?

Issue: https://github.com/dart-lang/sdk/issues/30512

Probably not, although if we switch to a wrapper script like I mentioned, that should fix it. But FWIW, I don't think the error here is that those files are missing. Apt has been looking for those files (and others, like optional translation files) for a long time, and unless something has changed very recently, only treats it as a warning if they're missing. The real problem is that cloud storage is returning a 403 auth error instead of a 404 notfound error, which apt apparently treats as a more serious issue, I seem to recall we had a similar problem with dl.google.com at one point, but that was fixed long ago, and apt works fine with the repos there.

[malkia]

Bazel's packages (for example) work without a problem:
http://storage.googleapis.com/bazel-apt

On Thu, Oct 19, 2017 at 10:28 AM, William Hesse [e-mail]
wrote:

The problem may be that it is returning a 403, not a 404. A message from
the repository-creator author says:

. Are there plans to update repository_creator to create a new-style linux
repository with an InRelease file?

Issue: #30512 https://github.com/dart-lang/sdk/issues/30512

Probably not, although if we switch to a wrapper script like I mentioned,
that should fix it. But FWIW, I don't think the error here is that those
files are missing. Apt has been looking for those files (and others, like
optional translation files) for a long time, and unless something has
changed very recently, only treats it as a warning if they're missing. The
real problem is that cloud storage is returning a 403 auth error instead of
a 404 notfound error, which apt apparently treats as a more serious issue,
I seem to recall we had a similar problem with dl.google.com at one
point, but that was fixed long ago, and apt works fine with the repos there.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/30512#issuecomment-337979319,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADKGr5Zy0HUQfxywdsGjBMpnbTdUEDTks5st4a7gaJpZM4O-KQG
.

--
Dimiter "malkia" Stanev,
ICQ: 21875894
[email protected]
[email protected]

[malkia]

I'm not sure whether this makes any difference, but the response header
from:
http://storage.googleapis.com/bazel-apt
returned this:
x-goog-metageneration
https://cloud.google.com/storage/docs/xml-api/reference-headers#xgoogmetageneration
:
4

and nothing like it, was returned by:
https://storage.googleapis.com/download.dartlang.org/linux/debian

(That maybe expected if there was an error to begin with, just noticed it -
Not much familiar with this myself)

On Thu, Oct 19, 2017 at 11:07 AM, Dimiter "malkia" Stanev [e-mail]
wrote:

Bazel's packages (for example) work without a problem: http://storage.
googleapis.com/bazel-apt

On Thu, Oct 19, 2017 at 10:28 AM, William Hesse [e-mail]
wrote:

The problem may be that it is returning a 403, not a 404. A message from
the repository-creator author says:

. Are there plans to update repository_creator to create a new-style
linux repository with an InRelease file?

Issue: #30512 https://github.com/dart-lang/sdk/issues/30512

Probably not, although if we switch to a wrapper script like I mentioned,
that should fix it. But FWIW, I don't think the error here is that those
files are missing. Apt has been looking for those files (and others, like
optional translation files) for a long time, and unless something has
changed very recently, only treats it as a warning if they're missing. The
real problem is that cloud storage is returning a 403 auth error instead of
a 404 notfound error, which apt apparently treats as a more serious issue,
I seem to recall we had a similar problem with dl.google.com at one
point, but that was fixed long ago, and apt works fine with the repos there.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/30512#issuecomment-337979319,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADKGr5Zy0HUQfxywdsGjBMpnbTdUEDTks5st4a7gaJpZM4O-KQG
.

--
Dimiter "malkia" Stanev,
ICQ: 21875894
[email protected]
[email protected]

--
Dimiter "malkia" Stanev,
ICQ: 21875894
[email protected]
[email protected]

[ghost]

Is there a fix for this? Desperately trying to get dart set up so that I can use a tool for visualizing AWS configs.

I guess I can just use https://storage.googleapis.com/dart-archive/channels/stable/release/latest/linux_packages/dart_1.24.2-1_amd64.deb

[whesse]

Yes, just download the debian package directly and install with dpkg -i

[MeanEYE]

This is not a satisfactory resolution to this issue and severely hinders any application of Dart as a server-side language as it would require system administrators to manually check versions and keep them up to date on servers. This is even further amplified if number of servers for maintenance is high. Very unprofessional.

More to the point, if this is your "solution" then Linux Install page needs to be updated to reflect this decision so people don't come in wondering that the hell is going on.

[Lisenish]

Do you have plans to resolve this issue? How Dart can become popular if it's official installing instructions doesn't work and language repository is broken several months and seems like Google doesn't care about it :(.

[tuxgirl]

Trying to install, and the .deb link on the install page, and the link given above both are failing for me...

This XML file does not appear to have any style information associated with it. The document tree is shown below.

NoSuchKey
The specified key does not exist.

I don't really want to build from source, and I find it rather frustrating that it is so hard to install the language!

[whesse]

We have a problem with the package builder right now, that we are trying to
fix, so the most recent stable and dev versions don't have .deb packages.

I'll try and fix this on Monday, at least the links. You can fetch the
latest versions (and older ones) as zip files at
https://www.dartlang.org/install/archive for all operating systems,
including Linux. They can be unzipped anywhere, and the dart-sdk/bin
directory added to your path.

Sorry about the trouble. You definitely don't have to build from source.

On Dec 17, 2017 06:50, "tuxgirl" [e-mail] wrote:

Trying to install, and the .deb link on the install page, and the link
given above both are failing for me...

This XML file does not appear to have any style information associated
with it. The document tree is shown below.

NoSuchKey
The specified key does not exist.

I don't really want to build from source, and I find it rather frustrating
that it is so hard to install the language!

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/30512#issuecomment-352234104,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AErapwd301sNcFbslGEcskgMfTLeIK3Aks5tBKungaJpZM4O-KQG
.

[whesse]

By changing the access rights on the download.dartlang.org bucket, I have made InRelease return a 404 instead of a 403 when fetched, including from an incognito window.

I think that apt-get should now work, for those users who found it broken before. Please try this and verify that it is fixed.

I have also fixed the debian package builder, and built new signed repositories serving the latest stable and dev versions of Dart as debian packages, so you should now be able to fetch 1.24.3 or 2.0.0-dev.14.0, depending on whether you fetch stable or unstable.

[zmeggyesi]

Hi,

I can confirm that it's working on Debian Buster/APT 1.6~alpha5 (amd64).
I'm seeing Dart v1.24.3-1 after an update.

Thank you for the fix!

Best Regards,

Zalan Meggyesi
Chief Support Engineer
Skawa Innovation Kft.

Phone: [phone]
Mobile: [phone]

On Wed, Dec 20, 2017 at 5:25 PM, William Hesse [e-mail]
wrote:

By changing the access rights on the download.dartlang.org bucket, I have
made InRelease return a 404 instead of a 403 when fetched, including from
an incognito window.

I think that apt-get should now work, for those users who found it broken
before. Please try this and verify that it is fixed.

I have also fixed the debian package builder, and built new signed
repositories serving the latest stable and dev versions of Dart as debian
packages, so you should now be able to fetch 1.24.3 or 2.0.0-dev.14.0,
depending on whether you fetch stable or unstable.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/30512#issuecomment-353110604,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APfHbkZ_2BlcJ2hIeVKf1k2i0kcnqCHWks5tCTT5gaJpZM4O-KQG
.

[MeanEYE]

Confirming working state on Debian Testing. Thanks!