Core: File has unexpected size https://packages.microsoft.com/ubuntu/16.04/prod/dists/xenial/

Created on 24 Jun 2020  路  33Comments  路  Source: dotnet/core

Issue Title

It appears the checksums in the InRelease file have not been updated after a release.

General

2020-06-24T11:29:07.1014141Z Step 37/72 : RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
2020-06-24T11:29:07.1428743Z  ---> Running in 8f1f1f72af1b
2020-06-24T11:29:07.4318822Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2020-06-24T11:29:07.4324906Z                                  Dload  Upload   Total   Spent    Left  Speed
2020-06-24T11:29:07.4326186Z 
2020-06-24T11:29:07.5018269Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: apt-key output should not be parsed (stdout is not a terminal)
2020-06-24T11:29:07.5511675Z 
2020-06-24T11:29:07.5512807Z 100   983  100   983    0     0   8191      0 --:--:-- --:--:-- --:--:--  8191
2020-06-24T11:29:08.0488990Z OK
2020-06-24T11:29:09.0280731Z Removing intermediate container 8f1f1f72af1b
2020-06-24T11:29:09.0281722Z  ---> 4116ce353101
2020-06-24T11:29:09.0282130Z Step 38/72 : RUN curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list | tee /etc/apt/sources.list.d/msprod.list
2020-06-24T11:29:09.0788654Z  ---> Running in 7172bb67770f
2020-06-24T11:29:09.3827705Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2020-06-24T11:29:09.3828204Z                                  Dload  Upload   Total   Spent    Left  Speed
2020-06-24T11:29:09.3828460Z 
2020-06-24T11:29:09.4058013Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2020-06-24T11:29:09.4059060Z 100    79  100    79    0     0   3434      0 --:--:-- --:--:-- --:--:--  3434
2020-06-24T11:29:09.4059838Z deb [arch=amd64] https://packages.microsoft.com/ubuntu/16.04/prod xenial main
2020-06-24T11:29:10.0336367Z Removing intermediate container 7172bb67770f
2020-06-24T11:29:10.0337490Z  ---> b41c61050097
2020-06-24T11:29:10.0337947Z Step 39/72 : RUN apt-get update
2020-06-24T11:29:10.0794144Z  ---> Running in 21de1eb50b1e
2020-06-24T11:29:10.4557979Z Hit:1 http://deb.debian.org/debian buster InRelease
2020-06-24T11:29:10.4562997Z Hit:2 http://deb.debian.org/debian buster-updates InRelease
2020-06-24T11:29:10.4563741Z Hit:3 http://security.debian.org/debian-security buster/updates InRelease
2020-06-24T11:29:10.4611596Z Get:4 https://packages.microsoft.com/ubuntu/16.04/prod xenial InRelease [4003 B]
2020-06-24T11:29:10.9540746Z Get:5 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 Packages [142 kB]
2020-06-24T11:29:10.9541253Z Err:5 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 Packages
2020-06-24T11:29:10.9541718Z   File has unexpected size (142137 != 142079). Mirror sync in progress? [IP: 13.90.56.68 443]
2020-06-24T11:29:10.9545512Z   Hashes of expected file:
2020-06-24T11:29:10.9546659Z    - Filesize:142079 [weak]
2020-06-24T11:29:10.9568588Z    - SHA512:1e2d73b7082e18c33f3b4a6a584b7929344cdc87b5c96d25e2dcb3e84d72733014364d098028d9dcaddb7fd194ec11f3a74df489a9f05f67e84638cd033525e1
2020-06-24T11:29:10.9569494Z    - SHA256:32a93cdfa7390dbc737271d543f7ae198d267a09aba44ca3e27beca49cc8471c
2020-06-24T11:29:10.9570119Z    - SHA1:3243a484d2fa337b05e6e4be925d4ed96ba2ca38 [weak]
2020-06-24T11:29:10.9571059Z    - MD5Sum:70ea695335d795cdf852fcdac44cb18e [weak]
2020-06-24T11:29:10.9573326Z   Release file created at: Fri, 19 Jun 2020 08:51:18 +0000
2020-06-24T11:29:10.9619688Z Fetched 4003 B in 1s (7692 B/s)
2020-06-24T11:29:11.6849439Z Reading package lists...
2020-06-24T11:29:11.7104230Z E: Failed to fetch https://packages.microsoft.com/ubuntu/16.04/prod/dists/xenial/main/binary-amd64/Packages.bz2  File has unexpected size (142137 != 142079). Mirror sync in progress? [IP: 13.90.56.68 443]
2020-06-24T11:29:11.7104962Z    Hashes of expected file:
2020-06-24T11:29:11.7105626Z     - Filesize:142079 [weak]
2020-06-24T11:29:11.7106397Z     - SHA512:1e2d73b7082e18c33f3b4a6a584b7929344cdc87b5c96d25e2dcb3e84d72733014364d098028d9dcaddb7fd194ec11f3a74df489a9f05f67e84638cd033525e1
2020-06-24T11:29:11.7107166Z     - SHA256:32a93cdfa7390dbc737271d543f7ae198d267a09aba44ca3e27beca49cc8471c
2020-06-24T11:29:11.7107756Z     - SHA1:3243a484d2fa337b05e6e4be925d4ed96ba2ca38 [weak]
2020-06-24T11:29:11.7108289Z     - MD5Sum:70ea695335d795cdf852fcdac44cb18e [weak]
2020-06-24T11:29:11.7108667Z    Release file created at: Fri, 19 Jun 2020 08:51:18 +0000
2020-06-24T11:29:11.7109071Z E: Some index files failed to download. They have been ignored, or old ones used instead.
2020-06-24T11:29:11.9092606Z The command '/bin/sh -c apt-get update' returned a non-zero code: 100
2020-06-24T11:29:11.9218316Z ##[error]The command '/bin/sh -c apt-get update' returned a non-zero code: 100
2020-06-24T11:29:11.9237722Z ##[error]The process '/usr/bin/docker' failed with exit code 100
area-setup

Most helpful comment

All 33 comments

Same issue here with the same package. Please let us know if we can help giving more info.

Same applies also for:
https://packages.microsoft.com/debian/10/prod/dists/buster/main/binary-amd64/Packages.bz2

Get:5 https://packages.microsoft.com/debian/10/prod buster/main amd64 Packages [43.7 kB]
 Err:5 https://packages.microsoft.com/debian/10/prod buster/main amd64 Packages
   File has unexpected size (43969 != 43653). Mirror sync in progress?
   Hashes of expected file:
    - Filesize:43653 [weak]
    - SHA512:770b9079119523e27b68efe2742e196564815069321388ad9b1ce270da25b9c7f432fb57b210a6af4788d4460c1f80877c2a34de3a89e3d1584d74ed16c5f7c4
    - SHA256:cd6baf1fba149709cb0a9904cb455666f295e1ceb1cef6b92dc3ba138b90a8f3
    - SHA1:3cb42e61744f3cdb436f96e65d9669aa90623c88 [weak]
    - MD5Sum:4864e88c059a44fe020d60d1e2a80872 [weak]
   Release file created at: Wed, 17 Jun 2020 00:12:12 +0000

I'm taking a look at this (filing a ticket with the team, etc.) and will report back here with more info shortly.

(Edit: Filed https://icm.ad.msft.net/imp/v3/incidents/details/193848733/home tracking ticket.)

@dagood re: #4167, it has been significantly longer than that amount of "Expected time". Also, based on that issue, it appears the maintainers are not addressing the issue given the timeline, (nor contact-able). :(

I have the same:

[12:51:31]  [Step 3/5] Preparing to unpack .../apt-transport-https_1.8.2.1_all.deb ...
[12:51:31]  [Step 3/5] Unpacking apt-transport-https (1.8.2.1) ...
[12:51:31]  [Step 3/5] Setting up apt-transport-https (1.8.2.1) ...
[12:51:31]  [Step 3/5] Hit:1 http://security.debian.org/debian-security buster/updates InRelease
[12:51:31]  [Step 3/5] Hit:2 http://deb.debian.org/debian buster InRelease
[12:51:31]  [Step 3/5] Hit:3 http://deb.debian.org/debian buster-updates InRelease
[12:51:31]  [Step 3/5] Get:4 https://packages.microsoft.com/debian/9/prod stretch InRelease [4009 B]
[12:51:31]  [Step 3/5] Get:5 https://packages.microsoft.com/debian/9/prod stretch/main amd64 Packages [93.2 kB]
[12:51:31]  [Step 3/5] Err:5 https://packages.microsoft.com/debian/9/prod stretch/main amd64 Packages
[12:51:31]  [Step 3/5]   File has unexpected size (93446 != 93230). Mirror sync in progress? [IP: 13.66.3.153 443]
[12:51:31]  [Step 3/5]   Hashes of expected file:
[12:51:31]  [Step 3/5]    - Filesize:93230 [weak]
[12:51:31]  [Step 3/5]    - SHA512:6d9952cd289566e58aad057b0c24d1d450c2dc8adfd3871b06f843c6a8ea84853f6cda10e390ffddc9621299d2dcfe516e488a6fea215928d5446a52754dfa9a
[12:51:31]  [Step 3/5]    - SHA256:c5c7ad271c940d9533231e61043d923d93f5bde7ab9927d8da9be3964c440bd4
[12:51:31]  [Step 3/5]    - SHA1:46696f97ceccbacdcdd9dda6d68fdbc1f7b76cd0 [weak]
[12:51:31]  [Step 3/5]    - MD5Sum:49e53481a1cbdac343dd7d54cc3c2c52 [weak]
[12:51:31]  [Step 3/5]   Release file created at: Wed, 17 Jun 2020 00:12:39 +0000
[12:51:31]  [Step 3/5] Fetched 4009 B in 1s (6488 B/s)
[12:51:32]  [Step 3/5] Reading package lists...
[12:51:32]  [Step 3/5] E: Failed to fetch https://packages.microsoft.com/debian/9/prod/dists/stretch/main/binary-amd64/Packages.bz2  File has unexpected size (93446 != 93230). Mirror sync in progress? [IP: 13.66.3.153 443]
[12:51:32]  [Step 3/5]    Hashes of expected file:
[12:51:32]  [Step 3/5]     - Filesize:93230 [weak]
[12:51:32]  [Step 3/5]     - SHA512:6d9952cd289566e58aad057b0c24d1d450c2dc8adfd3871b06f843c6a8ea84853f6cda10e390ffddc9621299d2dcfe516e488a6fea215928d5446a52754dfa9a
[12:51:32]  [Step 3/5]     - SHA256:c5c7ad271c940d9533231e61043d923d93f5bde7ab9927d8da9be3964c440bd4
[12:51:32]  [Step 3/5]     - SHA1:46696f97ceccbacdcdd9dda6d68fdbc1f7b76cd0 [weak]
[12:51:32]  [Step 3/5]     - MD5Sum:49e53481a1cbdac343dd7d54cc3c2c52 [weak]
[12:51:32]  [Step 3/5]    Release file created at: Wed, 17 Jun 2020 00:12:39 +0000
[12:51:32]  [Step 3/5] E: Some index files failed to download. They have been ignored, or old ones used instead.
[12:51:32]  [Step 3/5] The command '/bin/sh -c apt-get install -y apt-transport-https && apt-get update && apt-get install -y aspnetcore-runtime-3.1 dotnet-runtime-3.1 cron' returned a non-zero code: 100
[12:51:32]  [Step 3/5] 
[12:51:32]  [Step 3/5] Process exited with code 100
[12:51:32]  [Step 3/5] Process exited with code 100 (Step: Docker build (Docker))
[12:51:32]  [Step 3/5] Step Docker build (Docker) failed

same problem!

```
root@9075ffe6ea60:/# apt-get update
Hit:1 http://archive.ubuntu.com/ubuntu eoan InRelease
Hit:2 http://archive.ubuntu.com/ubuntu eoan-updates InRelease
Hit:3 http://archive.ubuntu.com/ubuntu eoan-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu eoan-security InRelease
Get:5 https://packages.microsoft.com/ubuntu/19.10/prod eoan InRelease [10.5 kB]
Get:6 https://packages.microsoft.com/ubuntu/19.10/prod eoan/main armhf Packages [504 B]
Get:7 https://packages.microsoft.com/ubuntu/19.10/prod eoan/main arm64 Packages [762 B]
Get:8 https://packages.microsoft.com/ubuntu/19.10/prod eoan/main amd64 Packages [27.7 kB]
Err:8 https://packages.microsoft.com/ubuntu/19.10/prod eoan/main amd64 Packages
File has unexpected size (27870 != 27700). Mirror sync in progress? [IP: 104.214.230.139 443]
Hashes of expected file:

  • Filesize:27700 [weak]
  • SHA512:9b70e7c196b4aa1c30d66c9f3557217c32bff6ac6d330d9ed31cbb2993eb1146fd8820fe806668279d6f5cd6ee820b3a94494b67c122aa7d14c1291d18a85bbf
  • SHA256:7ff431b24135808bcb270b508ca9e81ef4b42b2c8fb24d9e7468aedbdd22449d
  • SHA1:4b94fc1de31196082b326cb61465a0c2c694a678 [weak]
  • MD5Sum:3f192fc18261e734bf26080c713ac5a0 [weak]
    Release file created at: Wed, 17 Jun 2020 00:13:22 +0000
    Fetched 11.7 kB in 1s (22.2 kB/s)
    Reading package lists... Done
    E: Failed to fetch https://packages.microsoft.com/ubuntu/19.10/prod/dists/eoan/main/binary-amd64/Packages.gz File has unexpected size (27870 != 27700). Mirror sync in progress? [IP: 104.214.230.139 443]
    Hashes of expected file:

    • Filesize:27700 [weak]

    • SHA512:9b70e7c196b4aa1c30d66c9f3557217c32bff6ac6d330d9ed31cbb2993eb1146fd8820fe806668279d6f5cd6ee820b3a94494b67c122aa7d14c1291d18a85bbf

    • SHA256:7ff431b24135808bcb270b508ca9e81ef4b42b2c8fb24d9e7468aedbdd22449d

    • SHA1:4b94fc1de31196082b326cb61465a0c2c694a678 [weak]

    • MD5Sum:3f192fc18261e734bf26080c713ac5a0 [weak]

      Release file created at: Wed, 17 Jun 2020 00:13:22 +0000

      E: Some index files failed to download. They have been ignored, or old ones used instead.

      ````

@dagood , I know this team is not responsible for the msodbcsql17 package, but it is also impacting that package. Like others have said, the packages.microsoft.com ubuntu repo is hosed for 16.04 across the board.

I've opened up a support ticket with azure as it is impacting our production workloads on databricks

re: #4167, it has been significantly longer than that amount of "Expected time".

Yes, that "expected time" is just an advisory that the service might get healthy again in some situations. This is clearly an exceptional case.

On my end I've filed https://icm.ad.msft.net/imp/v3/incidents/details/193848733/home with the team. (Internal link, posting here for our own easy access in future.)

Same issue here:

apt-get update
Hit:1 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu xenial InRelease
Hit:2 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:3 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu xenial-backports InRelease
Hit:4 http://ppa.launchpad.net/ansible/ansible/ubuntu xenial InRelease
Get:5 https://packages.microsoft.com/ubuntu/16.04/prod xenial InRelease [4,003 B]
Hit:6 http://security.ubuntu.com/ubuntu xenial-security InRelease
Get:7 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 Packages [142 kB]
Ign:7 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 Packages
Get:7 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 Packages [158 kB]
Ign:7 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 Packages
Get:7 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 Packages [791 kB]
Err:7 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 Packages
  Writing more data than expected (792466 > 791391)
Fetched 4,003 B in 0s (7,632 B/s)
Reading package lists... Done
E: Failed to fetch https://packages.microsoft.com/ubuntu/16.04/prod/dists/xenial/main/binary-amd64/Packages  Writing more data than expected (792466 > 791391)
E: Some index files failed to download. They have been ignored, or old ones used instead.

Is this related to unable to install netcore 2.2 on ubuntu 16.04, when doing apt-get install dotnet-sdk-2.2:

E: Unable to locate package dotnet-sdk-2.2
E: Couldn't find any package by glob 'dotnet-sdk-2.2'
E: Couldn't find any package by regex 'dotnet-sdk-2.2'

Is this related to unable to install netcore 2.2 on ubuntu 16.04, when doing apt-get install dotnet-sdk-2.2:

Yes, I expect your apt update failed earlier, and this caused the package metadata about dotnet-sdk-2.2 to not be downloaded. Ubuntu 16.04 is one of the distros where this problem is occurring. (I feel obliged to point out that 2.2 is EOL: you should try to upgrade to 3.1, or use 2.1 if the downgrade doesn't break you and it's more feasible.)


So far these repos have been reported to be affected (all Microsoft packages unavailable), and I've repro'd a few of them to confirm:

  • Ubuntu 16.04
  • Ubuntu 19.10
  • Debian 9
  • Debian 10

The Linux package repository team is working on applying a fix, no ETA yet.

(I feel obliged to point out that 2.2 is EOL: you should try to upgrade to 3.1, or use 2.1 if the downgrade doesn't break you and it's more feasible.)

Upgrade to 3.1 is in full progress, but until it's done, 2.2 needs to work :-)

Upgrade to 3.1 is in full progress, but until it's done, 2.2 needs to work :-)

Cool. 馃檪 And yeah, it's definitely our intention to keep EOL versions available on download endpoints and fix up issues like this as soon as we can.

@dagood what about .net core 3.1? i can't install it. So it's not about 2.2

what about .net core 3.1? i can't install it. So it's not about 2.2

You're correct, this outage applies to the entire Microsoft Linux package repository (every package) for each affected distro. The fix that the Linux package repo team is applying should restore access to all packages on these feeds, once they do it.

The reason for the side conversation is that 2.2 is EOL, and I wanted to make it clear that we will fix up both the EOL and non-EOL packages.

Hi there. Same here:

*****@**********:/# apt-get update
Hit:1 http://security.debian.org stretch/updates InRelease
Ign:2 http://deb.debian.org/debian stretch InRelease
Hit:3 http://deb.debian.org/debian stretch-updates InRelease
Hit:4 http://deb.debian.org/debian stretch Release
Get:7 https://packages.microsoft.com/debian/9/prod stretch/main amd64 Packages [103 kB]
Err:7 https://packages.microsoft.com/debian/9/prod stretch/main amd64 Packages
  Writing more data than expected (103561 > 103377)
  Hashes of expected file:
   - Filesize:103377 [weak]
   - SHA512:fa3d4bc9b2634e2c25be0c0afa362c32933f1df199213972bf7b45676afd852da61f6efc8b1ca2dd8617ec9d672425d354e528d4baf599e7ae621649c1e0ce9e
   - SHA256:9fe010504f92e67c9fb22d37e6af9e02f72fff3de54edc28f5e726ece0d2a7d6
   - SHA1:40ebf0eb5d1fb7bd2916d2cc2b8b334309c6daf9 [weak]
   - MD5Sum:ee0f5824b41c62a11fb2dda476171a7e [weak]
  Release file created at: Wed, 17 Jun 2020 00:12:39 +0000
Reading package lists... Done
E: Failed to fetch https://packages.microsoft.com/debian/9/prod/dists/stretch/main/binary-amd64/Packages.gz  Writing more data than expected (103561 > 103377)
   Hashes of expected file:
    - Filesize:103377 [weak]
    - SHA512:fa3d4bc9b2634e2c25be0c0afa362c32933f1df199213972bf7b45676afd852da61f6efc8b1ca2dd8617ec9d672425d354e528d4baf599e7ae621649c1e0ce9e
    - SHA256:9fe010504f92e67c9fb22d37e6af9e02f72fff3de54edc28f5e726ece0d2a7d6
    - SHA1:40ebf0eb5d1fb7bd2916d2cc2b8b334309c6daf9 [weak]
    - MD5Sum:ee0f5824b41c62a11fb2dda476171a7e [weak]
   Release file created at: Wed, 17 Jun 2020 00:12:39 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.

Same issue here for debian 10.

Same here for both Debian 10 and 9

it works now for ubuntu 18.04

Looks like back up for 16 as well. Closing.

Just tried debian 10 and still not working

NVM, working now

Fixed in debian 9 from now.

Fixed on Ubuntu 16.04

Thanks all for checking and posting status. The repo admins haven't gotten back to us since their last update, but @rbhanda applied a fix that worked for us in the past that seems to have cleared things up.

Fixed on Debian 10

The repo admins found some more repositories that needed fixups (Ubuntu 18.10, 17.10, 19.04), and they've resolved those and closed our ticket--all repositories should be working at this point.

@dagood thank you for facilitating with the internal team!

@dagood thnx man!

@dagood the https://packages.microsoft.com/ubuntu/18.04/prod/dists/bionic/main/binary-amd64/Packages.bz2 File has unexpected size happens again in ubuntu 18.04 when apt-get update:

E: Failed to fetch https://packages.microsoft.com/ubuntu/18.04/prod/dists/bionic/main/binary-amd64/Packages.bz2 File has unexpected size (141364 != 141252). Mirror sync in progress? [IP: 13.90.56.68 443]
Hashes of expected file:
- Filesize:141252 [weak]
- SHA512:dffbaf15f537ae643df110b279248993b2828e99352dff51a5df9648cf39531ffb7022ec2f0d9e553287c9889addcb5e734d9df95f749c456db04fd328a67ef4
- SHA256:6757f7f09fb2a0491657aac899faa29bd67fe7b9dfd849844f7fafca9afcc33e
- SHA1:37be76673db70e808db91294af54fddf476aae7f [weak]
- MD5Sum:ecc0d5d8d9373eee7aeceda58d7be8f9 [weak]
Release file created at: Tue, 13 Oct 2020 22:35:00 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.

Thanks for reporting, I will follow up on this and keep you posted. Is this an issue with both .NET Core 2.1 and 3.1?

It looks like the new outage was also reported at https://github.com/dotnet/core/issues/5373. It affects the entire 18.04 feed, not specific .NET packages.

I'm going to close this issue as resolved: https://github.com/dotnet/core/issues/5373 tracks the new 18.04 outage, so check there for the latest info.

Was this page helpful?
0 / 5 - 0 ratings