Winget-cli: The latest DockerDesktopEdge 2.3.2.1 does not show up

[StefanScherer]

Brief description of your issue

A winget search docker still shows a previous version of Docker Desktop Edge, although a PR to update the package version is already merged.

Steps to reproduce

$ winget search docker

Expected behavior

$ winget search docker
Name                Id                       Version       Matched
--------------------------------------------------------------------------
Docker Desktop      Docker.DockerDesktop     2.3.0.3.45519 Moniker: docker
Docker Desktop Edge Docker.DockerDesktopEdge 2.3.2.1.46329

Actual behavior

$ winget search docker
Name                Id                       Version       Matched
--------------------------------------------------------------------------
Docker Desktop      Docker.DockerDesktop     2.3.0.3.45519 Moniker: docker
Docker Desktop Edge Docker.DockerDesktopEdge 2.3.2.0.46268

It still shows DockerDesktopEdge 2.3.2.0 instead of 2.3.2.1 🤔

The PR for 2.3.2.0 11 days ago: https://github.com/microsoft/winget-pkgs/pull/2121
The PR for 2.3.2.1 5 days ago: https://github.com/microsoft/winget-pkgs/pull/2173

Environment

$ winget --info
Windows Package Manager v0.1.41331 Vorschau
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.19041.329
Package: Microsoft.DesktopAppInstaller v1.0.41331.0

Links:
  Privacy Statement: https://aka.ms/winget-privacy
  License agreement: https://aka.ms/winget-license
  3rd Party Notices: https://aka.ms/winget-3rdPartyNotice
  Homepage:          https://aka.ms/winget

[OfficialEsco]

No merges after 1th July is showing up.

[gorup]

This and microsoft/winget-pkgs#2274 seem like duplicates, and the root question is: How often can we expect the winget 'source' to get updated with merged-in packages once a user does winget source update? Understandably this product has not launched 1.0 yet, so

• What can we expect in terms of time-to-show-up from now until 1.0 launches?
• Is there a planned target time-to-show-up for 1.0?

Thanks!

[OfficialEsco]

From my own PR experience you don’t have to do winget source update (or it does it in the background very often).

New merges should show up after a few hours max.

Something is clearly broken because its been working like it should until 1th july.

[denelon]

We have been working through some issues. Another process is started when a new package (or version of an existing package) is merged into the main branch. That process has some additional logic related to how packages are displayed, and search results (mostly related to letter casing). Generally speaking we've seen the process take a few minutes to complete after a PR has been merged on GitHub. Sometimes it takes a bit longer. We've also been instrumenting the infrastructure to notify us when something is not behaving as expected.

[denelon]

The default configuration for the client is a five minute TTL (Time To Live) for the local index that is pulled down to the client machine. The bigger issue is how often the index package is being refreshed. Right now that's not on a timer. It's driven by manifests when they are added, modified, or removed.

[gorup]

When you say the index package, do you mean some central file that winget source update retrieves (from https://winget.azureedge.net/cache ?) and caches locally? And that is having issues being refreshed? That is my guess.

The winget source only retrieves Data: Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe after a remove/reset/update cycle

PS C:\Users\Ryan> winget source list winget
Name   : winget
Arg    : https://winget.azureedge.net/cache
Data   : Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe
Updated: 2020-7-7 07:34:28.000
PS C:\Users\Ryan> winget source remove winget
Removing source: winget...
  ██████████████████████████████  100%
Done.
PS C:\Users\Ryan> winget source list winget
Did not find a source named: winget
PS C:\Users\Ryan> winget source update
Updating all sources...
PS C:\Users\Ryan> winget source list winget
Did not find a source named: winget
PS C:\Users\Ryan> winget source reset
Reseting all sources ... Done
PS C:\Users\Ryan> winget source list winget
Name   : winget
Type   : Microsoft.PreIndexed.Package
Arg    : https://winget.azureedge.net/cache
Data   :
Updated: <never>
PS C:\Users\Ryan> winget source update
Updating all sources...
Updating source: winget...
  ██████████████████████████████  100%
Done.
PS C:\Users\Ryan> winget source list winget
Name   : winget
Type   : Microsoft.PreIndexed.Package
Arg    : https://winget.azureedge.net/cache
Data   : Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe
Updated: 2020-7-7 09:48:36.000
PS C:\Users\Ryan> winget search ebbflow
No app found matching input criteria.
PS C:\Users\Ryan>
PS C:\Users\Ryan> winget --info
Windows Package Manager v0.1.41331 Preview
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.19041.329
Package: Microsoft.DesktopAppInstaller v1.0.41331.0

Links:
  Privacy Statement: https://aka.ms/winget-privacy
  License agreement: https://aka.ms/winget-license
  3rd Party Notices: https://aka.ms/winget-3rdPartyNotice
  Homepage:          https://aka.ms/winget
PS C:\Users\Ryan> winget -v
v0.1.41331 Preview

[JohnMcPMS]

The CDN was stuck on an older version of the file and wasn't updating. We have some ideas to investigate as to what is causing this, but for now I have purged the CDN and it appears to be up to date now:

PackageFullName   : Microsoft.Winget.Source_2020.707.556.192_neutral__8wekyb3d8bbwe

> winget search docker
Name                Id                       Version       Matched
--------------------------------------------------------------------------
Docker Desktop      Docker.DockerDesktop     2.3.0.3.45519 Moniker: docker
Docker Desktop Edge Docker.DockerDesktopEdge 2.3.2.1.46329

In addition, we obviously need to put in a watchdog to ensure that the CDN is actually taking updates.

[megamorf]

Sounds like a check that compares the sqlite index from the CDN with the index from the origin would make sense to determine if cache is up-to-date.

[gorup]

Ok thank you. I now see Microsoft.Winget.Source_2020.707.556.192_neutral__8wekyb3d8bbwe, but the packages still don't show as expected (I'm getting the old dockerdesktopedge 46268).

And when I try to complete the same steps as earlier, of removing / resetting / updating, I now get an error

PS C:\Users\Ryan> winget source list winget
Name   : winget
Type   : Microsoft.PreIndexed.Package
Arg    : https://winget.azureedge.net/cache
Data   : Microsoft.Winget.Source_2020.707.556.192_neutral__8wekyb3d8bbwe
Updated: 2020-7-7 13:48:47.000
PS C:\Users\Ryan> winget search docker
Name                Id                       Version       Matched
--------------------------------------------------------------------------
Docker Desktop      Docker.DockerDesktop     2.3.0.3.45519 Moniker: docker
Docker Desktop Edge Docker.DockerDesktopEdge 2.3.2.0.46268
PS C:\Users\Ryan> winget source update winget
Updating source: winget...
  ██████████████████████████████  100%
Done.
PS C:\Users\Ryan> winget source list winget
Name   : winget
Type   : Microsoft.PreIndexed.Package
Arg    : https://winget.azureedge.net/cache
Data   : Microsoft.Winget.Source_2020.707.556.192_neutral__8wekyb3d8bbwe
Updated: 2020-7-7 13:49:29.000
PS C:\Users\Ryan> winget search docker
Name                Id                       Version       Matched
--------------------------------------------------------------------------
Docker Desktop      Docker.DockerDesktop     2.3.0.3.45519 Moniker: docker
Docker Desktop Edge Docker.DockerDesktopEdge 2.3.2.0.46268
PS C:\Users\Ryan> winget source reset
The following sources will be reset:
Name   Arg
-----------------------------------------
winget https://winget.azureedge.net/cache
Do you wish to continue? (Y|N)
Y
Reseting all sources ... Done.
PS C:\Users\Ryan> winget search docker
Name                Id                       Version       Matched
--------------------------------------------------------------------------
Docker Desktop      Docker.DockerDesktop     2.3.0.3.45519 Moniker: docker
Docker Desktop Edge Docker.DockerDesktopEdge 2.3.2.0.46268
PS C:\Users\Ryan> winget source remove winget
Removing source: winget...
An unexpected error occurred while executing the command:
Install failed: Windows cannot remove Microsoft.Winget.Source_2020.707.556.192_neutral__8wekyb3d8bbwe because the current user does not have that package installed. Use Get-AppxPackage to see the list of packages installed.
0x80073cf1 : Package was not found.
PS C:\Users\Ryan>

[JohnMcPMS]

@gorup, can you post the output of Get-AppxPackage *winget*

I don't see how you could have a newer version with older data...

[gorup]

@JohnMcPMS Seemingly points to old somehow. As you can see in my earlier messages, I've been resetting-updating-removing the winget source in a million different ways in an effort to poke the cache, but it seems I've entered a rather weird state!

PS C:\Users\Ryan> Get-AppxPackage *winget*


Name              : Microsoft.Winget.Source
Publisher         : CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
Architecture      : Neutral
ResourceId        :
Version           : 2020.630.2349.618
PackageFullName   : Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe
InstallLocation   : C:\Program Files\WindowsApps\Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe
IsFramework       : False
PackageFamilyName : Microsoft.Winget.Source_8wekyb3d8bbwe
PublisherId       : 8wekyb3d8bbwe
IsResourcePackage : False
IsBundle          : False
IsDevelopmentMode : False
NonRemovable      : False
IsPartiallyStaged : False
SignatureKind     : Store
Status            : Ok



PS C:\Users\Ryan> winget search ebbflow
No app found matching input criteria.
PS C:\Users\Ryan> winget source list winget
Name   : winget
Type   : Microsoft.PreIndexed.Package
Arg    : https://winget.azureedge.net/cache
Data   : Microsoft.Winget.Source_2020.707.556.192_neutral__8wekyb3d8bbwe
Updated: 2020-7-7 15:15:28.000
PS C:\Users\Ryan> winget search docker
Name                Id                       Version       Matched
--------------------------------------------------------------------------
Docker Desktop      Docker.DockerDesktop     2.3.0.3.45519 Moniker: docker
Docker Desktop Edge Docker.DockerDesktopEdge 2.3.2.0.46268

[JohnMcPMS]

Indeed, this is a strange state. Unless you've got the new package staged but not registered (APPX/MSIX for files on the disk but not installed for you), I don't know what else is going on. And if you are in that state I have no idea how you got there.

From admin Get-AppxPackage *winget* -AllUsers would show if the newer version is just sitting there waiting.

winget logs here %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\DiagOutputDir

The latest files there should give some indication as to what is happening (whether it is trying to update, which it should be). If you would like me to take a look at them instead, you can file a Feedback Hub issue in the category Apps > Windows Package Manager and it will gather up the latest of the logs. Then just post whatever link you get here.

If instead you just want to get it working again (which I can understand), I think manually removing the source package will probably help:

Get-AppxPackage *winget* | Remove-AppxPackage
winget source update
winget search docker

[gorup]

Thanks for the help. I don't see the new version for AllUsers. The old one still shows up.

I submitted a Feedback Hub thing: https://aka.ms/AA8xd78

And continue below to see that even after remove the old one sticks around. '630' is a zombie that cannot be killed

PS C:\Users\Ryan> Get-AppxPackage *winget* -AllUsers


Name                   : Microsoft.Winget.Source
Publisher              : CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
Architecture           : Neutral
ResourceId             :
Version                : 2020.630.2349.618
PackageFullName        : Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe
InstallLocation        : C:\Program Files\WindowsApps\Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe
IsFramework            : False
PackageFamilyName      : Microsoft.Winget.Source_8wekyb3d8bbwe
PublisherId            : 8wekyb3d8bbwe
PackageUserInformation : {S-1-5-21-4073537815-3022606501-667747716-1001 [Ryan]: Installed}
IsResourcePackage      : False
IsBundle               : False
IsDevelopmentMode      : False
NonRemovable           : False
IsPartiallyStaged      : False
SignatureKind          : Store
Status                 : Ok



PS C:\Users\Ryan>

Also, unfortunately, that did not help :(

Here are the commands I ran

PS C:\Users\Ryan> Get-AppxPackage *winget* | Remove-AppxPackage
PS C:\Users\Ryan> Get-AppxPackage *winget* -AllUsers
PS C:\Users\Ryan> winget search docker
Failed to open the source; try removing and re-adding it
An unexpected error occurred while executing the command:
0x8a15000f : Data required by the source is missing
PS C:\Users\Ryan> winget source update
Updating all sources...
Updating source: winget...
  ██████████████████████████████  100%
Done.
PS C:\Users\Ryan> winget search docker
Name                Id                       Version       Matched
--------------------------------------------------------------------------
Docker Desktop      Docker.DockerDesktop     2.3.0.3.45519 Moniker: docker
Docker Desktop Edge Docker.DockerDesktopEdge 2.3.2.0.46268
PS C:\Users\Ryan> Get-AppxPackage *winget* -AllUsers


Name                   : Microsoft.Winget.Source
Publisher              : CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
Architecture           : Neutral
ResourceId             :
Version                : 2020.630.2349.618
PackageFullName        : Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe
InstallLocation        : C:\Program Files\WindowsApps\Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe
IsFramework            : False
PackageFamilyName      : Microsoft.Winget.Source_8wekyb3d8bbwe
PublisherId            : 8wekyb3d8bbwe
PackageUserInformation : {S-1-5-21-4073537815-3022606501-667747716-1001 [Ryan]: Installed}
IsResourcePackage      : False
IsBundle               : False
IsDevelopmentMode      : False
NonRemovable           : False
IsPartiallyStaged      : False
SignatureKind          : Store
Status                 : Ok



PS C:\Users\Ryan>

And the three log files, for the three winget commands I ran (search, update, search) all concatenated:

2020-7-7 16:01:07.160 [CORE] WinGet, version [0.1.41331], activity [{BDFFACFE-230B-445B-876D-FA7661071CD1}]
2020-7-7 16:01:07.161 [CORE] OS: Windows.Desktop v10.0.19041.329
2020-7-7 16:01:07.161 [CORE] Package: Microsoft.DesktopAppInstaller v1.0.41331.0
2020-7-7 16:01:07.161 [CLI ] WinGet invoked with arguments: 'search' 'docker'
2020-7-7 16:01:07.161 [CLI ] Found subcommand: search
2020-7-7 16:01:07.161 [CLI ] Leaf command to execute: root:search
2020-7-7 16:01:07.171 [CLI ] Executing command: search
2020-7-7 16:01:07.174 [REPO] Default source requested, using first source: winget
2020-7-7 16:01:07.175 [REPO] Named source requested, found: winget
2020-7-7 16:01:07.179 [CORE] Did not find extension: PFN = Microsoft.Winget.Source_8wekyb3d8bbwe, ID = IndexDB
2020-7-7 16:01:07.179 [REPO] Package not found Microsoft.Winget.Source_2020.707.556.192_neutral__8wekyb3d8bbwe
2020-7-7 16:01:07.180 [FAIL] C:\BA\138\s\external\pkg\src\AppInstallerRepositoryCore\Microsoft\PreIndexedPackageSourceFactory.cpp(139)\AppInstallerCLI.exe!00007FF7B0B58E17: (caller: 00007FF7B0B57A51) Exception(1) tid(26c4) 8A15000F 
2020-7-7 16:01:07.276 [CLI ] Caught wil::ResultException: C:\BA\138\s\external\pkg\src\AppInstallerRepositoryCore\Microsoft\PreIndexedPackageSourceFactory.cpp(139)\AppInstallerCLI.exe!00007FF7B0B58E17: (caller: 00007FF7B0B57A51) Exception(1) tid(26c4) 8A15000F 

2020-7-7 16:01:10.859 [CORE] WinGet, version [0.1.41331], activity [{976F8C4F-BB7B-4E02-A6ED-DD71ECA284B7}]
2020-7-7 16:01:10.859 [CORE] OS: Windows.Desktop v10.0.19041.329
2020-7-7 16:01:10.859 [CORE] Package: Microsoft.DesktopAppInstaller v1.0.41331.0
2020-7-7 16:01:10.860 [CLI ] WinGet invoked with arguments: 'source' 'update'
2020-7-7 16:01:10.860 [CLI ] Found subcommand: source
2020-7-7 16:01:10.860 [CLI ] Found subcommand: update
2020-7-7 16:01:10.860 [CLI ] Leaf command to execute: root:source:update
2020-7-7 16:01:10.870 [CLI ] Executing command: update
2020-7-7 16:01:10.874 [REPO] Named source to be updated, found: winget
2020-7-7 16:01:10.879 [CORE] Did not find extension: PFN = Microsoft.Winget.Source_8wekyb3d8bbwe, ID = IndexDB
2020-7-7 16:01:10.892 [CORE] Downloading to path: "C:\\Users\\Ryan\\AppData\\Local\\Packages\\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\\TempState\\WinGet\\Microsoft.Winget.Source_2020.707.556.192_neutral__8wekyb3d8bbwe.msix"
2020-7-7 16:01:10.893 [CORE] Started applying motw to "C:\\Users\\Ryan\\AppData\\Local\\Packages\\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\\TempState\\WinGet\\Microsoft.Winget.Source_2020.707.556.192_neutral__8wekyb3d8bbwe.msix"
2020-7-7 16:01:10.894 [CORE] Finished applying motw
2020-7-7 16:01:10.894 [CORE] Downloading from url: https://winget.azureedge.net/cache/source.msix
2020-7-7 16:01:10.991 [CORE] Download completed.
2020-7-7 16:01:10.995 [CORE] Starting RequestAddPackage operation #0: file:///C:/Users/Ryan/AppData/Local/Packages/Microsoft.DesktopAppInstaller_8wekyb3d8bbwe/TempState/WinGet/Microsoft.Winget.Source_2020.707.556.192_neutral__8wekyb3d8bbwe.msix
2020-7-7 16:01:11.000 [CORE] Begin waiting for deployment #0
2020-7-7 16:01:11.000 [CORE] Begin blocking for deployment #0
2020-7-7 16:01:12.169 [CORE] Successfully deployed #0
2020-7-7 16:01:12.171 [CLI ] Leaf command succeeded: root:source:update


2020-7-7 16:01:14.212 [CORE] WinGet, version [0.1.41331], activity [{59DB0AF4-7B72-4E10-A32D-2FA2DDFA0638}]
2020-7-7 16:01:14.212 [CORE] OS: Windows.Desktop v10.0.19041.329
2020-7-7 16:01:14.212 [CORE] Package: Microsoft.DesktopAppInstaller v1.0.41331.0
2020-7-7 16:01:14.212 [CLI ] WinGet invoked with arguments: 'search' 'docker'
2020-7-7 16:01:14.213 [CLI ] Found subcommand: search
2020-7-7 16:01:14.213 [CLI ] Leaf command to execute: root:search
2020-7-7 16:01:14.222 [CLI ] Executing command: search
2020-7-7 16:01:14.226 [REPO] Default source requested, using first source: winget
2020-7-7 16:01:14.226 [REPO] Named source requested, found: winget
2020-7-7 16:01:14.237 [CORE] Examining extension: PFN = Microsoft.Winget.Source_8wekyb3d8bbwe, ID = IndexDB
2020-7-7 16:01:14.237 [CORE] Found matching extension.
2020-7-7 16:01:14.248 [REPO] Opening SQLite Index for ImmutableRead at 'C:\Program Files\WindowsApps\Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe\Public\index.db'
2020-7-7 16:01:14.248 [SQL ] Opening SQLite connection: 'file:/C:/Program Files/WindowsApps/Microsoft.Winget.Source_2020.630.2349.618_neutral__8wekyb3d8bbwe/Public/index.db?immutable=1' [1, 40]
2020-7-7 16:01:14.248 [REPO] Opened SQLite Index with version [1.0], last write [2020-6-30 15:48:55.000]
2020-7-7 16:01:14.328 [REPO] Performing search: Query:'docker'[Substring]
2020-7-7 16:01:14.349 [CLI ] Leaf command succeeded: root:search

[StefanScherer]

Version 2.3.2.1 is showing up now. Thanks gor fixing the underlying issue.