Msys2-packages: Update package database and core system packages: Cannot open package file

[ZuBsPaCe]

Hi there. I'm new to MSYS2 and would like to evaluate it to build some software on windows which needs to be built with gcc.

Unfortunately I get an error on a fresh MSYS2 installation when running pacman -Suy. Sorry, can't tell if I did something stupid.

zub@space MSYS ~
$ pacman -Syu
:: Synchronizing package databases...
 mingw32 is up to date
 mingw64 is up to date
 msys is up to date
:: Starting core system upgrade...
warning: terminate other MSYS2 programs before proceeding
resolving dependencies...
looking for conflicting packages...

Packages (8) bash-4.4.023-2  filesystem-2020.02-2  libzstd-1.4.4-2  mintty-1~3.1.4-1  msys2-runtime-3.1.4-1  pacman-5.2.1-6  pacman-mirrors-20200329-1  zstd-1.4.4-2

Total Installed Size:  65.76 MiB
Net Upgrade Size:      -3.96 MiB

:: Proceed with installation? [Y/n] Y
(8/8) checking keys in keyring                                                                                [#################################################################] 100%
(8/8) checking package integrity                                                                              [#################################################################] 100%
(8/8) loading package files                                                                                   [#################################################################] 100%
error: could not open file /var/cache/pacman/pkg/msys2-runtime-3.1.4-1-x86_64.pkg.tar.zst: Child process exited with status 127
error: failed to commit transaction (cannot open package file)
Errors occurred, no packages were upgraded.

zub@space MSYS ~
$ ps
      PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
     2172    2171    2172      18456  pty0     1049767 17:09:46 /usr/bin/bash
     2171       1    2171      18704  ?        1049767 17:09:46 /usr/bin/mintty
     1729       1    1729      27052  ?        1049767 17:00:47 /usr/bin/gpg-agent
     2285    2172    2285      31244  pty0     1049767 17:10:25 /usr/bin/ps

[KittyMcMitty]

Came here to find info about this issue. Happened on two machines, one of which is a clean Windows VM

[lazka]

Sorry for the inconvenience :/ There is some breakage today with the upgrade path from the last stable installer.

Please try the nightly installer: https://github.com/msys2/msys2-installer/releases/download/nightly-x86_64/msys2-x86_64-latest.exe

[loganmc10]

This issue is also affecting AppVeyor builds

[KittyMcMitty]

I seem to be able to install and get setup with that new installer. Thanks!

[mati865]

You can fix this issue by upgrading pacman first:

pacman -Sy pacman
pacman -Su

[ZuBsPaCe]

Thank you @mati865 . You're workaround worked, but there's a quirk. After updating pacman I called _pacman -Su_. During the installation, I got the prompt to close down the terminal window, restart msys2.exe and run the command again. The thing is, that msys2.exe did not start again. I tried multiple times and the process just quit right after.

It worked after a reboot, though. I could open msys2.exe successfully and the _packman -Su_ ran through.

[mati865]

@ZuBsPaCe it should be printed in the terminal that you should close all remaining MSYS2 processes (or simply reboot) after the update before opening the shell again.

[ZuBsPaCe]

@mati865 Just for reference. The exact wording is:

warning: terminate MSYS2 without returning to shell and check for updates again
warning: for example close your terminal window instead of calling exit

I tried again, just to be sure. I close the terminal and mintty.exe shuts down. msys2.exe cannot be started thereafter. I see gpg-agent.exe and dimngr.exe hanging around, but that's all. Nowhere does it say, that you need a reboot, but I'm happy, that I figured that one out.

Thanks for your help. 👍

Edit: Found this in the troubleshooting section of the documentation:
Hunt down these processes in your favorite task manager and kill them, or just reboot your system. I better read the docs next time. 😅

[eine]

Sorry for the inconvenience :/ There is some breakage today with the upgrade path from the last stable installer.

Please try the nightly installer: msys2/msys2-installer@nightly-x86_64 msys2-x86_64-latest.exe (download)

Hi @lazka, I would like to put absolute no pressure, because this must be already stressful for you. However, I'm trying to setup MSYS2 on GitHub Actions, and there seems to be no solution since yesterday...

The context is setup-msys2, where the distributed release package is unpacked and (optionally) pacman -Syu --noconfirm is executed: https://github.com/eine/setup-msys2/blob/master/main.js#L24.

• It worked 5 days ago: https://github.com/eine/setup-msys2/actions/runs/104936430
• With 20200517, it works as long as pacman -Syu is not used: https://github.com/eine/setup-msys2/actions/runs/109156856
  
  
  
  • Otherwise, an error is produced: 0 [main] bash (7000) shared_info::initialize: size of shared memory region changed from 51128 to 40888.
  
  
• The "nightly" release produces the same result as 20200517. I think this is because, ATM, both are the same artifact: https://github.com/eine/setup-msys2/actions/runs/109159120

@k-takata (from #1961), note that I cannot reboot the system because this is being executed on a GitHub Actions workflow. As shown in the logs, I print tasklist after setup/update and before using it again. Unfortunately, I cannot see any process which belongs to MSYS2: https://github.com/eine/setup-msys2/runs/688612762?check_suite_focus=true#step:5:7. Killing gpg-agent explicitly was already tested by @Kha in https://github.com/leanprover/lean4/pull/141.

Ref #1960

[mati865]

@eine as said above you can upgrade pacman first (assuming you installed with latest stable installed from 2019-05-24), then normal upgrade will work.

[eine]

@mati865, I would say I have already tried that with no success. However, I'm going to try it again.

[lazka]

@eine yeah, that needs fixing somehow. dirmngr.exe is also part of gnupg

@mati865 I think this is the "as long as any pre upgrade processes are still running nothing works" case, unrelated to pacman

[eine]

FTR, upgrading pacman first produces the same result, either with 20190524 or 20200517. See https://github.com/eine/setup-msys2/actions/runs/109176396 and https://github.com/eine/setup-msys2/runs/688664492.

@lazka, you rock! https://github.com/eine/setup-msys2/runs/688691639?check_suite_focus=true Of course, it'd be good to release a newer nightly (or hopefully "stable" release) which does not require this. Or to automatically handle it otherwise. However, it works for now! Thanks a lot! Is there any list of the tasks we should search for?

[SolarBear]

Hello! I just stumbled upon this problem on 2 machines.

1. Machine #1 simply exits with error code 127 whatever the command, whether it's pacman or ls or....
2. Machine #2 simply won't open msys2 anymore. I've tried opening it from a terminal and the command silently fails. Strangely enough, echo %ERRORLEVEL% returns 0.

[mati865]

Machine #1 simply exits with error code 127 whatever the command, whether it's pacman or ls or....

You might be missing some dll, maybe /usr/bin/msys-2.0.dll?

Machine #2 simply won't open msys2 anymore. I've tried opening it from a terminal and the command silently fails. Strangely enough, echo %ERRORLEVEL% returns 0.

You have running processes that have handle on msys-2.0.dll from before the update. Either kill them or reboot the system.

[SolarBear]

You might be missing some dll, maybe /usr/bin/msys-2.0.dll?

I compared the list of files in the directory with a working install: no missing files.

You have running processes that have handle on msys-2.0.dll from before the update. Either kill them or reboot the system.

Strange, but that was it! Thank you.

[loganmc10]

The workaround of updating pacman first seems to have stopped working today

bash -c "pacman -Sy --noconfirm pacman"
:: Synchronizing package databases...
downloading mingw32.db...
downloading mingw32.db.sig...
downloading mingw64.db...
downloading mingw64.db.sig...
downloading msys.db...
downloading msys.db.sig...
resolving dependencies...
looking for conflicting packages...
Packages (2) zstd-1.4.4-2  pacman-5.2.1-7
Total Download Size:   14.08 MiB
Total Installed Size:  41.57 MiB
Net Upgrade Size:      -5.34 MiB
:: Proceed with installation? [Y/n] 
:: Retrieving packages...
downloading zstd-1.4.4-2-x86_64.pkg.tar.xz...
downloading pacman-5.2.1-7-x86_64.pkg.tar.zst...
checking keyring...
checking package integrity...
loading package files...
error: could not open file /var/cache/pacman/pkg/pacman-5.2.1-7-x86_64.pkg.tar.zst: Child process exited with status 127
error: failed to commit transaction (cannot open package file)
Errors occurred, no packages were upgraded.
Command exited with code 1

[loganmc10]

If I try to install zstd first, then update pacman, I get:

upgrading pacman...
      0 [main] pacman 203 dofork: child -1 - CreateProcessW failed for 'C:\msys64\usr\bin\pacman.exe', errno 2
error: could not open file /var/cache/pacman/pkg/pacman-5.2.1-7-x86_64.pkg.tar.zst: Can't initialize filter; unable to run program "zstd -d -qq"
error: could not commit transaction
error: failed to commit transaction (transaction aborted)
Errors occurred, no packages were upgraded.
Command exited with code 1

[eine]

@loganmc10, see https://github.com/msys2/MSYS2-packages/issues/1965#issuecomment-631567674.

[loganmc10]

@eine I'm not sure what you are trying to allude to, what would I do to fix my problem?

[eine]

with latest base (2020-05-17) you don't have to upgrade pacman separately.

If upgraded pacman depends on newer msys2-runtime than installed one you will end up with broken pacman. The same could be said about bash.

So, what is your recommendation?

pacman --noconfirm -Syuu
taskkill /IM gpg-agent.exe /F
taskkill /IM dirmngr.exe /F
pacman --noconfirm -Suu

Should be good for now but keep in mind that situation with leftover processes can change in the future.

[emptyVoid]

@eine, that workaround doesn't work too:

MSYS ~
$ pacman --noconfirm -Syuu
:: Synchronizing package databases...
 mingw32                  462.4 KiB   430K/s 00:01 [#####################] 100%
 mingw32.sig              119.0   B  0.00B/s 00:00 [#####################] 100%
 mingw64                  465.0 KiB  2.32M/s 00:00 [#####################] 100%
 mingw64.sig              119.0   B  0.00B/s 00:00 [#####################] 100%
 msys                     183.3 KiB  10.5M/s 00:00 [#####################] 100%
 msys.sig                 119.0   B  0.00B/s 00:00 [#####################] 100%
:: Starting core system upgrade...
warning: terminate other MSYS2 programs before proceeding
resolving dependencies...
looking for conflicting packages...

Packages (8) bash-4.4.023-2  filesystem-2020.02-2  libzstd-1.4.4-2
             mintty-1~3.1.4-1  msys2-runtime-3.1.4-2  pacman-5.2.1-7
             pacman-mirrors-20200329-1  zstd-1.4.4-2

Total Download Size:   19.95 MiB
Total Installed Size:  66.40 MiB
Net Upgrade Size:      -3.32 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages...
 msys2-runtime-3.1.4...     3.0 MiB  2.94M/s 00:01 [#####################] 100%
 bash-4.4.023-2-x86_64   1934.3 KiB  2.30M/s 00:01 [#####################] 100%
 filesystem-2020.02-...    44.9 KiB  11.0M/s 00:00 [#####################] 100%
 mintty-1~3.1.4-1-x86_64  668.9 KiB  2.39M/s 00:00 [#####################] 100%
 pacman-mirrors-2020...    20.3 KiB  1689K/s 00:00 [#####################] 100%
 libzstd-1.4.4-2-x86_64   259.0 KiB  9.03M/s 00:00 [#####################] 100%
 zstd-1.4.4-2-x86_64      304.7 KiB  8.50M/s 00:00 [#####################] 100%
 pacman-5.2.1-7-x86_64     13.8 MiB  2.45M/s 00:06 [#####################] 100%
(8/8) checking keys in keyring                     [#####################] 100%
(8/8) checking package integrity                   [#####################] 100%
(8/8) loading package files                        [#####################] 100%
error: could not open file /var/cache/pacman/pkg/msys2-runtime-3.1.4-2-x86_64.pkg.tar.zst: Child process exited with status 127
error: could not open file /var/cache/pacman/pkg/pacman-5.2.1-7-x86_64.pkg.tar.zst: Child process exited with status 127
error: failed to commit transaction (cannot open package file)
Errors occurred, no packages were upgraded.

[goalkeeper007]

@emptyVoid this might help you work around https://github.com/msys2/MSYS2-packages/issues/1960#issuecomment-631917722

[mati865]

@emptyVoid you have to use latest installer dated 2020-05-17 for it to work.

[emptyVoid]

I see no such installer on the public repositories (https://sourceforge.net/projects/msys2/files/Base/x86_64/ and http://repo.msys2.org/distrib/x86_64/).

[mati865]

@emptyVoid you can get the link from MSYS2 website: https://www.msys2.org/ it points to https://github.com/msys2/msys2-installer/releases/

[emptyVoid]

Thank you!

@mati865, do you know by chance if MSYS2 GitHub releases are persistent? Since I don't see any besides the latest.

[mati865]

@lazka ^

[lazka]

We'll continue hosting them on github and also try to mirror them to the repo (for users with limited access to github)

[luxalpa]

So am I seeing this right that it's currently not possible to install this software programmatically / unattended?

[jtanx]

No it's definitely possible, this is what I do for appveyor

Here's another alternate way to clean up all the zombie processes

taskkill /f /fi "MODULES eq msys-2.0.dll"

[emptyVoid]

Or you can kill them "natively":

/usr/bin/gpgconf --homedir /etc/pacman.d/gnupg --kill all

[lazka]

Here's another alternate way to clean up all the zombie processes

taskkill /f /fi "MODULES eq msys-2.0.dll"

This seems like a good solution to me. What about we run this at the end of every core update? (with confirmation in the tty case)

[emptyVoid]

I wouldn't recommend killing GnuPG daemons with taskkill since it does not discriminate by image (executable/library) path and will mess with unrelated MSYS2 installations (e.g. https://github.com/microsoft/vcpkg/issues/9427).

[eine]

@jtanx, thanks a lot for that so valuable hint! I will test it in https://github.com/eine/setup-msys2/blob/master/main.js#L62-L63

This seems like a good solution to me. What about we run this at the end of every core update? (with confirmation in the tty case)

@lazka, I'd be willing to help test such an approach by contributing a GitHub Actions workflow. However, given the volume of repos and packages, I'd need some guidance.

[lazka]

@emptyVoid what about something like echo "taskkill /f$(for pid in $(cat /proc/*/winpid); do echo -n " /pid $pid"; done)" -> taskkill /f /pid 4084 /pid 5532 /pid 9272 /pid 9000 /pid 8476

[emptyVoid]

Looks good, going to try it out.
I guess you are planning to filter out the terminal's process, otherwise it would be a bit confusing for the user if gets suddenly closed.

[lazka]

I guess you are planning to filter out the terminal's process, otherwise it would be a bit confusing for the user if gets suddenly closed.

I was thinking about something like

:: To complete the update all MSYS2 processes including this terminal need to be closed. Confirm to proceed [Y/n]

[emptyVoid]

What about scripted installations -- will MSYS2 processes be auto-closed with --noconfirm?

[lazka]

What about scripted installations -- will MSYS2 processes be auto-closed with --noconfirm?

sure. I still need to figure out how to make those cases not report a non-error exit status. Maybe skip pacman itself and the parent process if it is a bash and we are noconfirm.

edit: SIGTERM and finally taskkill seems to do the trick
edit2: somehow not needed if I skip the calling process, shrug

[lazka]

core packges are .xz again now and pacman will autokill all msys packages on a runtime update.

I'll release a new installer later today which includes the new pacman version.

[gwilku]

Still not solved...