Vcpkg: [vcpkg_fixup_pkgconfig] build failure with "Could not find PKGCONFIG using the following names: pkg-config"

Created on 24 Jul 2020 · 15Comments · Source: microsoft/vcpkg

Host Environment

OS: Microsoft Windows [Version 10.0.19041.388]
Compiler:
- Visual Studio 2017 Version: 15.9.25 VS Tools Version: 14.16.27023
- Visual Studio 2019 Version: 16.6.5 VS Tools Version: 14.26.28801

To Reproduce
Steps to reproduce the behavior:
After latest update to vcpkg #11550 Commit https://github.com/microsoft/vcpkg/commit/458c20e94b9f558b523f187ce403bd062e131cdb

set VCPKG_DEFAULT_TRIPLET=x64-windows

./vcpkg upgrade --no-dry-run
./vcpkg install bzip2

Failure logs
- Cut and past the appropriate build messages from the console output.
1. vcpkg upgrade console log
The following packages will be rebuilt:
* bzip2[core]:x64-windows
Additional packages (*) will be modified to complete this operation.
Starting package 1/2: bzip2:x64-windows
Removing package bzip2:x64-windows...
Removing package bzip2:x64-windows... done
Elapsed time for package bzip2:x64-windows: 6.266 ms
-- Using cached D:/Git/vcpkg.downloads/bzip2-1.0.6.tar.gz
-- Cleaning sources at D:/Git/vs2019.test/buildtrees/bzip2/src/1.0.6-8838f21f8e.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source D:/Git/vcpkg.downloads/bzip2-1.0.6.tar.gz
-- Applying patch fix-import-export-macros.patch
-- Applying patch fix-windows-include.patch
-- Using source at D:/Git/vs2019.test/buildtrees/bzip2/src/1.0.6-8838f21f8e.clean
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Fixing pkgconfig
CMake Error at scripts/cmake/vcpkg_fixup_pkgconfig.cmake:279 (find_program):
Could not find PKGCONFIG using the following names: pkg-config
Call Stack (most recent call first):
ports/bzip2/portfile.cmake:51 (vcpkg_fixup_pkgconfig)
scripts/ports.cmake:79 (include)

Error: Building package bzip2:x64-windows failed with: BUILD_FAILED
Elapsed time for package bzip2:x64-windows: 2.98 s

Total elapsed time: 4.544 s

RESULTS
bzip2:x64-windows: vcpkg::Commands::Build::BuildResult_NULLVALUE: 6.266 ms
bzip2:x64-windows: BUILD_FAILED: 2.98 s

SUMMARY
vcpkg::Commands::Build::BuildResult_NULLVALUE: 1
SUCCEEDED: 0
BUILD_FAILED: 1
POST_BUILD_CHECKS_FAILED: 0
FILE_CONFLICTS: 0
CASCADED_DUE_TO_MISSING_DEPENDENCIES: 0
EXCLUDED: 0

vcpkg instal bzip2:x64-windows
Computing installation plan...
The following packages will be built and installed:
bzip2[core]:x64-windows
Detecting compiler hash for triplet x64-windows...
Starting package 1/1: bzip2:x64-windows
Building package bzip2[core]:x64-windows...
-- Using cached D:/Git/vcpkg.downloads/bzip2-1.0.6.tar.gz
-- Extracting source D:/Git/vcpkg.downloads/bzip2-1.0.6.tar.gz
-- Applying patch fix-import-export-macros.patch
-- Applying patch fix-windows-include.patch
-- Using source at D:/Git/vs2017.test/buildtrees/bzip2/src/1.0.6-8838f21f8e.clean
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Fixing pkgconfig
CMake Error at scripts/cmake/vcpkg_fixup_pkgconfig.cmake:279 (find_program):
Could not find PKGCONFIG using the following names: pkg-config
Call Stack (most recent call first):
ports/bzip2/portfile.cmake:51 (vcpkg_fixup_pkgconfig)
scripts/ports.cmake:79 (include)

Error: Building package bzip2:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with.\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: bzip2:x64-windows
Vcpkg version: 2020.06.15-nohash

Additionally, attach any relevant sections from the log files above.

Please attach any additional failure logs mentioned in the console output.
No failure in logs, appears to compile/link properly, just fails to install due to error listed.

Additional context

tool-update

Source

StarGate-One

👍5

Most helpful comment

@BillyONeal @JackBoosY @Neumann-A
cc: @jmelas @ImmaturityRules

I found the cmake find_program "required" option was added between 3.17.3 and 3.18.0, at least it is stated on the 3.18.0 doc page: https://cmake.org/cmake/help/latest/command/find_program.html where the 3.17.3 does not mention the required parameter: https://cmake.org/cmake/help/v3.17/command/find_program.html?highlight=find_program
The "required" option has this definition in cmake 3.18.0 documentation
REQUIRED
Stop processing with an error message if nothing is found.
Line 279 of script vcpkg_fixup_pkgconfig.cmake is only checking the locations bin, /usr/bin or /usr/local/Cellar/pkg-config/0.29.2_3 for the pkg-config executable
This why it works on all platforms using an *NIX OS works, but for windows, it uses MSYS found in the <vcpkg-root>\downloads\toos\MSYS\msys<bitness> aka MSYS_ROOT.
I figure out if I make the function evaluate for CMAKE_HOST_WIN32 first with the following changes starting at line 278 to script vcpkg_fixup_pkgconfig.cmake it works as follows:
- Original: Evaluates non-windows hosts first

- Modified: Evaluates windows hosts first

This keeps the non-windows host find_program from executing/failing and stopping an otherwise successful build on windows but still allows the non-windows host to run the find_program for the pkg-config executable.
Also discovered by testing, on a windows host, with VS2017/VS2019 installed with MS version of CMake (C:\Program Files (x86)\Microsoft Visual Studio\<vs_version>\<vs_type>\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe) and/or the vcpkg downloaded/installed cmake (3.17.20032601-MSVC_2) the script in the current state works, as versions of CMake prior to 3.18.0 does not stop the find_program for executing and failing with the REQUIRED parameter.
This fix also works on the vcpkg upgrade --no-dry-run with the same failure message.
I did not see any *NIX issues building the same ports using this change/fix - openSUSE Leap 15.2 on WSL2 and Hyper-V.
I do not know how this will affect the CI pipeline builds as I do not have access/account for those.

Let me know if this would be an OK change and if you want me to submit a PR to make this change?

Thank you and have a wonderful week!!!

StarGate-One on 25 Jul 2020

👍8 🎉1

All 15 comments

@BillyONeal @JackBoosY @Neumann-A
I am trying to figure out the changes in [vcpkg/scripts/pkgconfig] rewrite pkg-config check for libraries (#11550) and how to fix for port bzip2 (my most important port) as it fails now after applying latest:

git pull - upgrades vcpkg to PR #11550
bootstrap-vcpkg.bat -verbose -disableMetrics -win64 -withVSPATH %VCPKG_VISUAL_STUDIO_PATH% did this because some vcpkg source files changed.
- vcpkg upgrade --no-dry-run fails with Could not find PKGCONFIG using the following names: pkg-config
- vcpkg install bzip2 fails with Could not find PKGCONFIG using the following names: pkg-config
Then I tried by delete/recreate vcpkg instance:
- Delete vcpkg instance
- Clone new vcpkg instance
- bootstrap as mentioned above
- vcpkg install bzip2 still fails with Could not find PKGCONFIG using the following names: pkg-config

Any assistance would be greatly appreciated.

Attached is vcpkg install bzip2 --debug console log, it may/may not help.
bzip2-console-with-DEBUG.log

Here is the rest of the logs from buildtrees\bzip2 that have data in them - I did not include the 0 byte files.
config-x64-windows-out.log
install-x64-windows-dbg-out.log
install-x64-windows-rel-out.log
patch-x64-windows-0-err.log
patch-x64-windows-1-err.log
stdout-x64-windows.log
x64-windows.vcpkg_abi_info.txt

Output from vcpkg upgrade --no-dry-run --debug (only bzip2 is being upgrade):
vcpkg-upgrade-2019.test-no-dry-run.log

StarGate-One on 24 Jul 2020

I get exactly the same message when installing zlib with vcpkg!
After latest update to vcpkg #11550 Commit 458c20e
I use VCPKG_DEFAULT_TRIPLET=x64-windows-static

jmelas on 24 Jul 2020

Looks like they've started using MSYS for configuration and have broken vcpkg building certain packages for now. It's something to do with the signature of the packages MSYS's pacman is trying to install.

ImmaturityRules on 25 Jul 2020

@BillyONeal @JackBoosY @Neumann-A
cc: @jmelas @ImmaturityRules

I found the cmake find_program "required" option was added between 3.17.3 and 3.18.0, at least it is stated on the 3.18.0 doc page: https://cmake.org/cmake/help/latest/command/find_program.html where the 3.17.3 does not mention the required parameter: https://cmake.org/cmake/help/v3.17/command/find_program.html?highlight=find_program
The "required" option has this definition in cmake 3.18.0 documentation
REQUIRED
Stop processing with an error message if nothing is found.
Line 279 of script vcpkg_fixup_pkgconfig.cmake is only checking the locations bin, /usr/bin or /usr/local/Cellar/pkg-config/0.29.2_3 for the pkg-config executable
This why it works on all platforms using an *NIX OS works, but for windows, it uses MSYS found in the <vcpkg-root>\downloads\toos\MSYS\msys<bitness> aka MSYS_ROOT.
I figure out if I make the function evaluate for CMAKE_HOST_WIN32 first with the following changes starting at line 278 to script vcpkg_fixup_pkgconfig.cmake it works as follows:
- Original: Evaluates non-windows hosts first

- Modified: Evaluates windows hosts first

This keeps the non-windows host find_program from executing/failing and stopping an otherwise successful build on windows but still allows the non-windows host to run the find_program for the pkg-config executable.
Also discovered by testing, on a windows host, with VS2017/VS2019 installed with MS version of CMake (C:\Program Files (x86)\Microsoft Visual Studio\<vs_version>\<vs_type>\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe) and/or the vcpkg downloaded/installed cmake (3.17.20032601-MSVC_2) the script in the current state works, as versions of CMake prior to 3.18.0 does not stop the find_program for executing and failing with the REQUIRED parameter.
This fix also works on the vcpkg upgrade --no-dry-run with the same failure message.
I did not see any *NIX issues building the same ports using this change/fix - openSUSE Leap 15.2 on WSL2 and Hyper-V.
I do not know how this will affect the CI pipeline builds as I do not have access/account for those.

Let me know if this would be an OK change and if you want me to submit a PR to make this change?

Thank you and have a wonderful week!!!

StarGate-One on 25 Jul 2020

👍8 🎉1

The above bugfix plus replacing the file <vcpkg-root>\scripts\cmake\vcpkg_acquire_msys.cmake with this version:

vcpkg_acquire_msys.cmake

fixed my vcpkg and I can upgrade bzip2 and zlib.

ImmaturityRules on 25 Jul 2020

👍4

@BillyONeal @JackBoosY @Neumann-A
cc: @jmelas @ImmaturityRules

I found the cmake find_program "required" option was added between 3.17.3 and 3.18.0, at least it is stated on the 3.18.0 doc page: https://cmake.org/cmake/help/latest/command/find_program.html where the 3.17.3 does not mention the required parameter: https://cmake.org/cmake/help/v3.17/command/find_program.html?highlight=find_program

The "required" option has this definition in cmake 3.18.0 documentation
REQUIRED
Stop processing with an error message if nothing is found.

Line 279 of script vcpkg_fixup_pkgconfig.cmake is only checking the locations bin, /usr/bin or /usr/local/Cellar/pkg-config/0.29.2_3 for the pkg-config executable

This why it works on all platforms using an *NIX OS works, but for windows, it uses MSYS found in the <vcpkg-root>\downloads\toos\MSYS\msys<bitness> aka MSYS_ROOT.

I figure out if I make the function evaluate for CMAKE_HOST_WIN32 first with the following changes starting at line 278 to script vcpkg_fixup_pkgconfig.cmake it works as follows:

Original: Evaluates non-windows hosts first

Modified: Evaluates windows hosts first

This keeps the non-windows host find_program from executing/failing and stopping an otherwise successful build on windows but still allows the non-windows host to run the find_program for the pkg-config executable.

Also discovered by testing, on a windows host, with VS2017/VS2019 installed with MS version of CMake (C:\Program Files (x86)\Microsoft Visual Studio\<vs_version>\<vs_type>\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe) and/or the vcpkg downloaded/installed cmake (3.17.20032601-MSVC_2) the script in the current state works, as versions of CMake prior to 3.18.0 does not stop the find_program for executing and failing with the REQUIRED parameter.

This fix also works on the vcpkg upgrade --no-dry-run with the same failure message.

I did not see any *NIX issues building the same ports using this change/fix - openSUSE Leap 15.2 on WSL2 and Hyper-V.

I do not know how this will affect the CI pipeline builds as I do not have access/account for those.

Let me know if this would be an OK change and if you want me to submit a PR to make this change?

Thank you and have a wonderful week!!!

This fix is working, many thanks.

Cobollatin on 25 Jul 2020

👍1

vcpkg_fixup_pkconfig.cmake needs to be reworked - PR #12569 is in the works but in the mean time you can make the changes shown by myself @StarGate-One to the script found in <vcpkg-root>\scripts\cmake to temporarily fix this issue.
vcpkg_fixup_pkgconfig.cmake.txt
vcpkg_acquire_msys.cmake needs to have the MSYS/mingw crypto keyrings added, @ImmaturityRules has attached a copy to this same issue which fix this issue, but a PR needs to be done with those changes. You can download dthe file and put it in the same location <vcpkg-root>\scripts\cmake to temporarily fix this issue.
vcpkg_acquire_msys.cmake.txt

I have attached a copy of both, download, remove the .txt extension and copy to <vcpkg-root>\scripts\cmake folder and it should work.

StarGate-One on 27 Jul 2020

I tried applying both fixes. The first fix regarding vcpkg_fixup_pkconfig.cmake worked for me, but apparently the second one didn't.
Error description:

CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:72 (message):
    Command failed: C:/dev/vcpkg/downloads/tools/msys2/msys64/usr/bin/bash.exe --noprofile --norc -c "pacman -S --noconfirm --needed pkg-config"
    Working Directory: C:/dev/vcpkg/downloads/tools/msys2
    Error code: 1
    See logs for more information:
      C:\dev\vcpkg\buildtrees\bzip2\msys-pacman-x86-windows-err.log

Call Stack (most recent call first):
  scripts/cmake/vcpkg_acquire_msys.cmake:149 (vcpkg_execute_required_process)
  scripts/cmake/vcpkg_fixup_pkgconfig.cmake:280 (vcpkg_acquire_msys)
  ports/bzip2/portfile.cmake:51 (vcpkg_fixup_pkgconfig)
  scripts/ports.cmake:79 (include)


Error: Building package bzip2:x86-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `.\vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: bzip2:x86-windows
  Vcpkg version: 2020.06.15-nohash

And the other error log, made by pacman (msys-pacman-x86-windows-err.log):

error: mingw32: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" is unknown
error: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" could not be looked up remotely
error: mingw64: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" is unknown
error: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" could not be looked up remotely
error: msys: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" is unknown
error: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" could not be looked up remotely
error: database 'mingw32' is not valid (invalid or corrupted database (PGP signature))
error: database 'mingw64' is not valid (invalid or corrupted database (PGP signature))
error: database 'msys' is not valid (invalid or corrupted database (PGP signature))

TheReclif on 27 Jul 2020

@TheReclif Have you tried removing the <vcpkg-root>\downloads\tools\msys2 folder and then trying to upgrade bzip2 again?

ImmaturityRules on 27 Jul 2020

I tried with a clean install of vcpkg and it worked. Thanks!

TheReclif on 27 Jul 2020

@Neumann-A Could you please take a look?

Thanks.

JackBoosY on 27 Jul 2020

It might be nice to add some minimal integration testing to your CI recipe (eg, install zlib after vcpkg build) so that regressions like this show up obviously and early.

rivy on 29 Jul 2020

👍4

@rivy, good point! thanks for the suggestion!

@strega-nil @BillyONeal, what do you think about installing zlib before test each PR in CI pipeline?

PhoebeHui on 29 Jul 2020

For this regression to show up in CI cmake 3.18 would have been necessary. Vcpkg is still testing against 3.17 in Ci. Furthermore the script change nowadays reruns all ports depending on it in CI since the internal logic was recently changed.
(So zlib was already rebuild in the PR before the merge)