Vcpkg: Questions about manifest mode

[studoot]

I like the direction that manifest mode is taking, having played with it a bit. However, for my work, I have a few queries, which may or may not already be on the roadmap...

I develop for both Linux and Windows, using WSL on Windows to be able to build for both platforms within the same filesystem location. I currently use a bash script to invoke Windows and Linux variants of vcpkg install to install some set of libraries for each of my desired triplets & then vcpkg export them so I can configure the libraries I use alongside my source (to decouple from the vcpkg location).

In a simple test project, I created a manifest:

{
   "name": "test-rc",
   "version-string": "0.0.1",
   "dependencies": [
      "rapidcheck", "catch2", "fmt"
   ]
}

This worked with my CMake toolchain, but I noted the following issues:

1. Part of the package installation process invoked by a CMake operation involved removing installed packages for other triplets.

   This means that configuring CMake for Windows/Linux (with cmake.exe -B winbuild -S . -DCMAKE_TOOLCHAIN_FILE=$VCPKG_ROOT/scripts/buildsystem/vcpkg.cmake for Windows, cmake -B linbuild -S . -DCMAKE_TOOLCHAIN_FILE=$VCPKG_ROOT/scripts/buildsystem/vcpkg.cmake for Linux) or performing a build with cmake(.exe) --build (winbuild|linbuild) deleted the 'other' triplet's installation.

   For example, if I made a build for Windows with

   rm -rf ./winbuild
   cmake.exe -B winbuild -S . -DCMAKE_TOOLCHAIN_FILE=$(wslpath -w $VCPKG_ROOT)/scripts/buildsystem/vcpkg.cmake
   cmake.exe --build winbuild
   

   then my vcpkg_installed directory would contain an x86-windows directory with my desired libraries. Subsequently making a Linux build directory with:

   rm -rf ./linbuild
   cmake -B linbuild -S . -DCMAKE_TOOLCHAIN_FILE=$VCPKG_ROOT/scripts/buildsystem/vcpkg.cmake
   

   would delete the x86-windows directory and there would be an x64-linux there instead.

   This obviously doesn't work too well for me, given how I work, and I suspect the same thing would happen when swapping between (say) x86-windows and x64-windows triplets, which is probably a more likely scenario to encounter. It would be nice if vcpkg manifests could be used to target multiple triplets more smoothly...

2. The build is still using the vcpkg.cmake from $VCPKG_ROOT - is there an intention to provide a manifest mode equivalent of vcpkg export, which exports sufficient of vcpkg's CMake scripts that there is no retained dependency on $VCPKG_ROOT?

Overall, as I say, I like the concept of a manifest file (especially if the ability to specify versions of dependencies becomes a thing)

[JackBoosY]

@strega-nil Can you take a look?

Thanks.

[strega-nil]

@studoot as for your first question, it's now been corrected slightly by putting the vcpkg_installed in a specific build directory. As for the second, we have not yet considered vcpkg export alongside manifest mode; not that it won't happen, but I don't think I personally feel comfortable defining how it should work.

[studoot]

as for your first question, it's now been corrected slightly by putting the vcpkg_installed in a specific build directory.

That's probably the best solution for vcpkg_installed with multiple triplets - thanks for that.

[meastp]

Could you please provide an example on how this is fixed? I don't understand, sorry...

When I run vcpkg in a folder with manifest and --triplet x64-windows, then run again with --triplet x86-windows, the vcpkg_installed\x64-windows is deleted.. ? How do I achieve installing dependencies for both triplets in a single step, please? :)

[strega-nil]

@meastp If one uses the CMake toolchain, it's fixed; if one uses it manually, then it hasn't been, unfortunately. It'd probably be best to define two distinct vcpkg_installed directories with the --x-installed-root option.

[meastp]

Failed to take the filesystem lock on C:\...\vcpkg_121\.vcpkg-root

@strega-nil I also found that I had to add --x-wait-for-lock to vcpkg\scripts\buildsystems\msbuild\vcpkg.targets because it is invoked a lot of times during compilation, and will fail with lock failures otherwise. Should I file a separate bug about this?

[strega-nil]

hrm. That's not great. I wonder why it's being called so many times? I feel like it should only be being called once for each build...

[meastp]

@strega-nil I, too, was surprised as the dependencies surely shouldn't change during the build. Once before the build like restoring nuget packages should suffice :)

[strega-nil]

right. In my testing, I didn't get that issue, but I also don't really understand MSBuild, so I may very well have written a bug. I'd definitely say open an issue.

[meastp]

@strega-nil Ok - should I also open a separate issue for using MSBuild

to define two distinct vcpkg_installed directories with the --x-installed-root option

because the original problem in this issue is solved only in CMake?

[strega-nil]

@meastp yes, correct.

[meastp]

@strega-nil I see that the lock-issue has been fixed (added to command line). I think I can live without --x-installed-root option for now. Thanks for your help - looking forward to all the exciting new features in vcpkg!

[JackBoosY]

Solved.