Vcpkg: Separate release and debug builds

[albertziegenhagel]

Currently the tool-chain file adds both installed/<triplet>/debug and installed/<triplet> to the CMAKE_PREFIX_PATH for Debug builds but only installed/<triplet> for release builds.

Because of this it is possible that a debug build silently picks up a release library if for some reason the debug library could not be found.
An example for this is the FindGLEW.cmake script, that is distributed by VTK. This file looks for glew32.lib only, but not for glew32d.lib for debug builds, so here silently the release library is picked, and no error is being generated. In my opinion it can never be correct for a debug build to use release libraries, so ideally we should not add installed/<triplet>.

I know that with the current folder structure we need to always add installed/<triplet> so that the share and tools folder is found correctly. So in my opinion we should instead put the release libraries into a sub-folder just like the debug libraries are.

Something like this:

vcpk/
└── installed/
│   └── x64-windows/
│      ├── debug
│      │   ├── bin/
│      │   └── lib/
│      ├── release/
│      │   ├── bin/
│      │   └── lib/
│      ├── include/
│      ├── share/
│      └── tools/
...

Ports that use vcpkg_fixup_cmake_targets could be transitioned easily. Other ports should be changed to make use of vcpkg_fixup_cmake_targets.

I think if it is decided to do this change it should be done as early as possible, before even more incompatible ports will be created.

[codicodi]

We can also simply tell cmake to ignore installed/<triplet>/lib by adding it to CMAKE_IGNORE_PATH in debug builds.

[albertziegenhagel]

Cool, I wasn't aware of that feature! That would make things very easy.

[ras0219-msft]

The reason we add both paths is to support Multi-Configuration generators like Visual Studio, Eclipse, etc.

When using a multi-configuration generator, CMake supports doing the following:

find_library(ZLIB_DEBUG zlibd)
find_library(ZLIB_RELEASE zlib)
link_libraries(debug ${ZLIB_DEBUG} optimized ${ZLIB_RELEASE})

This is unfortunately (still) widely used in the built-in CMake Modules, especially in older CMake versions.

More generally, if we don't add both paths to the search list, it is _impossible_ for any libraries supported by the builtin FindXYZ.cmake scripts to be properly handled; only -config.cmake providing libraries will work. There's an unfortunately large number of mature, important libraries that won't convert in the forseeable future.

We might be able to limit the problem by changing our behavior based on whether the generator is multi-config or not, but I'd like to treat that inconsistency as a last resort.

[albertziegenhagel]

I think you have an inconsistency already. First of all I think there are two different use cases: the first is what happens while vcpkg is building a package, the second is when a user is building an own project using packages build by vcpkg.

In the first case multi-configuration generators do not really matter, because after configuring, vcpkg always builds only one configuration. So if vcpkg configures for debug with the VS generator and only zlibd.lib is found, it would not be a problem, since we will build the debug configuration only. We are doing the same thing for release builds already.

For an end user it looks differently. He would want to configure a single VS project that includes both, debug and release libraries. But currently if he passes -DCMAKE_BUILD_TYPE=Release while he configures his VS project, he would only get the release libraries. The debug path is only added if CMAKE_BUILD_TYPE is debug or not set. Please keep in mind that it is not uncommon to set CMAKE_BUILD_TYPE even when using multi-configuration generators. I've even seen some CMake scripts out there where CMAKE_BUILD_TYPE will be set to a default value (most often release) when it is not set by the user. Usually this is not a problem, because by all native CMake functions the build type is just ignored for multi-configuration generators.

So maybe it makes sense to distinguish between those use cases: if something like VCPKG_IN_VCPKG is set, we set only one, either release or debug patch (or better use CMAKE_IGNORE_PATH). So vcpkg will create consistent release and debug builds. If it is not set, we always add both paths. Maybe we could use CMAKE_BUILD_TYPE to decide which of the paths comes first.

Am I missing something?

[ras0219-msft]

But currently if he passes -DCMAKE_BUILD_TYPE=Release while he configures his VS project, he would only get the release libraries. The debug path is only added if CMAKE_BUILD_TYPE is debug or not set. Please keep in mind that it is not uncommon to set CMAKE_BUILD_TYPE even when using multi-configuration generators. I've even seen some CMake scripts out there where CMAKE_BUILD_TYPE will be set to a default value (most often release) when it is not set by the user.

This is definitely true (unfortunately). However, I reason that because it's _in the user's project_, it's hopefully something they control and could change.

We could switch based on inside-vcpkg versus outside-vcpkg, but it really doesn't feel like this is the right solution since it should really affect both cases. As an aside, the fact that we pretend VS is a single configuration generator is really an internal workaround for this problem. It would be amazing if we could use MSBuild internally as a multi-config generator since it would remove a CMake configure pass and significantly reduce build times for many libraries.

If it is not set, we always add both paths. Maybe we could use CMAKE_BUILD_TYPE to decide which of the paths comes first.

What about:

• CMAKE_BUILD_TYPE unset: add both paths, release first.
• CMAKE_BUILD_TYPE = Debug: add only debug
• CMAKE_BUILD_TYPE = Release: add only release

This means users who set CMAKE_BUILD_TYPE during a multi-config generator will "forcibly" get that type of library. Some projects will break that currently work, but it also serves as a workaround for libraries that have the same name across debug and release -- you can generate separate solutions for release and debug if one of your libraries is misbehaving. This is the same workaround that we use inside vcpkg, which avoids that particular inconsistency and would fix package builds.

[VikingExplorer]

I desperately need the output structure that @albertziegenhagel suggests. What can I change to achieve this?

[MikeGitb]

@ras0219-msft: Slightly off topic, but as it was mentioned here: I was also wondering why the release directory is "missing". Could you maybe explain the reasoning behind this when you have a moment?

[richardchien]

so here silently the release library is picked, and no error is being generated

Actually, while using fmt (at commit 90e627c), an error of "mismatch detected for 'RuntimeLibrary'" (LINK2038) will be generated. So I think seperating of release and debug build is a good suggestion.

[degski]

@albertziegenhagel @ras0219-msft You seem to say you are linking release and debug libraries. If you do this with C-libraries (the example you are quoting) this will work. If you do this with C++-libraries, this will not work, as you will have iterator debug level mismatch errors (LNK2038 error). This is further explained in this answer on SO, I'm sure it's all explained on MSDN as well. This is a feature of the MS-STL (I'm not ironic here) that is very very useful, and should in no way be circumvented by vcpkg (IMHO). The fact that this is possible on nix-systems does not make this good practice. vcpkg should, as it does now, build a release and a debug version of all packages and nudge people to adopt the correct practice. VS2017, now, also supports the /JMC-switch, which avoids having to step through linked library code.

@ras0219-msft I would urge you not to change that behavior (certainly not for windows targets).

I do agree, though, that having that the proposed directory structure in the original issue (the top post) makes more sense (but not for the reasons put forward, but for simplicity, debug builds are not second class citizens, they are a crucial element of developing bug-free (as free as it will ever get) software).

[JasonDictos]

Question guys, how are we supposed to be able to use debug libs, with custom configuration types and this change? E.g. I want to define SanitizeDebug/DebugSanitize both don't match the pattern for ^Debug$, so I simply can't define a custom build mode, with vcpkg, if I want it to automatically add the debug lib path first now... correct?

Also isn't it tradition in cmake to just match on the prefix? E.g. if ("${CMAKE_BUILD_TYPE}" MATCHES "Rel") that way you can add custom configuration types that match, e.g. RelWithDebInfo on windows (the default). So I would think a more proper fix here is to match on just 'Deb' instead.

[alokpr]

Reviving an old thread. We started looking into vcpkg so that we can build third-party dependencies with the same build configuration as our source code. Specifically we want to enable -fsanitize. We have defined a custom CMAKE_BUILD_TYPE (ASAN, TSAN, etc) for our own source code. IIUC this is currently not possible with vcpkg because only Debug and Release build types are supported. Is my understanding correct?

What is the recommended way to add custom build configurations? It looks like many ports specifically only support Debug/Release configurations. Would vcpkg maintainers be open to officially support sanitize build types?