Conan: Non-intrusive integration with CMake

[ldionne]

According to the documentation here, the canonical way of using Conan with modern CMake is to add the following in my CMakeLists.txt:

include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup(TARGETS)

add_executable(my_app my_app.cpp)
target_link_libraries(my_app CONAN_PKG::FoobarLib)

However, this couples the build system (CMakeLists.txt) with the package manager (Conan). For example, if I want to use the above project inside my company which has its own package manager, I'll have to edit the CMakeLists.txt to _not_ use Conan.

On the CMake side of things, the canonical way of dealing with dependencies is to use find_package as such:

find_package(Foobar)
add_executable(my_app my_app.cpp)
target_link_libraries(my_app Foobar::FoobarLib)

The intent is that the FoobarConfig.cmake file that was installed when Foobar was installed (using _some_ package manager) can be found using the CMAKE_PREFIX_PATH, and everything works nicely. Of course, this requires the Foobar project to be CMake friendly by installing the FoobarConfig.cmake file, but this is a reasonable expectation because that file may express non-trivial requirements and logic (e.g. dependency on a C++ standard, etc.).

My question

Does Conan support the canonical, non-intrusive CMake workflow? More specifically, the workflow I'd like is the following:

$ cat CMakeLists.txt
> find_package(Foobar)
> add_executable(my_app my_app.cpp)
> target_link_libraries(my_app Foobar::FoobarLib)

$ cat conanfile.txt
> [requires]
> Foobar/1.0.0@someone/stable

# This would not generate any file, but would instead print a list of paths 
# to be added to CMAKE_PREFIX_PATH, basically a list of where the dependencies 
# are installed
$ PREFIX_PATH=$(conan install)

$ mkdir build && cd build && cmake .. -DCMAKE_PREFIX_PATH=${PREFIX_PATH}

This way, my CMakeLists.txt would know _nothing_ about Conan and there would be no need for magically-generated (and usually wrong (*)) Conan targets.

(*) The automatically-generated targets lack important information that can't reasonably be known to Conan, such as target-level requirements on specific C++ features. For example, the auto-generated target for Boost.Hana (a header-only library) only contains include directories, when additional target requirements such as cxx_std_14 should be present. This is not something Conan can reasonably know, so the only way to do this properly is for the library to install a HanaConfig.cmake file when it is installed.

[bilke]

We successfully use Conan in a completely optional way. We use standard find_package() to check libraries no matter Conan is involved or not. Also we configure the conan install step inside CMake to install optional dependencies when certain CMake options are set. Please have a look at our implementation.

[memsharded]

This would be something nice to have, but there are some considerations to think about. Mostly related to:

Of course, this requires the Foobar project to be CMake friendly by installing the FoobarConfig.cmake file, but this is a reasonable expectation because that file may express non-trivial requirements and logic (e.g. dependency on a C++ standard, etc.).

It is a reasonable expectation, but it is not satisfied in practice. More a wish I think. There are many packages (and many important ones) that are not using CMake at all. Also, some others that are cmake based, do not provide a FoobarConfig.cmake

Transitivity of cmake Find/Config files is still an issue. What happens when FooBar depends on other things, and there are several levels of transitive dependencies, including diamonds. I am not saying it is impossible, but I still have to see such a setup based in cmake that works robustly in practice.

$ PREFIX_PATH=$(conan install)

I am not sure how this would be implemented in practice. conan install does many things, output a lot of useful information and is a generic command that can install dependencies and prepared them to be consumed by different build systems. I don't think it is possible to make it return the PREFIX_PATH in a clean way

There are also other limitations, as the inability of Find_package to deal with multi-config or generator-expressions. So it is not possible to use multi-config environments (like VS) to target multiple configurations Release/Debug x86/Win32 simultaneously based on find_package.

So, while I understand your request and it is something that would be nice to have, it is not possible to have it. In practice, the problem is simply addressed by:

if (EXISTS ${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
     include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
     conan_basic_setup(TARGETS)
     ...
endif()

Which I'd say it is a very small issue compared with the provided benefit.

[lasote]

Hana (a header-only library) only contains include directories, when additional target requirements such as cxx_std_14 should be present. This is not something Conan can reasonably know

It depends on how you develop the recipe for the library, you can use the self.cpp_info object to let consumers know how to "link" (or not link in the example you are giving) with your library, so the Conan targets will be wrong if the recipe is wrong. We are currently working to find the better and standard (common to all recipes) solution to model the std version.

In the meantime, you could add some flags to self.cpp_info.cppflags to activate the cxx_std_14, I know, not ideal because the flag depends on the compiler, but the important thing is that you could do it reading the current setting to know the user's compiler. That is why I said that all depends on how you implement the recipe.

[ldionne]

@memsharded:

This would be something nice to have, but there are some considerations to think about. Mostly related to:

Of course, this requires the Foobar project to be CMake friendly by installing the FoobarConfig.cmake file, [...]

It is a reasonable expectation, but it is not satisfied in practice. More a wish I think.

The C++ community is still transitioning to modern CMake and they are starting to understand how this should work. It's becoming better and better, but Conan is not helping by providing yet another way of importing dependencies from CMake.

There are many packages (and many important ones) that are not using CMake at all.

That's not a problem at all! Even a Makefile-based project can install a XYZConfig.cmake file, and should if it wants to be CMake-friendly.

Also, some others that are cmake based, do not provide a FoobarConfig.cmake

They should if they want to be usable from CMake. The burden is not on Conan to make this available, it's on the project itself.

Also, it turns out that the targets generated by Conan are actually wrong in some cases. For example, a library I'm working on requires clients to use the -Wno-gnu-string-literal-operator-template flag when the compiler supports it. This is nicely encoded in the generated DynoConfig.cmake file, but Conan can't possibly have that knowledge and automatically generate a __correct__ CMake target. And I don't want to duplicate this knowledge both in my build system and my package management logic.

Transitivity of cmake Find/Config files is still an issue.

It's being worked on. See this CMake issue.

What happens when FooBar depends on other things, and there are several levels of transitive dependencies, including diamonds. I am not saying it is impossible, but I still have to see such a setup based in cmake that works robustly in practice.

It seems to me like Conan is trying to work around some deficiencies of CMake. While this is well intentioned, I think it's also slightly harmful since it does that by not supporting CMake best practices. It's okay if Conan provides this magic for people that want it, but I think it should at least support the canonical way to find dependencies with CMake for those that want their build system and package managers to stay decoupled.

$ PREFIX_PATH=$(conan install)

I am not sure how this would be implemented in practice. conan install does many things, output a lot of useful information and is a generic command that can install dependencies and prepared them to be consumed by different build systems. I don't think it is possible to make it return the PREFIX_PATH in a clean way

Conan could provide a semicolon-delimited list of paths to the root of where the dependencies were installed. It already provides something very similar in the conanbuildinfo.cmake file:

set(CONAN_CMAKE_MODULE_PATH "/Users/ldionne/.conan/data/Boost/1.64.0/..." "..." "...")
...
set(CMAKE_PREFIX_PATH ${CONAN_CMAKE_MODULE_PATH} ${CMAKE_PREFIX_PATH})

I'm not sure exactly what is included in CONAN_CMAKE_MODULE_PATH, but it seems like Conan has all the information required to output what should be added to the CMAKE_PREFIX_PATH. What exactly is included in CONAN_CMAKE_MODULE_PATH?

So, while I understand your request and it is something that would be nice to have, it is not possible to have it.

Please see above and let me know if you still think this is not implementable.

In practice, the problem is simply addressed by:

if (EXISTS ${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
     include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
     conan_basic_setup(TARGETS)
     ...
endif()

Which I'd say it is a very small issue compared with the provided benefit.

While this indeed seems to set the CMAKE_PREFIX_PATH correctly so that one can use find_package, as @bilke points out, it also automatically defines all those targets I don't want and requires me to change my CMakeLists.txt. Imagine what would happen if all the package managers I need to support required modifying my CMakeLists.txt? It would become messy pretty quickly. Instead, this mode of operation could easily be supported as an option (something like conan install --cmake-prefix-path) or something similar.

To be honest, I see the lack of support for this workflow as a pretty big issue in the integration between Conan and CMake, especially since the workflow Conan supports contradicts CMake best practices.

@lasote:

In the meantime, you could add some flags to self.cpp_info.cppflags to activate the cxx_std_14, I know, not ideal because the flag depends on the compiler, but the important thing is that you could do it reading the current setting to know the user's compiler. [...]

That's the whole point; this is not package manager level information, it's build system level information. I think if you go down that path fully (and not only for cxx_std_xyz), you'll quickly realize you're in build system land.

[progician]

I think have somewhat similar ideas with Iodonne. Basically in our set up we only use the the conan_set_find_paths() function which sets the prefix paths. From then on, the standard find_package system takes over.

The transitivity of Config files are actually the following: Generally, you don't generate, but rather hand roll the XXXXConfig.cmake file which will call find_package of the needed libraries and then includes all the generated target exports. Not necessary the most automatic way of dealing with it, but works reasonably easily (interdependence of targets in cmake can be a pain to deal with this way, so it is advised to install targets together into a single export file). Even better, CMake provide the find_dependency() macro which gives a nicer error tracking for these dependencies and only search a package if it wasn't found already. This can sometimes a problem when using with library collections, such as Boost, but in that case you can just fall back to find_package().

So, in our workflow, we really don't have any use of the rest of the conanbuildinfo.cmake file, it's only the prefix path list we really need. If Conan can generate the this variable already, it would be nice to be able to optionally just get that.

[ldionne]

If Conan can generate the this variable already, it would be nice to be able to optionally just get that.

Right, basically what I'm asking is for a way to surface only that part, and without having to add anything to my CMakeLists.txt. @memsharded @lasote When worded that way, do you think this is something that can be done? That would make Conan clearly compatible with CMake best practices.

[memsharded]

I have put a simple proof of concept here: https://github.com/memsharded/cmake_find_generator

The generator implementation is straightforward: https://github.com/memsharded/cmake_find_generator/blob/master/conanfile.py

Right now it should be generated locally (follow README), but also could be built-in or the generator can be contributed to conan-center and use it as a dependency.

It uses a dedicated generator that will generate a conan_cmake_find_paths file which is just a list (separated with ;) of package paths, that could be loaded and fed to cmake from the command line.
As long as cmake is able to successful run find_package routines with those paths, everything should work. But please let me note and clearly highlight:

• Some (many) packages won't have a find cmake script at all, so they won't work
• Some (many) packages find cmake scripts might not work completely for all use cases, platforms and systems.
• There are issues with transitive dependencies, that cmake still hasn't solved, so transitive dependencies are very likely to have issues and fail

[progician]

@memsharded What do you mean by CMake hasn't solved the transitive dependency issue? The config files can call the find_dependency macro, which can be used for transitive dependencies. What is the issue with it? Sure it is not automatically generated by the "instal( TARGETS ... EXPORT ... )" command but it can be expressed and I'm using that in our company's packages already with no issues.

[memsharded]

When you depend on A=>B,C, then B=>D, C=>D, E, and D=>F,G, and E=>G, etc, etc. I haven't seen find cmake scripts working reliably for this use case. I don't say it is impossible, as you said:

The config files can call the find_dependency macro

just that the mainstream find cmake scripts are for very flat use cases, and the ecosystem is not well prepared to handle more complex dependency graphs. CMake users are increasingly adopting modern cmake practices, but that doesn't mean that it is still there, so in practice it might fail in some cases.

[lasote]

We are going to improve the "non-intrusive" CMake integration for 1.4

a) A new generator "cmake_toolchain" that will generate a cmake file to be included as a toolchain like vcpkg does.

b) A new virtual environment for cmake (pending technical viability confirmation) that once activated will set the env vars needed by cmake to locate the deps, find_package, etc.

Both of them won't require modifying the CMakeLists.txt, and its usage will be more limited than the classic approach of the "conanbuildinfo.cmake" file but enough for the operation mode requested in this issue.

[lasote]

I think finally for the 1.4 we are good developing "a)" because the generated cmake file could be included in other user custom toolchain in case he needs to use a toolchain for cross building.