Conan: CMake generators including .cmake in builddirs

[lasote]

Some packages like Protoc require to export some CMake macros.
Typically these macros are declared in the findXXX script when calling find_package, but packages in the new conan-center index, read #KB-H016, doesn't allow to export findXXX or configXXX files.

So, there should be a way that a package like protoc can declare some macros to be reused by the consumers. For example when:

• Calling conan_basic_setup()
• Calling find_package and using a generated by conan find/module script.

These cmake files to be imported should be located in a build_dir https://docs.conan.io/en/latest/reference/conanfile/attributes.html#cpp-info

But I don't think we can import all of them automatically without breaking something, it should be or an opt-in mechanism in the recipe or a file name convention, like conan_export*.cmake or similar?

We need to discuss it @memsharded @jgsogo @uilianries @SSE4

[uilianries]

Protoc is a good example. The macros are separated in 3 files. So we would need to extract manually to an unique file and export it to package_folder/res/cmake maybe.

When running conan_basic_setup it could search for the pattern conan_export*.cmake in package_folder/res/cmake or package_folder/lib/cmake and include them automatically. Of course, we could add a new parameter to be more explicit with conan_basic_setup.

conan_basic_setup(CMAKE_EXPORTS lib/cmake/protoc/conan_exports_protobuf.cmake)

[lasote]

Of course, we could add a new parameter to be more explicit with conan_basic_setup.

No, that kind of code in the consumer is what a package manager try to solve.
I see three approaches:

• naming convention: All the cmake files included in buildirs following a pattern will be included.
• opt-in in the original recipes (protoc in this case). Something line a new cppinfo.something?
• include all cmake files in builddirs automatically, but I think this might break.

[jgsogo]

I want to tell a different point of view related to this topic. When I've worked with protobuffers in my last projects and Conan was involved there, I've ended up calling protoc from the ConanFile::source method. IMHO it is an error of the library to provide the functionality only for the CMake consumers, I find it has many points in common with the cpp_info arguments. I'm not sure if this is something we want to (or can) add to the equation.

---

Said that, to keep compatibility for existing code bases (thinking about protoc macros), the functionality has to be added automatically to the consumer somehow as it was added before using Conan (I think it shouldn't be an opt-in): if these extra _buildir/functionaly.cmake_ files are declared in cppinfo.something they have to be included from the FindXXX.cmake or the conan_basic_setup, it shouldn't be that hard.

Nevertheless, for the protobuf/protoc there is an extra problem: the consumer will find_package(protobuf), but those macros are provided by the protoc package. If those are different packages, how can we find those extra _cmake files_ provided by protoc? Is it legit for protobuf package to include files from a protoc one just because we know it is the way it works?

[lasote]

cppinfo.something they have to be included from the FindXXX.cmake or the conan_basic_setup, it shouldn't be that hard.

Agree

Is it legit for protobuf package to include files from a protoc one just because we know it is the way it works?

I would say yes. we can do the trick

[KerstinKeller]

Hi guys,
I think what you are trying to achieve might be close to impossible, and you're hitting the corner cases for the generation of Find<Package>.cmake files.

Take the protobuf package example.
From a CMake point of view, protobuf and protoc are one package, that comes with libraries and the protoc executable. There is just one protobuf-config.cmake. I guess the reason why there are two packages is cross compiliation? That makes it even more complicated, but the above also holds for other packages which declare CMake helper functions.

Anyways, I think that packages which are meant to be consumed by CMake (such as Protobuf) should be consumed through the CMake files which they produce. Not by files which are create by an external tool like Conan. It will be very hard for Conan to get it right. Because even if you allow extra include files, you can never make sure that they will work correctly and that Conan will declare all targets, variables, functions, ... that are needed to ensure the correct usage of the extra files.
In the Protobuf case, some CMake functions that it offers (in this case protobuf_generate (for protobuf 3.8.0)) are even declared in the protobuf-config.cmake file, wich you won't be able to include, because itself includes and declares all the build targets.

protobuf-config.cmake

# Imported targets
include("${CMAKE_CURRENT_LIST_DIR}/protobuf-targets.cmake")

function(protobuf_generate)
...

Have you considered that the Conan generated "FindProtobuf.cmake" could just do an include(${PackageRoot}/cmake/protobuf-config.cmake and you can opt-in in your recipe to use the natively produced files instead of Conan generated ones.
Something like self.cpp_info.use_cmake_config = "cmake/protobuf-config.cmake"? In that case, Conan won't generate any targets related to protobuf, but just reuse the already existing functionality.

The CMake generator as it is, is a big help for packages not build with CMake, or not properly installable with CMake.
But for all those projects that are perfectly installable with CMake, I would always prefer to consume them through their provided Config files.
For me it feels wrong to add some workaround, add more variables, ... just so that Conan can build up a file, which CMake creates automatically.

[lasote]

Anyways, I think that packages which are meant to be consumed by CMake (such as Protobuf) should be consumed through the CMake files which they produce.

Yes to the files containing functions/macros. No to the config files. Why? Because the information on how to reuse the package belongs to the package manager, not to the library.

It will be very hard for Conan to get it right.

For your own packages, it is totally ok if you want to trust the config generated files, for the central repository I could argue the opposite: it is not going to work using the config provided files (targets without transitive dependencies and incomplete targets due to the information Conan has about the dep graph: settings, options etc) and it is against the principles of how should it be.

That's why I said if we have to trick the library, let's do this, but we cannot break our own model as a package manager. Is it doable? I think it is, let's try and see how it works.

[KerstinKeller]

No to the config files. Why? Because the information on how to reuse the package belongs to the package manager, not to the library.

From my opion: the information how to find a package, where it is located, what configurations it has, if it is compatible to another package, this information belongs to the package manager. I totally agree. But the information about what are it's header files, how to link it, ... that info should be part of the package, and intrinsic to the package. Unfortunately for C++ there is no standardized way to describe it, but pkgconfig files are one way, <package>-config.cmake files another one. If you are trying to keep track of all these infos, you are writing a build system, not a package manager.

For your own packages, it is totally ok if you want to trust the config generated files, for the central repository I could argue the opposite: it is not going to work using the config provided files (targets without transitive dependencies and incomplete targets due to the information Conan has about the dep graph: settings, options etc) and it is against the principles of how should it be.

I never said that they should be used. But I meant that, if and only if a package was written in such a way that it transparently finds other packages through find_package and links other libraries with target_link_libraries using only imported targets, then I think that you should give a recipe at least the option to make use of that.

But I didn't mean to hijack this thread, it just reflects my opinion, but please consider that you might run into more problems...

[lasote]

No problem, feedback is always welcome! thanks.

[lasote]

We have been discussing:

• Add cpp model cpp_info.build_modules: A list of complete relative paths to module files for the consumer build systems: e.j protobuf/macros.cmake or whatever/mystuff.props.
• That field will have a different meaning of the builddirs. In that case the generators are adjusting the module path to allow include the files. In this case, the generators will include the files automatically.
• All this has to be first a POC to see if extracting the mentioned macros from protobuf works nicely.

Refs:
https://github.com/Kitware/CMake/blob/master/Modules/FindProtobuf.cmake
https://cmake.org/cmake/help/v3.0/command/include.html