Conan: Consider creating a CPack "backend" for conan packages

[mcraveiro]

Hi,

As you are probably aware, CMake has a "companion tool" called CPack[1] which generates debs, rpms and so forth. Many projects use CMake to generate these, albeit they tend to generate packages that are not 100% compliant with the distribution policies. Anyways, to the point: I was wondering if it is possible to extend CPack to support the native package format used by conan, such that one could simply execute CPack and out comes a package ready to be uploaded to conan servers. Now, please take this suggestion with a big grain of salt since I have no idea about: a) the conan package format b) the complexities of adding a new CPack backend.

However, I do think this is worth considering because if it is totally trivial to generate a package for conan and if the generation process is integrated with CMake, presumably the number of people willing to generate the packages will increase.

When I have some spare time - really not sure when! - I am contemplating having a quick look at CPack code and seeing how hard it would be to add a new package format.

Thanks for your time.

[1] https://cmake.org/Wiki/CMake:Packaging_With_CPack

[lasote]

Its another thing to investigate. Thanks!

[monsdar]

This would also allow to release deb/rpm/... packages to end-users while releasing conan packages for automatic downstream builds. Could be very useful for Go Pipelines and the like.

[memsharded]

I have been having a look at this idea, trying it, (again, it is not the first time I use CPack), and I think this feature is not clear at all:

• CPack usage doesn't seem very extended: 10 followers, 105 questions (total) in the StackOverflow "cpack" tag. I definitely not see modifying/forking CPack to account for conan packages.
• CPack doesn't seem really designed to be run as an independent process, but really connected to cmake build scripts. I have tried to run it isolated, and I haven't been able to, the docs are very scarse or old, few online help... Together with cmake makes much more sense and works much better.
• As it works within the cmake build system, it is very highly coupled to it, thus only useful for CMake consumers (which are not all package consumers)

The conan package structure is very simple, just a conanfile.py and a folder with files. Those files will be typically arranged in subfolders:

• include: for the headers
• lib: .lib, .a, .so. dylib
• bin: .exe, linux binaries, .dll

The conanfile contains package_info() method that can define flags (include paths, c++ or linker flags, etc), but that is all. That info will be translated to respective cmake, xcode, vsprops... files by the respective conan generators.

Furthermore, consider the use case for creating a deb, win-installer, rpm installer. It is probable that you want to do it for your own project, which doesn't necessarily is a conan package. The workflow could be:

1. I want to create an application that will be distributed with a Win installer, and that depends on, lets say, the Poco libraries.
2. I create my project, define my build system, write my code files. I write my conanfile.txt, with a [requires] section containing the Poco/1.7.2@lasote/stable dependency.
3. I run conan install so the dependencies are retrieved. It will generate conanbuildinfo.cmake or whatever other files I need for my build system.
4. I build my application
5. Now, I want to bundle my app with the installer. My app details are NOT managed by conan, you could have created several executables, with several dlls, or whatever. You know that you have to bundle them, together maybe with the Poco shared libraries and some other files as licenses, and so on.
6. The Poco and other transitive dependencies shared libraries can be already retrieved by an [imports] section in the conanfile.txt, or you already have their paths in the conanbuildinfo.cmake

So it seems that creating the installer is up to the final user. I have not used CPack in production, but I have used deb, rpm, exe (NSIS) packages, and typically you have to deal with their configuration, customize names, icons, licenses... There is very little that conan (or a package manager in general) can automate here, without introducing another layer of abstraction, that could be initially buggy, undocumented, etc. A study of other package managers (for other languages too), we can see that no other handles this task, and it seems for good reasons. It sounds more like a totally new and independent project by its own: a new tool for creating installers/packages for all systems. I'd love to have such a tool :)

In any case, if consumers are already using CMake in their own project, it is quite easy for them to add CPack functionality, and packaging/bundling dependencies should be completely straightforward with the variables defined in conanbuildinfo.cmake.

So I vote for closing this feature request, nothing really relevant that we could do here. What do you think @mcraveiro @monsdar ?

[CausticYarn]

I am trying to build an automated workflow (in Jenkins) that is somewhat related to this question. I have libraries with which I want to create both a Conan package and an RPM or Windows installer (depending on the platform). The CMake file has instructions that can be used by CPack to automatically generate the package after the CMake build has finished.

I could always build the Conan package first, then use the source to build the installer, but I was trying to come up with a good way to build once and create both packages. I could put the CPack command into the conanfile.py build instructions, and just reach into the Conan build folder if I know the path and yank out the RPMs. Or possibly I could include the installer in the Conan package, maybe in a 'dist' folder and then possibly use a custom generator to install or upload it.

I am curious if anyone has any suggestions or has implemented any kind of similar workflow.

[memsharded]

I have been playing recently with "re-packaging" recipes, in my case, for releasing with conan runtimes (exes + dlls + data). Maybe the same concept could be applied to create CPack installations:

• Have a "packaging" recipe, with a CMakeLists fo CPack stuff
• The recipe can imports() all the exes, dlls, and data from the dependencies
• Dependencies can be build_requires

Nevertheless, I have little idea of how CPack works, so we would need time to investigate this. I think that conan users doing this, they are doing as a post-conan step, just use imports() to get the files they want from conan cache to a user folder, and CPack them from there with their scripts.

[mcraveiro]

Hi @memsharded, @CausticYarn

My original intention with this ticket was to explore on how easy it was to extend CPack to support conan packaging. I personally use CPack at the moment to generate Windows packages and Debs with very little effort so its a win for me. I was hoping I could just add another target (a one-liner) and out would come conan packages. No need to even know about conan (in the same way I don't really know much about Debs or WiX at present).

From reading all the posts on this discussion, it seems like this is not feasible. I think we probably should close this ticket and open up new ones regarding the different ideas people are having, which are not directly connected with the initial suggestion. Unless anyone disagrees.

A thank you to everyone who contributed.

[memsharded]

Yes, this ticket has been here for a long time. It doesn't seem to have a clear good fit/need.
If anything, it could make sense to incorporate CPack to some of the "pack" functionality that @sigmoidal is contributing in: https://github.com/conan-io/conan/pull/2077. This still needs some work, but it is a nice place to follow conversation if necessary. Many thanks!