Conan: [build system] [cmake] Passing a preprocessor definition from conanfile. Consumer recipe

[chreniuc]

To help us debug your issue please explain:

• [x] I've read the CONTRIBUTING guide.
• [x] I've specified the Conan version, operating system version and any tool that can be relevant.
• [x] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

Short version:
I want to pass from the conanfile(consumer recipe, building an executable) a preprocessor definition which will be used in the building of the current project(will be used in build() method), just like cpp_info.defines does in the package_info method.

Long version:
Let's say I have a library libA that has those options :

"build_time": [True, False],
"build_file": [True, False],
"build_settings": [True, False],
"build_log": [True, False],
"build_email": [True, False]

In the test package(or any consumer recipe that is building an executable) i want to test the package based on those options:

// main.cpp
.....
#ifdef build_time
 do_this_time();
#endif

#ifdef build_log
 do_this_log();
#endif
...

Those are not unit tests, they are some calls that demonstrate that the library is working correctly. How can i pass those preprocessor definitions from the conanfile of the consumer recipe?

I don't want to pass them from the recipe of libA(using self.cpp_info.defines) or using cmake.defines["CMAKE_CXX_FLAGS"]="build_time" in the consumer recipe, or even cmake.defines["build_time""]="build_time" in the consumer recipe and then in CMakeLists.txt : if(DEFINED build_time) add_definition(build_time) endif().

I am wondering if there is way to pass them as we do it with self.cpp_info.defines in the package_info method, but in the build() method.

If there isn't, would it be helpful to have it? Or it doesn't make any sense to have this feature.

I'm thinking of something like this:

def build():
    ...
    cmake.cpp_defines.append("build_log");
    cmake.configure()
    cmake.build()
....

[memsharded]

No, it is not possible. If you check from the docs (https://docs.conan.io/en/latest/reference/commands/consumer/install.html) the evaluation order of the recipes is:

build()
package()
package_info()

The package_info() method is the information provided to the consumers of the package. It is not called until the package is build, because actually it could use some information that is generated at build time. So it is not something that could be implemented, it is impossible by definition.

I am missing some motivation, why do you want to put cmake.cpp_defines.append("build_log") in package_info() vs putting cmake.definitions["..."]=... in the build() method? What does it really matters for your use case? As a general rule, conan is not a build systems, and does not call directly the compilers, everything is done via the build systems, so conan cannot provide preprocessor directives to the compilers if not via the build system.

Please tell a bit more about your use case. Thanks!

[chreniuc]

You misunderstood me. I don't want cmake.cpp_defines.append("build_log") in package_info(), I want it in method build(). I was just giving an example of how something already exists in the package_info method. This is a consumer recipe, it has only the build() method. A test package.

class test_package_libA(ConanFile):
    settings = "os", "compiler", "build_type", "arch"
    generators = "cmake"

    def build(self):
        cmake = CMake(self)
        cmake.parallel = True
        cmake.verbose = True
        if self.options["libA"].build_log == True:
            cmake.cpp_defines.append("build_log")
        cmake.configure()
        cmake.build()

 def test(self):
        if not tools.cross_building(self.settings):
            os.chdir("bin")
            self.run(".%stest_package" % os.sep)

Or another case where it isn't a test package, only a consumer app:

class test_package_libA(ConanFile):
    settings = "os", "compiler", "build_type", "arch"
    generators = "cmake"
    requires = ("libA/1.0.0@user/stable")

    def build(self):
        cmake = CMake(self)
        cmake.parallel = True
        cmake.verbose = True
        if self.options["libA"].build_log == True:
            cmake.cpp_defines.append("build_log")
        cmake.configure()
        cmake.build()

Note: I also edited the first post, to explicitly say that i want it in the build() method

[memsharded]

Thanks for clarifying. So if I understood correctly the problem with:

class test_package_libA(ConanFile):
    settings = "os", "compiler", "build_type", "arch"
    generators = "cmake"
    requires = ("libA/1.0.0@user/stable")

    def build(self):
        cmake = CMake(self)
        cmake.parallel = True
        cmake.verbose = True
        cmake.definitions["BUILD_LOG"] = self.options["libA"].build_log
        cmake.configure()
        cmake.build()

Is that you need to add to your CMakeLists.txt the if(DEFINED build_time) add_definition(build_time) endif(), and you would like not to have to do that.

Unfortunately, it is not possible to do this. Conan does not do any "magic" with the build system, it basically calls it, or wraps it execution. There is no way (even without conan) that such definition can be "injected" in a CMake build, so it doesn't seem a feature that conan could implement.

[chreniuc]

@memsharded Thanks :)
I still wonder, if this can be done in the package_info method(this will be passed to the project that will be using this package), why couldn't be done in the build method for the current project?

Having this recipe:

class libA_recipe_conan(ConanFile):
    name = "libA"
    version = "1.0.0"
    ....
    def package_info():
        self.cpp_info.defines = ["build_log", "build_time"]

And this consumer recipe:

class consumer_package(ConanFile):
    name="executable"
    settings = "os", "compiler", "build_type", "arch"
    generators = "cmake"
    requires = ("libA/1.0.0@user/stable")

    def build(self):
        cmake = CMake(self)
        cmake.parallel = True
        cmake.verbose = True
        cmake.configure()
        cmake.build()

build_log and build_time will be passed as preprocessor definitions(because they were defined in the package_info method in libA), when building this consumer recipe(executable).

Now what happens behind the scenes of this line self.cpp_info.defines = ["build_log", "build_time"] from the libA recipe? How does conan pass those preprocessor definitions to the compiler and why we couldn't apply the same thing in the build() method for the current build?

I'm sorry that i keep insisting, but it seems to me that the line self.cpp_info.defines = ["build_log", "build_time"] in the package_info() does exactly what you said conan couldn't do :

There is no way (even without conan) that such definition can be "injected" in a CMake build, so it doesn't seem a feature that conan could implement.

[memsharded]

Hi @chreniuc

Now I understand what you mean. The separation of the generators/build-helpers is the reason for the current behavior: generators estrictly generated information about the dependencies, not the current package/consumer, and build-helpers (like CMake) dealt only with the current package/consumer build system invocation, but not the dependencies.

The build-helpers doesn't define (so far) any means to modify the build besides the strictly required invocation, translating conan settings (os, compiler) to the correspondent cmake command line arguments or flags (like -s arch=x86 might translate to -m32).

So conan doesn't provide a high level abstraction over the current build, besides that. And the information that comes in the generator files is from dependencies only. So right now, you should use some build-system modifications.

The good news, is that we are aware of this demand. Users are asking conan for such high-level abstraction over the build systems and we are going to provide it. There are still many unknowns, and it is still not a top priority, so it might take a while.

Please stay tuned, I am renaming the title and labeling accordingly. Thanks for your feedback!

[mjschuster]

I recently stumbled upon the same issue ... are there any news on this?

Could preprocessor definitions maybe just be passed via the CMAKE_CXX_FLAGS?
(see https://stackoverflow.com/a/55564298 and https://stackoverflow.com/a/52179619)
As far as I can see, the CMAKE_CXX_FLAGS are already optionally used to pass architecture flags in https://github.com/conan-io/conan/blob/master/conans/client/build/cmake_flags.py#L320

Would it be an easy solution to have another cmake-build-helper function or parameter to pass additional CMAKE_CXX_FLAGS or, having a nicer interface, a dictionary of preprocessor macros that are converted to CMAKE_CXX_FLAGS?

If that could work, maybe the issue doesn't need to be labelled / seen as "complex: high" and could easily be resolved? Or am I missing some potential problems here?

Thanks in advance for any help or comments here!

[egendron93]

I recently stumbled upon the same issue ... are there any news on this?

Could preprocessor definitions maybe just be passed via the CMAKE_CXX_FLAGS?
(see https://stackoverflow.com/a/55564298 and https://stackoverflow.com/a/52179619)
As far as I can see, the CMAKE_CXX_FLAGS are already optionally used to pass architecture flags in https://github.com/conan-io/conan/blob/master/conans/client/build/cmake_flags.py#L320

Would it be an easy solution to have another cmake-build-helper function or parameter to pass additional CMAKE_CXX_FLAGS or, having a nicer interface, a dictionary of preprocessor macros that are converted to CMAKE_CXX_FLAGS?

If that could work, maybe the issue doesn't need to be labelled / seen as "complex: high" and could easily be resolved? Or am I missing some potential problems here?

Thanks in advance for any help or comments here!

I believe the issue with using CMAKE_CXX_FLAGS is the fact that it is a global environment variable and against the effective cmake best practices. I also think that it makes the conan developers job more difficult by providing a way for the user to inject anything into the build commands without affecting the package id.

I'm also struggling with a proper solution to this issue. Like @memsharded said there is no cmake provided interface to add compile definitions to a cmake target without adding to the cmakes source code itself.

The ABI implications of this problem seem to be overwhelming. For example, if a recipe uses some preprocessor definitions that affect public facing code and then the consumer project uses that public facing code and compiles without the same preprocessor definitions undefined behavior may occur.

I am wondering if there is a recommended solution to handle this prior to Conan 2.0?

[memsharded]

The ABI implications of this problem seem to be overwhelming. For example, if a recipe uses some preprocessor definitions that affect public facing code and then the consumer project uses that public facing code and compiles without the same preprocessor definitions undefined behavior may occur.

That is what the package_info() method is for, check https://docs.conan.io/en/latest/reference/conanfile/methods.html#package-info. If a package needs its consumer to enable some preprocessor definition, they should be declared:

def package_info(self):
     self.cpp_info.defines = ["MY_PREPROCESSOR_DEFINITION"]

As commented above, there is no magic in the build system that Conan could exploit, so the only way as today is:

• Define some option to the recipe, to allow pass configuration that change the build. It can be even generic, not validated input with "ANY"
• Options by definition affect the package-ID, so we are good. You can also process the granularity of binaries you want based on that option inputs (you can use the package_id() method for that)
• Pass this option, or compute in the recipe the necessary flags, and pass them to the underlying build system
• Let the build system know and process those inputs.

Yes, some general flags mechanism can be used, like some kind of CMAKE_CXX_FLAGS, but that might be against best practices. In general, it seems to work better to model the variability of the binaries explicitly (either with settings or options). For example if you directly manage the optimization level, instead of trying to pass some CXX_FLAGS, you model optimization = [1, 2, 3, 4, 5] which depending on the scope could be in the settings (maybe a subsetting of compiler) or as an option. Then you let the recipes and/or the build helpers to translate that information to respective /OX flags or whatever they mean for the different compilers and systems.

[mjschuster]

Thank you for your answers. I'm not sure if we are talking about the same thing here. To clarify my comment:
I understand and agree that Conan settings or options would in most cases be the best way to pass flags into Conan and that they should be exported via package_info if required by consumer packages.
My question, however, is on how to pass C++ preprocessor definitions from Conan into the CMake-based build without having to modify the CMakeLists.txt?

[memsharded]

My question, however, is on how to pass C++ preprocessor definitions from Conan into the CMake-based build without having to modify the CMakeLists.txt?

That is the point, CMake doesn't have any magic mechanism that allows to do so. It is not intrisically a Conan thing, as pointed here https://stackoverflow.com/questions/8564337/how-to-define-a-c-preprocessor-macro-through-the-command-line-with-cmake the options are:

• Mess with the environment variables (system dependent, requires hacking and adaptation to every different system) to inject it
• Pass it as global CXX_FLAGS -D CMAKE_CXX_FLAGS=/DMY_MACRO=1 (global to all CMakeLists.txt script, not "modern" cmake)
• Modify the CMakeLists.txt

I was summarizing, just in case, how different information can be brought to the build() method from the outside if necessary, but in the boundary with CMake, the above is what can be done.

[egendron93]

That is what the package_info() method is for, check https://docs.conan.io/en/latest/reference/conanfile/methods.html#package-info. If a package needs its consumer to enable some preprocessor definition, they should be declared:

def package_info(self):
     self.cpp_info.defines = ["MY_PREPROCESSOR_DEFINITION"]

As commented above, there is no magic in the build system that Conan could exploit, so the only way as today is:

* Define some option to the recipe, to allow pass configuration that change the build. It can be even generic, not validated input with "ANY"

* Options by definition affect the package-ID, so we are good. You can also process the granularity of binaries you want based on that option inputs (you can use the `package_id()` method for that)

* Pass this option, or compute in the recipe the necessary flags, and pass them to the underlying build system

* Let the build system know and process those inputs.

@memsharded A variation of this approach is what I had planned on using for the recipes that I manage. Hearing it come from you is more reassuring that I'm headed in the right direction.

One of the main issues I've run into is for package recipes that I do not manage. The boost recipe in the conan center provides a limited amount of options compared to all the actual preprocessor definitions available throughout the boost libraries. For example BOOST_BEAST_USE_STD_STRING_VIEW is not represented as a package option. The boost recipe provides an additional_b2_flags option that uses the not validated "ANY" input. This is where I set the beast string view definition. The definitions added to the additional_b2_flags option do not get added to the cpp_defines member, therefore will not be accessible to consuming projects.

I guess this is more of a recipe specific design issue and I should make an issue for the boost recipe. Is creating the issue the next move or am I missing something?