Conan: [bug] cmake generator does not write build requirements info when using dual profiles

[theodelrieu]

Environment Details (include every applicable attribute)

• Operating System+version: macOS Big Sur
• Conan version: develop (1.33.0-dev)
• Python version: 3.8.5

Steps to reproduce (Include if Applicable)

With this conanfile.py:

from conans import ConanFile, CMake, tools


class BugConan(ConanFile):
    name = "bug"
    version = "0.1"
    license = "<Put the package license here>"
    author = "<Put your name here> <And your email here>"
    url = "<Package recipe repository url here, for issues about the package>"
    description = "<Description of Bug here>"
    topics = ("<Put some tag here>", "<here>", "<and here>")
    settings = "os", "compiler", "build_type", "arch"
    options = {"shared": [True, False], "fPIC": [True, False]}
    default_options = {"shared": False, "fPIC": True}
    generators = "cmake"

    def config_options(self):
        if self.settings.os == "Windows":
            del self.options.fPIC

    def build_requirements(self):
        self.build_requires("doctest/2.3.8")

    def source(self):
        pass

    def build(self):
        pass

    def package(self):
        pass

    def package_info(self):
        pass

Run conan install . --profile:host=default --profile:build=default

The generated conanbuildinfo.cmake does not contain doctest targets, but it does when conan install . --profile=default is used.

[theodelrieu]

This is because of the line all_flags = cmake_dependencies(dependencies=self.deps_build_info.deps).

self.deps_build_info is empty when using dual profiles

[theodelrieu]

I've added logs in `model/conan_generator.py:

With --profile=default:

self._deps_build_info:
self._deps_env_info
self._env_info None
self._deps_user_info DepsUserInfo(, {'doctest': {}})
self._user_info_build None

With dual profiles:

self._deps_build_info:
self._deps_env_info
self._env_info None
self._deps_user_info DepsUserInfo(, {})
self._user_info_build DepsUserInfo(, {'doctest': {}})

Seems that generators do not support user_info_build

[jgsogo]

Hi, @theodelrieu

This is probably a limitation of the cmake generator. With the current syntax there is no way to differentiate between a package coming from _host_ context and the same package from _build_ context if both have the same name (i.e.: protobuf). It would require variables like CONAN_USER_<PKG-NAME>_<VAR-NAME> and CONAN_USER_<CONTEXT>_<PKG-NAME>_<VAR-NAME>.

Doable, but not sure if we want to go that way. If the preferred generator cmake_find_package[_multi] doesn't provide this functionality, IMHO we shouldn't add it to a generator that might be removed in Conan v2.0.... or at least we shouldn't add it before implementing it in the recommended generator.

But we totally need to unblock this issue, there are many related to this same problem or closer ones. I really think there is something missing in CMake itself, but we need to bet big and offer a way to work around current limitations. I'm thinking about opening an RFC with some risky proposal.

[theodelrieu]

If the preferred generator cmake_find_package[_multi] doesn't provide this functionality

Indeed, the Finddoctest.cmake is only generated when --profile default is used.

I don't know how I can workaround this issue, I don't want to revert to os_build/arch_build but I might not have a choice here...

[madebr]

@theodelrieu
Why do you add doctest as a build requirement?
doctest is a header-only library. Shouldn't it be a requirement?

[jgsogo]

Hi, @theodelrieu

We've been talking about this issue (and related ones) and the path will be as follows:

• Conan will (1.34?) generate a file with all the information from cpp_info in plain CMake format. This file will contain the information as plain as possible. It will be hard to define the name of all the variables, but the file will contain information from all the packages in _host_ and _build_ context that are needed to build the package. (Note.- Maybe it is not only one file, but one per requirement, TBD).
• This file will be accessible only if using generator CMakeDeps (in fact, this generator will use that file/s to populate the targets). Generator CMakeDeps will be the recommended (only?) one in Conan 2.0 and we encourage everyone to try it and start using it to smooth the migration.
• User can include that file from their CMakeLists.txt and will have access to all these variables.
• Eventually, Conan will provide some abstraction layers on top of this file to provide targets for the components coming from the _build_ context (protoc, doxygen,...). And some convenient CMake functions (find_package_build) or overrides (new definition of find_package) that will help with these scenarios and make it possible some transparent integration of Conan with existing CMake files.

[blackliner]

In my case I need CapnProto to be compiled for the build system, generate sources, and then be available as find_package for building the host binaries.

Is there any workaround for cross compilation right now?

EDIT: maybe a two step conan install, so instead of
conan install .. --profile:build build_profile --profile:host host_profile
we split it up and call
conan install package_A package_B --profile build_profile
--> copy away the files you need...
conan install package_C package_D --profile host_profile
--> ... and restore them again
but for that I would need to evaluate the conanfile.py manually I guess 😞

[blackliner]

As seen below, --profile:build just completely skips the generation of any *.cmake files for packages listed as build_requirements. For cmake_paths, conan_paths.cmake does not contain any info about them as well.

For this conanfile.py:

    generators = "cmake_find_package"
    ...
    def build_requirements(self):
        self.build_requires("capnproto/0.8.0")

    def requirements(self):
        self.requires("catch2/2.13.3")

conan install .. --profile:build default:

conanfile.py (LumPDK/None): Applying build-requirement: capnproto/0.8.0
conanfile.py (LumPDK/None): Generator cmake_find_package created FindCatch2.cmake

conan install .. --profile:host default:

conanfile.py (LumPDK/None): Applying build-requirement: capnproto/0.8.0
conanfile.py (LumPDK/None): Generator cmake_find_package created FindCapnProto.cmake
conanfile.py (LumPDK/None): Generator cmake_find_package created FindCatch2.cmake

conan install .. --profile default:

conanfile.py (LumPDK/None): Applying build-requirement: capnproto/0.8.0
conanfile.py (LumPDK/None): Generator cmake_find_package created FindCapnProto.cmake
conanfile.py (LumPDK/None): Generator cmake_find_package created FindCatch2.cmake

[jgsogo]

@blackliner The way to go is to add capnproto to requires and build-requires:

    generators = "cmake_find_package"
    ...
    def build_requirements(self):
        self.build_requires("capnproto/0.8.0")

    def requirements(self):
        self.requires("capnproto/2.13.3")
        self.requires("catch2/2.13.3")

    def build(self):
        # All the environment provided by `capnproto` from the build context is available here.

and then use the two-profiles approach:

conan create conanfile.py --profile:host=host --profile:build=default

Conan will populate the environment before entering build() method with the information provided in the env_info of the capnproto recipe.

[blackliner]

Unfortunately we do not use conan to build our own package, we just use it to consume dependencies. I went for this solution/workaround for now:

def conan(args):
    if not (BASE_DIR / "conanfile.py").is_file():
        logging.info("Skipping conan step")
        return

    export_recipes(args)

    conan_command = [
        "conan",
        "install",
        "..",
        "--settings",
        "build_type=" + args.build_type,
        "--build=missing",
        "--build=acado",
    ]

    if args.conan_profile:
        conan_command.append(f"--profile={CALLER_DIR / args.conan_profile}")
    else:        
        if args.conan_build_profile:
            conan_command_build_profile = conan_command.copy()
            conan_command_build_profile.append(f"--profile={CALLER_DIR / args.conan_build_profile}")
            subprocess.check_call(conan_command_build_profile, cwd=BUILD_DIR)
            for cmake_file in BUILD_DIR.glob("*.cmake"):
                cmake_file.rename(cmake_file.with_suffix(".build_bak"))

            conan_command.append(f"--profile:build={CALLER_DIR / args.conan_build_profile}")

        if args.conan_host_profile:
            conan_command.append(f"--profile:host={CALLER_DIR / args.conan_host_profile}")

    logging.info("Conan command:")
    logging.info(" ".join(conan_command))

    subprocess.check_call(conan_command, cwd=BUILD_DIR)

    for cmake_bak_file in BUILD_DIR.glob("*.build_bak"):
        new_name = cmake_bak_file.with_suffix(".cmake")
        if not new_name.is_file():
            cmake_bak_file.rename(new_name)

So for now I save all the FindXXX.cmake from the build context and restore them later 🤷

[Hopobcn]

What about using force_host_context=True?
In my case, I'm experimenting with CMakeToolchain and two profiles. And unfortunately, I saw this issue when providing the same profile for build & host contexts.

If I add force_host_context=True:

    def build_requirements(self):
        if self.options.get_safe("build_tests"):
            self.build_requires("gtest/1.10.0", force_host_context=True)

I'm able to compile & run tests in the build method.