Conan: How to prevent building dependency package whose options/settings differ?

[ashinan]

I have two packages: fmt and spdlog.
fmt provides shared or static library and spdlog is header only library which depends on fmt.

Here are my recipes:

fmt:

from conans import ConanFile, CMake
from conans.tools import get

class FmtConan(ConanFile):
    name    = "fmt"
    version = "4.0.0"
    license = "BSD-2-clause"
    url     = "https://github.com/xylosper/conan-packages"
    settings = "os", "arch", "compiler", "build_type"
    description = "Small, safe and fast formatting library"
    build_policy = "missing"
    options = {"shared": [True, False]}
    default_options = "shared=True"

    def source(self):
        get("https://github.com/fmtlib/%s/releases/download/%s/%s-%s.zip" % (self.name, self.version, self.name, self.version))

    @property
    def src_dir(self):
        return "%s-%s" % (self.name, self.version)

    def build(self):
        cmake = CMake(self)
        defs = {
            "BUILD_SHARED_LIBS": self.options.shared,
            "FMT_TEST": False,
            "FMT_DOCS": False,
            "FMT_INSTALL": True,
            "CMAKE_INSTALL_PREFIX": self.package_folder,
            "CMAKE_POSITION_INDEPENDENT_CODE": self.settings.os != "Windows"
        }
        cmake.configure(defs=defs, source_dir=self.src_dir)
        cmake.install()

    def package(self):
        self.copy("LICENSE.rst", dst="", src=self.src_dir)

    def package_info(self):
        if self.options.shared:
            self.cpp_info.defines.append("FMT_SHARED")
        self.cpp_info.libs.append("fmt")

spdlog:

from conans import ConanFile
from conans.tools import get

class SpdlogConan(ConanFile):
    name    = "spdlog"
    version = "0.14.0"
    author  = "xylosper ([email protected])"
    url     = "http://github.com/xylosper/conan-packages"
    license = "MIT"
    description = "Super fast C++ logging library"
    generators = "cmake"
    build_policy = "missing"

    def requirements(self):
        self.requires("fmt/[>=4.0]@xylosper/stable")

    def source(self):
        get("https://github.com/gabime/%s/archive/v%s.zip" % (self.name, self.version))

    def package(self):
        src = "%s-%s" % (self.name, self.version)
        self.copy("*.h", dst="include", src="%s/include" % src, keep_path=True)
        self.copy("LICENSE", dst="", src=src)

    def package_info(self):
        self.cpp_info.defines.append("SPDLOG_FMT_EXTERNAL")

    def package_id(self):
        self.info.header_only()

Although spdlog depends on fmt, it doesn't matter what kind of fmt is installed because spdlog is header-only library. However, when I have only fmt with shared=False option, installing spdlog invokes building new fmt. How can I prevent this?

[lasote]

I recommend you to read this guide: http://docs.conan.io/en/latest/howtos/define_abi_compatibility.html

You can declare how a require of a package will affect to the package id itself, in this case I think it could fix:

def package_id(self):
    self.info.requires["fmt"].unrelated_mode()

No change in fmt will affect to sdplog.

[memsharded]

@lasote, the current package_id() implementation, already do this, check the header_only() call:

    def header_only(self):
        self.settings.clear()
        self.options.clear()
        self.requires.unrelated_mode()

So I think the issue could be somewhere else. Could you please provide @xylosper the steps to reproduce, and the command lines used? I will have a look.

BTW, the FMT lib has a header-only mode, that might be interesting too, you can check my recipe if you are interested: https://github.com/memsharded/conan-fmt/blob/testing/conanfile.py

[ashinan]

@lasote I tried your suggestion but nothing changed.

@memsharded Here's what I've tried:

1. install fmt with shared=False option in directory which contains conanfile.py for fmt

> conan create xylosper/stable -o shared=False

2.1.1 export spdlog in directory which contains conanfile.py for spdlog

> conan export xylosper/stable

2.1.2 run conan install with next conanfile.txt:

[requires]
spdlog/[>=0.14]@xylosper/stable

or

2.2 install spdlog directly in directory which contains conanfile.py for spdlog

> conan create xylosper/stable

In both case of 2.1 and 2.2, building process of fmt package is triggered although fmt is already installed.

FYI, I'm using 0.25.1 version on Windows 10 64bit.

And, I've already seen your recipe before but I do want to use header only library as less as possible.

[memsharded]

Are you sure you created the fmt in shared mode? Try:

$ conan create xylosper/stable -o fmt:shared=False

Try to print or to log the options inside the recipe, to really make sure you are building the shared library

Also, if you type:

$ conan search fmt/4.0.0@xylosper/stable

You should see your binary packages configuration, please check it too.

[ashinan]

@memsharded

Are you sure you created the fmt in shared mode?

You mean static mode, right? (shared=False means not using shared library here)

Here's the result of conan search fmt/4.0.0@xylosper/stable:

> conan search fmt/4.0.0@xylosper/stable
Existing packages for recipe fmt/4.0.0@xylosper/stable:

    Package_ID: 63da998e3642b50bee33f4449826b2d623661505
        [options]
            shared: False
        [settings]
            arch: x86_64
            build_type: Release
            compiler: Visual Studio
            compiler.runtime: MD
            compiler.version: 14
            os: Windows
        outdated from recipe: False

I tried conan create xylosper/stable -o fmt:shared=False and again, the result of conan search fmt/4.0.0@xylosper/stable:

> conan search fmt/4.0.0@xylosper/stable
Existing packages for recipe fmt/4.0.0@xylosper/stable:

    Package_ID: 63da998e3642b50bee33f4449826b2d623661505
        [options]
            shared: False
        [settings]
            arch: x86_64
            build_type: Release
            compiler: Visual Studio
            compiler.runtime: MD
            compiler.version: 14
            os: Windows
        outdated from recipe: False

As you can see, nothing changed.

And still conan install triggers building fmt:

> conan install
Version range '>=0.14' required by 'None' resolved to 'spdlog/0.14.0@xylosper/stable'
Version range '>=4.0' required by 'spdlog/0.14.0@xylosper/stable' resolved to 'fmt/4.0.0@xylosper/stable'
Requirements
    fmt/4.0.0@xylosper/stable from local
    spdlog/0.14.0@xylosper/stable from local
Packages
    fmt/4.0.0@xylosper/stable:c85f9b402dd4d46acdf074e1c63b768a41181d7a
    spdlog/0.14.0@xylosper/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9

fmt/4.0.0@xylosper/stable: Building package from source as defined by build_policy='missing'
fmt/4.0.0@xylosper/stable: Building your package in C:\Dropbox\conan\data\fmt\4.0.0\xylosper\stable\build\c85f9b402dd4d46acdf074e1c63b768a41181d7a
fmt/4.0.0@xylosper/stable: Copying sources to build folder
You pressed Ctrl+C!

You can notice that the hash of required fmt package(c85f9b402dd4d46acdf074e1c63b768a41181d7a) differs from that of installed one(63da998e3642b50bee33f4449826b2d623661505).

Here's the result of conan search fmt/4.0.0@xylosper/stable after installing fmt with no option:

> conan search fmt/4.0.0@xylosper/stable
Existing packages for recipe fmt/4.0.0@xylosper/stable:

    Package_ID: 63da998e3642b50bee33f4449826b2d623661505
        [options]
            shared: False
        [settings]
            arch: x86_64
            build_type: Release
            compiler: Visual Studio
            compiler.runtime: MD
            compiler.version: 14
            os: Windows
        outdated from recipe: False

    Package_ID: c85f9b402dd4d46acdf074e1c63b768a41181d7a
        [options]
            shared: True
        [settings]
            arch: x86_64
            build_type: Release
            compiler: Visual Studio
            compiler.runtime: MD
            compiler.version: 14
            os: Windows
        outdated from recipe: False

As you can see here, the hash for required fmt coincides that of fmt with no option(shared=True by default).

[ashinan]

By the way, is it possible to specify compatible options/settings for dependencies as well as ignore it?

Maybe a package should be always linked against dynamic library for legal issue or some kind of implementation limitations(for instance, a library built by gcc in msys can be dynamically linked against MSVC binary if def file is provided but static link is not possible).

Generally, with A package which provides some options to turn on or off features, B package which depends on A may require specific feature which can be turned on an option of A package. Or, C package which also depends on A may conflict with some features in package A and in that case, C package should require package A whose conflicting features are turned off.

Can conan handle this kind of dependency?

If conan allows to specify options/settings for deps, my problem would be solved by specifying all possible options(shared=[True, False], for instance).

[lasote]

About your last question, in a recipe (A) you can force the valid options of its dependencies (B) by using the configure method:

def configure(self):
      self.options["B"].shared = False

In this case, if another package in your project is requiring B as shared Conan will fail in the installation because of the conflict, unless one of the B requires had been specified as private: http://conanio.readthedocs.io/en/latest/reference/conanfile/methods.html?highlight=private#requirements

In the configure you could even raise if the options for our dependency is not valid:

def configure(self):
         if self.options["B"].shared:
             raise Exception("Lib A only works with static B")

I'll try to reproduce the main issue.

[lasote]

The problem with the packages is that fmt package has the shared option defaulted to shared=True.
If you generate the shared=False and then you install sdplog that requires fmt but without specifying any option, it will take the default, and it will build again the fmt.

[ashinan]

@lasote Thank you for your answers for my questions.

The problem with the packages is that fmt package has the shared option defaulted to shared=True.
If you generate the shared=False and then you install sdplog that requires fmt but without specifying any option, it will take the default, and it will build again the fmt.

Yes, I can see that from the hash key of required fmt package. So, is it what you suggest that I have to run conan install with -o fmt:shared=False even if my conanfile.txt does not require fmt package?(fmt is not direct dependency of my conanfile.txt) If that is the design of conan, I'll just follow it.

[memsharded]

Hi @xylosper

Some comments:

• You can also specify an [options] section in your conanfile.txt, so you don't have to type it all the time.
• Your conanfile is not "declaring" fmt package, but it does "require" it, transitively, so you don't see it in your conanfile.txt, but it is a requirement.
• The design and command line makes sense IMHO. If you are ok with the defaults of spdlog, you just install it, and the fact that you had another binary for fmt is irrelevant, as if you had another version for fmt. If you want to override the spdlog defaults, and you want to say: "ey, I want to link with fmt statically", then the command line conan install -o fmt:shared=False sounds good to me:
• Finally, if that is not the default you want, you can always change it easily in spdlog recipe.

[ashinan]

@memsharded

Thank you for kind answers.