Conan: [Discussion] how to model GLIBC version for conan packages

[SSE4]

To help us debug your issue please explain:

• [ ] I've read the CONTRIBUTING guide.
• [ ] I've specified the Conan version, operating system version and any tool that can be relevant.
• [ ] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

/cc @rdeterre @bc-lee @elizagamedev

this is more generic version of https://github.com/conan-io/conan/issues/3963
related issues:
https://github.com/bincrafters/community/issues/296
https://github.com/bincrafters/community/issues/267

historically, conan didn't model glibc version, relying only on compiler version for binary compatibility checks.
this causes some issues in the past, as binaries compiled with the same compiler weren't compatible across various linux distributions.
for instance, if we compiled libraries or executables via GCC 4.8 on Ubuntu 12.04, these executables and libraries fail to run/link on CentOS 7 machines. same situation with other popular Linux distributions, like RHEL, Fedora, etc.

[danimtb]

We should discuss this issue once we face the cross-building model one as this might affect the current settings and have further implications in Linux packages.

[scddev]

I am facing a similar issue regarding libc when building packages for glibc and musl (e.g. Alpine). Both are Linux, x86_64 but linked against a different libc.

We modified settings.yml like this:

compiler:
  ...
  gcc:
    ...
    libc: [None, glibc, musl]
  clang:
    ..
    libc: [None, glibc, musl]

This does not cover the fact of different versions of glibc or musl but covers at least our use case.

[vector-of-bool]

To note: There is more than just the glibc version that defines the system ABI on Linux. Things like dynamic linking SONAMEs to system libraries present another dimension of whether a given ELF can be loaded correctly on a given system. Issues with differing system libraries were my trouble with Qt, even if there were compatible glibc versions.

I have a local fork of Conan at $job and I experimented with the introduction of two settings: os_family and os_distribution. The former is more general, with values like RedHat, Debian, SUSE and Windows, while the latter is more specific, with things like CentOS, RHEL, Ubuntu, and OpenSUSE. Another setting, os_release would cover the major revisions of a particular OS family/distribution, with values like 7 (for RHEL 7 and CentOS 7), 16.04, or 19.04.

A tuple of (os_family, os_release) is almost always enough to distinguish the basic ABI of the system, since the libc version and important system SONAMEs are stable within a given release of a distribution. In my tests, I was able to successfully generate Qt packages using this scheme that reliably rebuilt between a Ubuntu and CentOS system without them accidentally sharing binaries.

Of course, not all packages are so sensitive to these system attributes, and may want to "opt-in" to become more granular.

One case this design _does not_ consider is that a CentOS 6 package is _probably_ compatible with CentOS 7 (as long as the proper SONAMEs match), but I personally don't consider that a compelling enough feature to warrant additional complexity.

[rdeterre]

As a workaround for this issue, is there a conan install option we could use to prevent a machine from using prebuilt binaries from the remote server, and rebuild packages from source if it does not have a locally built package?

Simply using --build solves the issue, except that all the dependencies get rebuilt every time even when perfectly valid prebuilt binaries are already available locally. This is problematic for big dependencies like Boost.

[Minimonium]

I mean, always building the stuff from public servers maybe not that nice, since then why do we store it all then.
Indeed, it would be great to have a distribution + its version combination to define the whole distribution environment, not just glibc.
But also we could consider providing an alternative approach of the glibc + standard library and its version combination for "compatible" binaries if your package doesn't use other system libraries. Maybe not that useful for public binaries, but it's good to have such default settings out of the box for private workshops to not reinvent the wheel.

[kwinsch]

Taking the suggestions from https://docs.conan.io/en/latest/extending/custom_settings.html as a base and incorporating the ideas from @scddev , the following custom settings are working for me:

os:
...
  Linux:
    distro: [None, RHEL6, RHEL7, Ubuntu18.04]
...
compiler:
  ...
  gcc:
    ...
    libc: [None, glibc, musl]
...

Whe also have some custom made distros for embedded device which we all have a distro name for. Unlike the glibc mess, working with musl as a libc, does usually not require the definition of the distro.

[kwinsch]

For the toolchain side, we also had to expand 'os_build', so that the from source generated toolchain works on all build nodes. The toolchain is added as a dependency in the profile via a '[build_requires]'. The following expansion of the 'settings.yml" seems to works for our embedded, cross compilation setup.

os_build: 
    Windows:
    WindowsStore:
    Linux:
        distro: [None, "alpine", "ubuntu1204", "ubuntu1804"]
    Macos:
    FreeBSD:
    SunOS:
    AIX:
...
os:
    ...
    Linux:
        distro: [None, "buildroot_foo", "buildroot_bar"]
        platform: [None, "foo", "bar"]
...
compiler:
  ...
  gcc:
    ...
    libc: [None, glibc, musl]
...

[datalogics-robb]

I'm in favor of the solution using libc and its version.
Distribution is more specific, but is orthogonal to the things that make a binary incompatible. If a package is created linking to system shared libraries, those libraries might not have been installed even though the same version of the same distribution is used. This same issue could also be resolved by using static "linking", which isn't feasible to do with glibc.

[puetzk]

We (with several linux targets, some public distros and some yocto-based cross-toolchains) do the following in settings.yml

os:
    Linux:
        distribution:
            None:
            # based on /etc/os-release $ID and $VERSION_ID
            ubuntu:
                version: [ "16.04", "16.10", "18.04", "18.10" ]
            wrlinux:
                version: [ "7.0", "8.0", "8.0.0.28", "8.0.0.30" ]
            fsl-imx:
                version: [ "4.14.98-2.0.0_ga" ]

Allowing distribution: None covers the existing scheme if the profile doesn't set it, but otherwise we use the https://www.freedesktop.org/software/systemd/man/os-release.html standard to get labeling (not for the first time making me wish there was a .py version of profiles like there is conanfile.txt and conanfile.py, so it could just read os-release directly).

Yocto supports this sort of labeling via the bitbake recipe https://git.yoctoproject.org/cgit.cgi/poky/plain/meta/recipes-core/os-release/

[madebr]

What about this:

libc:
  glibc: ["2.18", "2.19", "2.20", ..., "2.31"]
  musl: ["0.9", "1.0", "1.1", "1.2"]
compiler:
  gcc:
    libc: &libc (None for undefined or default)
  llvm:
    libc: &libc
  Visual Studio:
    [no libc]

For auto generation of the profile, the libc version can be detected by compiling+executing a simple file using gnu_get_libc_version, or extracting the version defintions from objdump -p /lib64/libc.so.*.

The bigger problem is: how will the hashing/package id work?

Packages compiled with different glibc versions, will generate a different package id.
But how should conan behave if a consumer has a newer glibc library? Should it require a package of the same glibc version?
Should it look for all available packages having a glic version lower as its own?
ALso, I"m afraid this range matching is incompatible with the hashing principle of conan.

[Minimonium]

That's the same issue as with compiler.cppstd. We need a way of constraining compatible settings from upstream.

[mpusz]

Another point to highlight the importance of this feature is https://github.com/conan-io/conan-docker-tools/issues/200. Even though there is gcc-9.3 available in Ubuntu 20.04 (used already for gcc-10 image) and it fixes some bugs in gcc-9.2 that prevent my library to compile on the older compiler, we can't upgrade the docker image because of glibc issue.