Conan: Allow packages to report their own package_id() versioning mode to consumers

[michaelmaguire]

Sadly, some teams in my company do not follow https://semver.org/

I'm trying to understand the correct way to modify the conanfile recipes for those libraries to ensure we properly react to changes in minor version numbers that are actually binary breaking.

I've read https://docs.conan.io/en/latest/creating_packages/define_abi_compatibility.html#versioning-schema and https://docs.conan.io/en/latest/conan.pdf Release 1.12.3 on minor_mode() and I think it's what I want, but I'm trying to understand how to use it.

The examples in https://docs.conan.io/en/latest/creating_packages/define_abi_compatibility.html#versioning-schema seem to show how I could override package_id() for my consumer in order to ensure that changes of minor version number in a package I depend on require a rebuild.

Questions:

1. Retaining base behaviours

If I override package_id() as shown there, e.g.:

def package_id(self):
    self.info.requires["MyOtherLib"].minor_mode()

do I lose the rest of the default behaviour in my package_id calculation? Should the example show calling the parent class somehow to maintain the rest of the desired behaviours?

2. Setting in package itself rather than consumers

Is there a way to 'set' minor_mode() in the package itself, rather than the consumers?

e.g. something like

def package_id(self):
    self.info.minor_mode()

Thank you,
Michael

P.S.: I think the doc section on minor_mode() has a typo -- it shows patch_mode()

To help us debug your issue please explain:

• [x] I've read the CONTRIBUTING guide.
• [x] I've specified the Conan version, operating system version and any tool that can be relevant.
• [x] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

Conan version 1.10.1 on Windows

[jgsogo]

Hi, @michaelmaguire,

Answering your questions (the typo is already reported, thanks):

• def package_id(self) modifies the values of a member variable, it is not returning anything and the default implementation is empty (maybe we should clarify it in the docs) so, there is no need to call super() unless your recipe inherits from a custom ConanFile and you have some code already implemented there.

• Setting the non-semver compatibility in the package itself instead of doing it in the consumers: IMO it is the responsibility of the consumer package to declare how its package_id is being affected by the changes of its dependencies, the ABI of a dependency might not affect my ABI or API so I could keep the same package_id... but Conan has a default implementation and maybe it is not the right one (not the default behavior based in SemVer, but the implementation itself) I'll try to explain myself:

  Right now it is implemented something like this by default: _"all my dependencies uses SemVer and then the consumer can override some of them"_.

  The same default behavior could be achieved in a different way: _"I will ask all my dependencies about the ABI compatibility model they are using (and by default every library responds with a SemVer mode)"_, the user can declare a different compatibility model for his library, and of course the consumer can override it.

  We can add this flexibility without breaking current behavior, but this can have collateral effects I'm not aware of, I need here the expert opinion of more people, maybe I'm missing something else (ping @conan-io/barbarians).

I'll keep this issue in the triagge stage to discuss about this second point.

Thanks for the feedback!

[michaelmaguire]

• The same default behavior could be achieved in a different way: _"I will ask all my dependencies about the ABI compatibility model they are using (and by default every library responds with a SemVer mode)"_

I don't know if this would all be obsoleted by upcoming features, but this seems a more natural way to scale -- each library reports what versioning mechanism it respects.

I'll keep this issue in the triagge stage to discuss about this second point.

Thanks!

[memsharded]

The main problem with how packages affect binary compatibility is that packages cannot define how they affect their consumers. Let me put some example:

• A package "LibraryA" builds a static library, and lets say they adhere exactly to the semver model. In theory they should be able to declare "I am following semver".
• Another package "LibraryB" is building another static library and depends on "LibraryA". This is ok, if "LibraryA" don't break something over the headers that make it break semver, "LibraryB" should be able to not re-build, as it is not necessary (changes are isolated in LibraryA implementation details)
• But now we have another "LibraryC" package that is building a shared library which completely embeds and hides "LibraryA". Nothing of "LibraryA" will be exposed in the package API, but it will be used internally as implementation detail. Now, "LibraryC" might not be ok with the semver mode, because even implementation detail changes will be linked inside the shared library. If consumers of "LibraryC" want to get the latest bug fixes of "LibraryA", we should re-build "LibraryC" for every change of "LibraryA", and that means: self.info.requires["LibraryA"].full_package_mode().

So in summary, ABI depends both on the producer, and if they actually respect the contract of defined versioning, but also on the consumer, and how the consumers use the dependencies packages.

[michaelmaguire]

@memsharded I agree with the decision that a consumer needs the full flexibility to define how it views changes to the producer, but I think today by default, with no changes or package_id() in a consumer, the assumption is that any package it consumes respects semver.

As I've learned the hard way, this is actually a pretty strong assumption.

I think it might be very useful if a package could publish the default consumers should assume, which could be overriden (or not) by the consumer's package_id() method as today.

[niosHD]

For reference, exactly the same issue was just recently raised in https://github.com/conan-io/conan/issues/4071#issuecomment-464361928 and last year in #3318.

[jgsogo]

Thanks for linking to those issues, @niosHD, I wasn't aware of them. I think that this one can be resumed in #3318 (#4017 is about a global configuration, which can also be implemented unrelated to this one). So I'm closing this one, linking there some comments and I will add that one to the triagging stage, an alternative should be easy to implement, but we need to agree on what to do.

Let's continue there. Thanks again.

[dribeas]

So in summary, ABI depends both on the producer, and if they actually respect the contract of defined versioning, but also on the consumer, and how the consumers use the dependencies packages.

I think we are conflating here the ABI with the desire to pick a new version. In the description you gave "LibraryC" should not have self.info.requires["LibraryA"].full_package_mode(). If it is hiding "LibraryA" completely from the user, it does not matter which version of "LibraryA" is statically linked _for the purpose of ABI compatibility_.

The purpose of hiding "LibraryA" inside "LibraryC" is allowing the client to run with a different, even incompatible, version. The purpose is ensuring that "LibraryA" is not part of the ABI of "LibraryC". Making the "LibraryA" part of the package id for "LibraryC" breaks this, as it enforces that the application and "LibraryC" agree on a version of "LibraryA".

Managing breaking versions of "LibraryA" inside "LibraryC" can be done in other ways that do not force the coordinating the version of "LibraryA" with the application.

My team manages a "LibraryC" implementation, and we have done this for over 10 years, our solution to broken versions of our dependencies is creating a new version of "LibraryC". A bug in "LibraryA" is not different than a bug in our own code —if we had chosen to just reimplement the functionality internally.