Conan: pre_export hook setting the reference version

[luckielordie]

Conan version : 1.17.0

I'm trying to write a hook that can be enabled on our Jenkins server to populate the metadata portion of the semver version to contain the build number and other properties.

When I create a pre_export hook that modifies the reference attribute it does not get set for the exported package, but the command still executes successfully. The hook code is as follows.

from conans import tools
from conans.errors import ConanInvalidConfiguration

def _is_on_jenkins_server():
    value = tools.get_env("JENKINS_URL")
    return value != None

def pre_export(output, conanfile, conanfile_path, reference, **kwargs):
    if False == _is_on_jenkins_server():
        raise ConanInvalidConfiguration("Hook not running on a Jenkins instance. Disable this hook if it is enabled on your developer machine. https://docs.conan.io/en/latest/extending/hooks.html#storage-activation-and-sharing")

    output.info("Adding build metadata to version")
    build_number = tools.get_env("BUILD_NUMBER")
    version = tools.Version(conanfile.version)
    version_string = f"{version.major}.{version.minor}.{version.patch}"
    version_string = f"{version_string}+build.{build_number}"
    reference = reference._replace(version=version_string)
    output.info(f"Setting version to {version_string}")

Here is the output of this hook being run with the command _conan create . luck/test_

PS F:\Projects\sharedpropertyfiles> conan create . luck/test
[HOOK - version_metadata.py] pre_export(): Adding build metadata to version
[HOOK - version_metadata.py] pre_export(): Setting version to 0.0.5+build.420
Exporting package recipe
props/0.0.5@luck/test exports: Copied 3 files: .gitignore, Jenkinsfile, Pipfile
props/0.0.5@luck/test exports: Copied 1 '.py' file: conanfile.py
props/0.0.5@luck/test exports: Copied 1 '.lock' file: Pipfile.lock
props/0.0.5@luck/test exports: Copied 1 '.h' file: Launcher.h
props/0.0.5@luck/test exports: Copied 4 '.props' files: CCLX.props, CCLXExample.props, CCLXSupport.props, CCLXUnitTest.props
props/0.0.5@luck/test: A new conanfile.py version was exported
props/0.0.5@luck/test: Folder: D:\conan_home\.conan\data\props\0.0.5\luck\test\export
props/0.0.5@luck/test: Exported revision: 27e4ced95324f39a00fa6f5d84830282
Configuration:
[settings]
arch=x86
arch_build=x86_64
build_type=Debug
compiler=Visual Studio
compiler.runtime=MDd
compiler.toolset=v100
compiler.version=10
os=Windows
os_build=Windows
[options]
[build_requires]
[env]

props/0.0.5@luck/test: Forced build from source
Installing package: props/0.0.5@luck/test
Requirements
    props/0.0.5@luck/test from local cache - Cache
Packages
    props/0.0.5@luck/test:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Build

Cross-build from 'Windows:x86_64' to 'Windows:x86'
props/0.0.5@luck/test: Configuring sources in D:\conan_home\.conan\data\props\0.0.5\luck\test\source
props/0.0.5@luck/test: Copying sources to build folder
props/0.0.5@luck/test: Building your package in D:\conan_home\.conan\data\props\0.0.5\luck\test\build\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
props/0.0.5@luck/test: Generator txt created conanbuildinfo.txt
props/0.0.5@luck/test: Calling build()
props/0.0.5@luck/test: WARN: This conanfile has no build step
props/0.0.5@luck/test: Package '5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9' built
props/0.0.5@luck/test: Build folder D:\conan_home\.conan\data\props\0.0.5\luck\test\build\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
props/0.0.5@luck/test: Generated conaninfo.txt
props/0.0.5@luck/test: Generated conanbuildinfo.txt
props/0.0.5@luck/test: Generating the package
props/0.0.5@luck/test: Package folder D:\conan_home\.conan\data\props\0.0.5\luck\test\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
props/0.0.5@luck/test: Calling package()
props/0.0.5@luck/test package(): Packaged 4 '.props' files: CCLX.props, CCLXExample.props, CCLXSupport.props, CCLXUnitTest.props
props/0.0.5@luck/test package(): Packaged 1 '.h' file: Launcher.h
props/0.0.5@luck/test: Package '5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9' created
props/0.0.5@luck/test: Created package revision 59a87e072c5b10b7c0d2ceb210259693

This potentially opens up the discussion about whether hooks should be allowed to alter the reference or other properties itself.

[lasote]

This potentially opens up the discussion about whether hooks should be allowed to alter the reference or other properties itself.

That is indeed a very good question.

I verified in the code that the changes in the reference are mostly ignored. I think it could be done but we need to discuss if we want to. @memsharded

[luckielordie]

If settings like version in the recipe are being changed. How can Conan account for that? Does it have to write into the un-exported conanfile? Does this open up people to have Conan Packages flying around that don't match the codebase they were exported from?

[Minimonium]

In a case like pre_export, Conan could intercept the user-defined reference and adjust it before, well, exporting just as well. After that, it would behave just as right now. For example, if the recipe would have a version set and pre_export would change it to a different one - it would produce an error.
That's, though, mostly a good example of pre_export, since it's a middleware between user input and Conan environment. I'm not sure about other hooks.

[lasote]

We will discuss it for the next release. Thanks

[lasote]

So, we think we shouldn't allow doing that. The reason is very close to the @Minimonium comment. If you have a recipe with version="1.0" and you try to export it indicating another one, like conan export . lib/1.0+build.420 it will fail.

We have to be consistent with the behavior. Now the question is, should Conan allow to "complete" the version with build information?

My point of view about that is that I don't like at all to mess with the version, that metadata doesn't feel right there. I think it should be stored somewhere else, like the concept of "build info" from Artifactory. If you have recipes that depends on that modified recipe, they would need to change the required reference too. So then the question should be.... should Conan discard the build information from the version? That would break all protocols with the servers and so on... So I would see this feature request as opening the gate to a path we don't want to walk.

[Minimonium]

@luckielordie Btw, just a note on how I do it in our internal CI. We have a VERSION file that's parsed by the recipe. On the pre-build step, CI checks against the branch/tag and changes the file to produce a different version. Maybe it could work for you too.