Conan: How to get the package reference after conan create

[CausticYarn]

As part of an automated build process, I am trying to build a conan package and then upload only that package to an artifact repository (specifically I don't want to use upload all, because I don't want to redundantly push dependencies that I just downloaded).

If I don't (redundantly) store the package name and version in the pipeline, is there any way to parse the package name and version from the conanfile.py? I couldn't find any option to conan info that only displayed the package name, although with the --only None option comes close as the reference with the @PROJECT annotation has what I need. However, there isn't a particularly easy way to parse the output of a conan command using the Artifactory plugin (that I am aware of), since it just echoes it and returns a buildInfo object for Artifactory.

Any suggestions are welcome.

[memsharded]

First, you shouldn't be that concerned about re-uploading, because the protocol is efficient, there are two layers: first conan will check that the binary artifacts haven't changed, and won't upload them. Also if using Artifactory, it uses checksum-deploy, so in most cases, it won't re-upload downloaded artifacts.

There is no evident way to achieve what you want, though it is true that an ongoing effort is being done towards improving the API, including json output for all commands, so the create command could contain a field with the reference of the package being created. For example, the info command has some --json field to create a json file, but only for the --build-order option. It could be extended, and parsing json in groovy is straightforward.

In the meantime, don't you know at least the name of the package? That would typically match the repo, the folder, or something in the job that can be identified. If you have that, you can probably do conan search PkgName* --raw. Otherwise, I would go at the moment for grepping with a regex for "name = (name)", and "version = (version)" in the bare conanfile.py file.

[CausticYarn]

It's not so much worrying about re-uploading from an efficiency perspective, it's that I am looking have an automated build that results in deploying a specific Conan package to the artifact repository and not be incidentally uploading other packages that may have been installed and are coming along for the ride.

I can think of a few ways to get the package name from the output of a Conan command, but the way the Artifactory plugin works, I am not sure how to retrieve the output of the command. Using the plugin method, the return value is a BuildInfo object which doesn't contain the output. I suppose I could run a shell command outside of the plugin, but it would be nice for that functionality to be available. Is this the place to request enhancements to the Conan aspect of the plugin?

[CausticYarn]

In the short term, I can just specify the name of the library in the pipeline, but as the goal is to make reusable pipelines that can build Conan packages, it would be helpful to have more auto-discovery around Conan package metadata.

[lasote]

Is this the place to request enhancements to the Conan aspect of the plugin?

Yes, we will talk with the team developing the plugin. The "bad news" is that the plugin development/release is not as agile as conan client. Probably has to wait to 2018 Q1

[memsharded]

Yes, we will also have it in mind while designing the json output of commands. Thanks for the feedback!

[Aalmann]

Probably you already have a solution for your problem. But for the rest of the world:
As a workaround you could add a Unix shell or Windos Batch step (I think you are using Jenkins in combination with Pipeline) to get the name.
On Windows:

SETLOCAL EnableDelayedExpansion

set USER_CHANNEL=MyUser/testing
set STR_REPLACE=@PROJECT
set REMOTE=myremote
for /f %%i in ('conan info . --only None ^| findstr %STR_REPLACE%') do (
    set _CONAN_NAME=%%i
    set CONAN_NAME=!_CONAN_NAME:%STR_REPLACE%=!
)

conan upload -r !REMOTE! --all --confirm !CONAN_NAME!@!USER_CHANNEL!

On Linux:

USER_CHANNEL=MyUser/testing
STR_REPLACE=@PROJECT
REMOTE=myremote
CONAN_NAME=`conan info . --only None | grep $STR_REPLACE`
CONAN_NAME=${CONAN_NAME/$STR_REPLACE/""}

conan upload -r $REMOTE --all --confirm $CONAN_NAME@$USER_CHANNEL

Not sure if one can also only export Variables from one step/task to another.

[sourcedelica]

Here's another way to get the name (and other attributes of the ConanFile)

from conans.client import loader_parse

conanfile_path = '.'
conanfile_class = loader_parse.load_conanfile_class(conanfile_path)
name = conanfile_class.name

[madpipeline]

From what I can see the buildInfo object returned by any client.run(command: "") contains only Jenkins details, and no package details. E.g.: Job name, job ID, job env. etc.

Since the plugin does not do the uploading, I don't really see the benefit of it knowing these things.

In Conan 1.2 the --json flag was added to conan create. This creates a JSON file that contains the package reference.

However: https://github.com/conan-io/conan/issues/2695

[ytimenkov]

If you use Python API this should work: (until next version, where create and install should return what is being serialized to Json)

        # Input:
        reference = "Mypkg/1.2.3@user/channel"
        profile = "default"
        options = [] # options to build

        # From tutorial
        conan_api, _client_cache, _user_io = Conan.factory()

        # vvvvv These 2 lines do the job
        deps_graph, _graph_updates_info, _project_reference = conan_api.info_get_graph(
            reference, profile_name=profile, options=options)
        conan_ref, conanfile = deps_graph.direct_requires()[0]

        if str(conan_ref) != reference: # Sanity check
            raise RuntimeError(
                "Package ID mismatch: Building for id {}, while dependency analysis returned {}".
                format(reference, conan_ref))

        package_id = conanfile.info.package_id()
        conan_api.upload(pattern=reference, package=package_id)

[lasote]

The version 1.2.3 have already a --json option in the conan install and conan create commands to output a json with the installed packages information:

http://docs.conan.io/en/latest/reference/commands.html#output

[lasote]

Let me know if this issue can be closed @causticyarn

[madpipeline]

Oneliner to get the package version:
Windows with cygwin:

conan info . --only None | grep "@PROJECT" | cut -d"/" -f 2 | cut -d"@" -f 1 > temp.txt && set /p PACKAGE_VERSION=<temp.txt

Linux:

export PACKAGE_VERSION=`conan info . --only None | grep "@PROJECT" | cut -d"/" -f 2 | cut -d"@" -f 1`

P.S.: Please avoid using cygwin at all costs.

[memsharded]

Update:

• The conan create command has now the json output that describes what have been done. That can be used to select what to upload in CI
• The new conan inspect command (to be released in 1.8), will allow to get the name, version of a conanfile.py. I think with both features, once this latter one is released, this issue could be closed.

[monsdar]

For completeness: Here's a oneliner using Python to read the json generated by conan create and fill a env variable with the actual reference:
export CONANREF = $(python -c "import json; j=json.load(open('packageInfo.json')); print(j['name'] + '/' + j['version'] + '@user/release')")