Conan: [question] Conan Package for associated git commit

[juba895]

Hello,

we are using the conan attribute scm and the automatic detection of the revision with the auto value.

class MyConanFile(ConanFile):
    ...
    scm = {
        "type": "git",
        "url": "auto",
        "revision": "auto"
    }

Our Jenkins is automatically uploading for a new commit the associated conan package to our artifactory.
In some cases i want to promote one of this conan packages to our "releases" artifactory.
Most times it is not so easy to find the associated package, because of the amount of packages. The order using the rrev in artifactory does not help here much.

Is there the possibility to find with conan search (or conan info) cmd the associated conan package by handing over the git revision?

• [x] I've read the CONTRIBUTING guide.

[solvingj]

SCM has a revision_mode attribute which can be changed from the default (hash to scm). This will make the Conan RREV value use the GIT revision instead of hashing the sources. Then, you can just conan install rref#git_revision. Is that correct @memsharded @jgsogo ?

[jgsogo]

@juba895 , have you tried it? Is this answering your question?

[juba895]

Thank you for your answer! I not yet tried out to use revision_mode=scm, but i will!

Nevertheless i tried it out with revision_mode=hash, but unfortunately i was not succesful.
What i need to to is to browse to the conanfile.py of the package which rrev im looking for and call conan install . #707ba8a13e8e291828e7c612a57c0b54? Because that is not downloading the package at least at my environment..

Additionaly it means there is no conan search command available to search for a specific conan package with a specific git revision? So to not have the need to have a particular conanfile.py in place + downloading the package?

[solvingj]

You can only install a revision using a full package reference:
conan install pkg/ver@user/channel#707ba8a13e8e291828e7c612a57c0b54

You cannot do the equivalent with the path:
conan install . #707ba8a13e8e291828e7c612a57c0b54

You are right, there is no "search" for a git revision. You can search for the package and pass the --revisions flag to get a list of revisions. At that point, you would have to grep/find for your git revision in the list. Again, this only works if you change the revision_mode to scm on all your recipes.

[juba895]

You can only install a revision using a full package reference:
conan install pkg/ver@user/channel#707ba8a13e8e291828e7c612a57c0b54

If the ver is not known (or also the user or channel) this command also do not help much unfortunately.

You are right, there is no "search" for a git revision. You can search for the package and pass the --revisions flag to get a list of revisions. At that point, you would have to grep/find for your git revision in the list. Again, this only works if you change the revision_mode to scm on all your recipes.

What i did as a workaround is:

1. conan search pkgName -r remote (to find out all available versions, user, channels ...)
2. for each returned item conan search item -r remote -rev (to find out all available revisions for one version)
3. for each returend rev conan inspect item#rev -r remote -a scm. The response contains the scm information including the git revision. With that it's possible to check if that particular version contains the wanted git revision.

It's working, but unfortunately quite slow (in case there are a lot of packages). That's why i was thinking it would be first of all a nice feature if this would be supported by conan search out of the box and also probably much faster.

Regards,
Julian

[solvingj]

If the ver is not known (or also the user or channel) this command also do not help much unfortunately.

Ahh, yes, this is a very good point. It's interesting.

In thinking about the high level need, I think the user story was imagined as:

It would be helpful to be able to correlate a Conan package revision back to the GIT/SVN revision number that was used to build it. Also, conversely, it would be helpful to correlate a GIT/SVN revision to the Conan Package revision which was built from it. So, by providing the option to set the revision_mode to scm, it will be easy to correlate Conan package revisions to SCM revisions.

However, you've now highlighted that the specifics of the implementation make some types of correlation awkward (a bit short of "easy"). Mostly, it's a matter of how much information you need to perform the conan search command.

• Name
• Version
• User
• Channel

You can only perform a search for an SCM/Conan revision with a complete reference. As you demonstrated, it's possible to iterate over the repository to achieve the goal, however the UX here is extremely undesirable.

Moving Forward:

I think the specific constraint of your use case is very general:

• Given only a package name and a GIT commit, find any packages built from that commit

Furthermore, I would add that this would be a valuable feature, regardless of SCM (relevant when using source() method, exports_sources, or even conan export-pkg).

I think the biggest challenge with making this search fast, easy, flexible, is that the current implementation relies on the revision being part of the package reference. Thus, I think this relates to another outstanding feature-request and initiative here:

https://github.com/conan-io/conan/issues/7207

In theory, if a new dimension of metadata was added to recipes and packages, then in theory the conan search command (or perhaps some new conan metadata search would make it possible and performant to search an entire repository for a value, in the metadata across all recipes and packages.

[jgsogo]

In ConanCenter we are storing the Git commit as a property in Artifactory (at recipe-revision level). Then one can use Artifactory API to search for it and _reconstruct_ the recipe-revision. I'm sure this is something easy to implement/abstract with JFrog CLI plugin.

[solvingj]

Great!
So, we can say that SCM revision is an example of a piece of metadata related to a recipe/recipe-revision-level which we wanted to attach directly to that recipe/revision for Conan Center. This is consistent with other user stories for private repositories, so it does seem to add support to the case of adding a first-class user-assignable metadata mechanism to Conan.

[jgsogo]

Well, IMHO, there is a lot of metadata that makes sense to add at a recipe revision level (and also at package revision level). We are going a bit offtopic wrt this issue, but in ConanCenter we are already storing a lot of information in properties... in fact, all the information we sent to the frontend's database.

[solvingj]

Well, IMHO, there is a lot of metadata that makes sense to add at a recipe revision level

I agree, that's the point of the reference to the ticket, which talks about addressing it in a general way

We are going a bit offtopic wrt this issue

So far I think everything mentioned has related very well to the original question. I think future discussion about a potential metadata-related solution for the general case should proceed under the feature request linked: https://github.com/conan-io/conan/issues/7207

In terms of this Github issue for @juba895 , I think he found the only real way to achieve his goal under the current feature set, so it's possible that this ticket is ready to close. (which was very clever by the way).

@juba895 let us know what you think about closing. Also, if you have further thoughts about #7207, please post them there.

[juba895]

Thanks a lot, also from my point of view we can close this issue. Would be happy if something like conan metadata search could be implemented in future to be able to search for metadata like scm attributes and only handing over the package name.

Regards!

[solvingj]

I will try to remember to ping you if/when we get something implemented.