Conan: How to update the recipe only

[uilianries]

Hi!

@ArekPiekarz asked to compare local cache vs remote cache:

how can I check if local packages have updated recipes on remotes?

The idea is run conan search and check if the local package are outdated when compared to remote cache recipe. However, AFAK it's not possible. The search command only check for outdated package in your local cache or in a remote:

conan search ref --outdated # local only
conan search ref -r foobar --outdated # remote only

My suggestion was, update the local cache and search for outdated packages. For example:

conan install ref --update

However, --update downloads a new recipe and a new binary package based on profile. Is it possible to update ONLY the recipe?

• [X] I've read the CONTRIBUTING guide.
• [X] I've specified the Conan version, operating system version and any tool that can be relevant.
• [X] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

[lasote]

You can run:

conan info ref -r remote -db -u

e.g:

$ conan info zlib/1.2.11@conan/stable -r conan-center -db -u

zlib/1.2.11@conan/stable: WARN: Forced build from source
zlib/1.2.11@conan/stable
    ID: d50a0d523d98c15bb147b18fa7d203887c38be8b
    BuildID: None
    Remote: conan-center=https://conan.bintray.com
    URL: http://github.com/conan-community/conan-zlib
    Homepage: https://zlib.net
    License: Zlib
    Author: Conan Community
    Recipe: Update available
    Binary: Build
    Binary remote: None
    Creation date: 2018-11-25 15:25:31
    Required by:

[uilianries]

TIL conan info is not just about dependencies graph :nerd_face:

[lasote]

Well, if you focus on a specific reference, the whole thing is a dependency graph of 1 node. :rofl:
The same should work for a dependency tree by the way.

[ArekPiekarz]

A downside of using conan info boost/1.69.0@conan/stable -r conan-center -db -u is it downloads recipes for bzip2 and zlib, even though they are optional and I don't need them. They are then visible when listing local recipes with conan search. I think it would be beneficial to have a way to answer "Are there any updates to the local recipes?" without downloading new ones.

[lasote]

even though they are optional and I don't need them

Are you sure? if they are not needed (according to the applied profile) they shoudln't be downloaded. Apply to the conan info the same profile you are using (if that is the issue).

About recipes (only recipes) being downloaded to the cache. We have been discussing many times and we started to think that a recipe in the cache is just a recipe in a cache. Some time ago it was a real issue when we hadn't the scm and exports_sources the recipes could contain a lot of data we wanted to skip downloading. That introduced very complicated parts in the Conan codebase and now there are less reasons to keep them.

Of course, this is a trade-off, we won't download them arbitrarily but in this case the conan info needs to compute the dependency graph and Conan will download them and for other pottential features if it simplify markedly the code, Conan will download also the recipes.

Said that, I'm not opossed to provide a new interface to check updates directly if this is really a must, but probably conan info is good enough.

[ArekPiekarz]

You're right, after adding -o boost:without_iostreams=True the additional recipes for bzip2 and zlib are not downloaded. In my project I define options in conanfile.py, but conan info doesn't accept it as an input.

I also tried using the -n parameter to display only the "Recipe" field, but it's not on the list of allowed variants. It's strange, but that is easily worked around with grep. Thanks for help!

[ArekPiekarz]

On a second thought, a more scalable solution should automatically look at current local packages, take their chosen option values and search for recipe updates. This way a user wouldn't need to copy the options from packages to conan info nor search for a minimal set of options that configure dependencies in the same way as current package. I did that last thing with Boost 1.69 recipe, where I found that disabling iostreams would be enough to prevent adding compression libs to requirements. Normally I use a lot more options, but they don't touch additional dependencies.

[lasote]

Could I ask what is the motivation for upgrading only the recipe? Just asking to learn you use case. It is a matter of not downloading a package not being used? Why don't you need the package?

[ArekPiekarz]

Looking at it from a high level, I'm searching for a Conan equivalent of the two step process of sudo apt update && apt list --upgradable and sudo apt upgrade. The first one fetches the metadata about packages and tells you which of them can be upgraded. Then at the second step you can decide if you want to actually download and install the updated binaries.

With Conan the decision is a little more complicated, because the new recipe can include backwards incompatible changes or new options, so it's useful to diff-compare the recipes contents. The building itself may take more time than downloading in apt and may fail for various reasons, so you have to schedule some time for it based on how many things needs updating.

I hope I answered your question :-)

Edit: To sum it up, I would like a process that lets me follow these steps:

1. Show a list of updatable recipes
2. Show changes within the recipes
3. Update all/chosen recipes
4. Rebuild all/chosen local packages

If you think it doesn't make sense to separate steps 3 and 4, I don't think I have strong objections.

[lasote]

Thanks for the feedback, I'll think about it.

About 2). How would you like to see/receive the diff of the recipe changes?

[uilianries]

Maybe conan search with a new argument could check for outdated recipe between remote vs local cache.

Diff is complex I think, we could run diff command, but even dpkg, pip or any other popular pm doesn't offer that feature.

[ArekPiekarz]

About 2). How would you like to see/receive the diff of the recipe changes?

So far I've been doing it in Meld, but I guess other people have different preferences. It could work as in git, where you can use git difftool and pass it a specific tool or set it beforehand in git config files.

Diff is complex I think, we could run diff command, but even dpkg, pip or any other popular pm doesn't offer that feature.

I'll assume you're right, because I haven't checked every other package system, but even they have ability to show changelogs of to-be-updated packages or recipes. I think Conan doesn't support it yet but it is already discussed in "Package revisions" issue (https://github.com/conan-io/conan/issues/798). Seeing changelogs could be an alternative to diffs, maybe even better then them if the descriptions of changes are properly maintained.