Conan: [bug] conan search output is not stable

[joelpurra]

When executing conan search --remote conan-center 'zlib/1.2.11@' twice (in parallel or maximum minutes apart) the output seems to differ between runs. In particular, a diff between two "identical" searches shows that the same packages are returned, but outdated flips boolean value for the same id.

• Is there any chance there's a search cache inconsistency? Locally or remotely?
• Which of the outputs should be "trusted"?

Notes

• It seems to be "relatively" non-transient; variations seem to happen when running the same searches 20+ minutes later. Have not awaited a recipe to be published again though.
• Using a --query seems to have the same issue.
• Setting CONAN_USER_HOME to an empty directory didn't help.
• Tested briefly with some other packages:
  
  
  
  • boost/1.72.0@: outdated differs.
  
  
  • libgettext/0.20.1: no difference.
  
  
  • libpng/1.6.37@: outdated differs.
  
  

Environment Details (include every applicable attribute)

• Operating System+version: Ubuntu 19.10
• Compiler+version: -
• Conan version: 1.24.0

Python version: 3.7.5

head --lines 1 "$(which conan)"

#!/usr/bin/python3

/usr/bin/python3 --version

Python 3.7.5

Steps to reproduce (Include if Applicable)

Run the search twice and compare the output.

Logs (Executed commands with output) (Include/Attach if Applicable)

Text output comparison

conan search --remote conan-center 'zlib/1.2.11@' > a.txt
sleep 5
conan search --remote conan-center 'zlib/1.2.11@' > b.txt
diff a.txt b.txt

diff a.txt b.txt

237c237
<         Outdated from recipe: False
---
>         Outdated from recipe: True
406c406
<         Outdated from recipe: False
---
>         Outdated from recipe: True
445c445
<         Outdated from recipe: True
---
>         Outdated from recipe: False
692c692
<         Outdated from recipe: False
---
>         Outdated from recipe: True
874c874
<         Outdated from recipe: False
---
>         Outdated from recipe: True
978c978
<         Outdated from recipe: True
---
>         Outdated from recipe: False
1095c1095
<         Outdated from recipe: True
---
>         Outdated from recipe: False
1251c1251
<         Outdated from recipe: False
---
>         Outdated from recipe: True
1264c1264
<         Outdated from recipe: False
---
>         Outdated from recipe: True

JSON output

JSON output comparison

Outputting JSON and using jq to sort the data for slightly more stable diffing. (Although it's still "just" text diffing.)

• https://stedolan.github.io/jq/

conan search --json >(jq --sort-keys '.' > a.json) --remote conan-center 'zlib/1.2.11@'
sleep 5
conan search --json >(jq --sort-keys '.' > b.json) --remote conan-center 'zlib/1.2.11@'
diff a.json b.json

diff a.json b.json

117c117
<               "outdated": false,
---
>               "outdated": true,
575c575
<               "outdated": true,
---
>               "outdated": false,
848c848
<               "outdated": false,
---
>               "outdated": true,
899c899
<               "outdated": false,
---
>               "outdated": true,
1103c1103
<               "outdated": true,
---
>               "outdated": false,
1120c1120
<               "outdated": true,
---
>               "outdated": false,
1221c1221
<               "outdated": true,
---
>               "outdated": false,
1358c1358
<               "outdated": false,
---
>               "outdated": true,
1426c1426
<               "outdated": false,
---
>               "outdated": true,
1613c1613
<               "outdated": true,
---
>               "outdated": false,
1664c1664
<               "outdated": true,
---
>               "outdated": false,
1698c1698
<               "outdated": false,
---
>               "outdated": true,
1884c1884
<               "outdated": true,
---
>               "outdated": false,
1970c1970
<               "outdated": false,
---
>               "outdated": true,
2106c2106
<               "outdated": true,
---
>               "outdated": false,
2140c2140
<               "outdated": true,
---
>               "outdated": false,
2242c2242
<               "outdated": false,
---
>               "outdated": true,
2276c2276
<               "outdated": true,
---
>               "outdated": false,
2344c2344
<               "outdated": true,
---
>               "outdated": false,

Selective JSON output comparison

Transforms the JSON output from the previous step to a simpler { id: outdated } format for more stable (text) diffing.

jq '.results[].items[].packages | sort_by(.id) | map({key: .id, value: .outdated}) | from_entries' a.json > a-ids.json
jq '.results[].items[].packages | sort_by(.id) | map({key: .id, value: .outdated}) | from_entries' b.json > b-ids.json
diff a-ids.json b-ids.json

diff a-ids.json b-ids.json

8c8
<   "0f136f157cacbc8ea97ec68d98151824896d64c2": false,
---
>   "0f136f157cacbc8ea97ec68d98151824896d64c2": true,
35c35
<   "3fb49604f9c2f729b85ba3115852006824e72cab": true,
---
>   "3fb49604f9c2f729b85ba3115852006824e72cab": false,
51c51
<   "689541ef75e69f6ec6dadbae4187260d6a3f4cfb": false,
---
>   "689541ef75e69f6ec6dadbae4187260d6a3f4cfb": true,
54c54
<   "6af9cc7cb931c5ad942174fd7838eb655717c709": false,
---
>   "6af9cc7cb931c5ad942174fd7838eb655717c709": true,
66,67c66,67
<   "7c292f54b7e6c224f121c640404bf7fbbeb41a8f": true,
<   "7d2ce8091910a92024e82e1030e9e75ba0100ecd": true,
---
>   "7c292f54b7e6c224f121c640404bf7fbbeb41a8f": false,
>   "7d2ce8091910a92024e82e1030e9e75ba0100ecd": false,
73c73
<   "8cf01e2f50fcd6b63525e70584df0326550364e1": true,
---
>   "8cf01e2f50fcd6b63525e70584df0326550364e1": false,
81c81
<   "961755854760682044d37dabfc2d998658edf961": false,
---
>   "961755854760682044d37dabfc2d998658edf961": true,
85c85
<   "a064fac9bef9f843f8878f385a5d4e6281f9a310": false,
---
>   "a064fac9bef9f843f8878f385a5d4e6281f9a310": true,
96c96
<   "af96383df8ed9748d2eed48fb04099c5914a718f": true,
---
>   "af96383df8ed9748d2eed48fb04099c5914a718f": false,
99c99
<   "b3c9f8c9c967c7b133104fb9264ae5ac4c657191": true,
---
>   "b3c9f8c9c967c7b133104fb9264ae5ac4c657191": false,
101c101
<   "b614d3b31d5e8feba6472fe56ecfe9a6fec1dbe7": false,
---
>   "b614d3b31d5e8feba6472fe56ecfe9a6fec1dbe7": true,
112c112
<   "c83d8b197f1a331ca3b55943846d427ad4f7f8e1": true,
---
>   "c83d8b197f1a331ca3b55943846d427ad4f7f8e1": false,
117c117
<   "c9ddecd1418a823818918c23e0d66337382afd99": false,
---
>   "c9ddecd1418a823818918c23e0d66337382afd99": true,
125c125
<   "d5cf5f134638cc8b43df8417b471909f6ad7bc28": true,
---
>   "d5cf5f134638cc8b43df8417b471909f6ad7bc28": false,
127c127
<   "db2ca884c9793e0b0fb54ec3f846326d1addacc8": true,
---
>   "db2ca884c9793e0b0fb54ec3f846326d1addacc8": false,
133c133
<   "ea598ec51f506a4c0ce579fe1e9cbc6299458ebb": false,
---
>   "ea598ec51f506a4c0ce579fe1e9cbc6299458ebb": true,
135c135
<   "f16aa6248c005c19f226df75e351b1b7fbea3949": true,
---
>   "f16aa6248c005c19f226df75e351b1b7fbea3949": false,
139c139
<   "f7e573cb501ccfc49e9e4d84de886bc1ef2e6ebb": true,
---
>   "f7e573cb501ccfc49e9e4d84de886bc1ef2e6ebb": false,

Using a JSON diffing tool looks slightly cleaner. Using Node.js' npx to run json-diff.

• https://www.npmjs.com/package/npx
• https://www.npmjs.com/package/json-diff

npx json-diff a-ids.json b-ids.json

npx json-diff a-ids.json b-ids.json

 {
-  0f136f157cacbc8ea97ec68d98151824896d64c2: false
+  0f136f157cacbc8ea97ec68d98151824896d64c2: true
-  3fb49604f9c2f729b85ba3115852006824e72cab: true
+  3fb49604f9c2f729b85ba3115852006824e72cab: false
-  689541ef75e69f6ec6dadbae4187260d6a3f4cfb: false
+  689541ef75e69f6ec6dadbae4187260d6a3f4cfb: true
-  6af9cc7cb931c5ad942174fd7838eb655717c709: false
+  6af9cc7cb931c5ad942174fd7838eb655717c709: true
-  7c292f54b7e6c224f121c640404bf7fbbeb41a8f: true
+  7c292f54b7e6c224f121c640404bf7fbbeb41a8f: false
-  7d2ce8091910a92024e82e1030e9e75ba0100ecd: true
+  7d2ce8091910a92024e82e1030e9e75ba0100ecd: false
-  8cf01e2f50fcd6b63525e70584df0326550364e1: true
+  8cf01e2f50fcd6b63525e70584df0326550364e1: false
-  961755854760682044d37dabfc2d998658edf961: false
+  961755854760682044d37dabfc2d998658edf961: true
-  a064fac9bef9f843f8878f385a5d4e6281f9a310: false
+  a064fac9bef9f843f8878f385a5d4e6281f9a310: true
-  af96383df8ed9748d2eed48fb04099c5914a718f: true
+  af96383df8ed9748d2eed48fb04099c5914a718f: false
-  b3c9f8c9c967c7b133104fb9264ae5ac4c657191: true
+  b3c9f8c9c967c7b133104fb9264ae5ac4c657191: false
-  b614d3b31d5e8feba6472fe56ecfe9a6fec1dbe7: false
+  b614d3b31d5e8feba6472fe56ecfe9a6fec1dbe7: true
-  c83d8b197f1a331ca3b55943846d427ad4f7f8e1: true
+  c83d8b197f1a331ca3b55943846d427ad4f7f8e1: false
-  c9ddecd1418a823818918c23e0d66337382afd99: false
+  c9ddecd1418a823818918c23e0d66337382afd99: true
-  d5cf5f134638cc8b43df8417b471909f6ad7bc28: true
+  d5cf5f134638cc8b43df8417b471909f6ad7bc28: false
-  db2ca884c9793e0b0fb54ec3f846326d1addacc8: true
+  db2ca884c9793e0b0fb54ec3f846326d1addacc8: false
-  ea598ec51f506a4c0ce579fe1e9cbc6299458ebb: false
+  ea598ec51f506a4c0ce579fe1e9cbc6299458ebb: true
-  f16aa6248c005c19f226df75e351b1b7fbea3949: true
+  f16aa6248c005c19f226df75e351b1b7fbea3949: false
-  f7e573cb501ccfc49e9e4d84de886bc1ef2e6ebb: true
+  f7e573cb501ccfc49e9e4d84de886bc1ef2e6ebb: false
 }

[czoido]

Hi @joelpurra,
Thanks a lot for reporting, this is in fact a bug reported here: https://www.jfrog.com/jira/browse/RTFACT-20919
You can read more about this here too: https://github.com/conan-io/conan-center-index/issues/200
After uploading different revisions of a Conan package to Artifactory and the searching that package through Conan with revisions disabled in the client, Artifactory is returning an incorrect revision (it should return the last one).
If you want to avoid this problem you could enable revisions in the Conan client.
Hope this helps.

[joelpurra]

@czoido: thanks for checking!

Ran some quick tests. Repeated searches have no differences after conan config set general.revisions_enabled=1.

• https://docs.conan.io/en/latest/versioning/revisions.html