Conan: [bug] conan install --build package does not always build package

Created on 6 May 2020  路  6Comments  路  Source: conan-io/conan

Environment Details (include every applicable attribute)

  • Operating System+version: All
  • Compiler+version: All
  • Conan version: 1.24

Steps to reproduce (Include if Applicable)

In older versions of Conan when you ran conan install .. and it was missing a package, say xxx it would give you an error message saying Missing prebuilt package for xxx/1.0. Then you would run conan install .. --build xxx and it would build xxx, then you could move on with another conan install ... Now if you run conan install .. --build xxx it won't necessarily build xxx. It will probably find a different missing package. Then when you try to --build that one it will find the first one again, or another one. The only thing that works is conan install .. --build=missing. Is this a bug?

Logs (Executed commands with output) (Include/Attach if Applicable)

$ conan install ..
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Debug
compiler=gcc
compiler.libcxx=libstdc++
compiler.version=5
os=Linux
os_build=Linux
[options]
[build_requires]
[env]

WARN: boost/1.72.0: requirement zlib/1.2.11 overridden by your conanfile to zlib/1.2.8
WARN: cppzmq/4.3.0@bincrafters/stable: requirement zmq/4.2.5@bincrafters/stable overridden by your conanfile to zmq/4.3.2@bincrafters/stable
WARN: libcurl/7.69.1: requirement openssl/1.1.1f overridden by your conanfile to openssl/1.1.1g
WARN: libcurl/7.69.1: requirement zlib/1.2.11 overridden by your conanfile to zlib/1.2.8
tbb/2020.1: WARN: Intel-TBB strongly discourages usage of static linkage
boost/1.72.0: WARN: Package is corrupted, removing folder: /userhome/epederson/.conan/data/boost/1.72.0/_/_/package/0d14d262cdde8dde978c9dee82e8fb4c0ddf70c1
conanfile.py: Installing package
Requirements
    boost/1.72.0 from 'conan-idb-third-party' - Cache
    bzip2/1.0.8 from local cache - Cache
    caf/0.17.3.1@bincrafters/stable from 'conan-idb-third-party' - Cache
    catch2/2.2.2@bincrafters/stable from 'conan-idb-third-party' - Cache
    cppzmq/4.3.0@bincrafters/stable from 'conan-idb-third-party' - Cache
    jsoncpp/1.9.2 from 'conan-idb-third-party' - Cache
    libcurl/7.69.1 from 'conan-idb-third-party' - Cache
    libiconv/1.15 from 'conan-idb-third-party' - Cache
    libsodium/1.0.18 from 'conan-idb-third-party' - Cache
    llm/3.0.74.2887@llm/prod from 'ifi-conan-local' - Cache
    odbc/2.3.7 from 'conan-idb-third-party' - Cache
    openssl/1.1.1g from 'conan-idb-third-party' - Cache
    pistache/2f04981.1@idbdevbuild/stable from 'conan-idb-third-party' - Cache
    protobuf/3.9.1@bincrafters/stable from 'conan-idb-third-party' - Cache
    protoc_installer/3.9.1@bincrafters/stable from 'conan-idb-third-party' - Cache
    spdlog/0.16.3.1@idbdevbuild/stable from 'conan-idb-third-party' - Cache
    tbb/2020.1 from 'conan-idb-third-party' - Cache
    zlib/1.2.8 from 'conan-idb-third-party' - Cache
    zmq/4.3.2@bincrafters/stable from 'conan-idb-third-party' - Cache
Packages
    boost/1.72.0:0d14d262cdde8dde978c9dee82e8fb4c0ddf70c1 - Missing
    bzip2/1.0.8:a5fb342591cd091faa8b36194fe6a476ae195f44 - Cache
    caf/0.17.3.1@bincrafters/stable:6343484dfa90a2e1ec36a9397a8ce2a2ca53da04 - Missing
    catch2/2.2.2@bincrafters/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    cppzmq/4.3.0@bincrafters/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    jsoncpp/1.9.2:435f6d82215d51c7350154060d2d6634250003de - Cache
    libcurl/7.69.1:5bf21f754405cd5cdae4509e0ed60c17fc8ca929 - Missing
    libiconv/1.15:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3 - Missing
    libsodium/1.0.18:6236a09dcd18d9f4591a2476c0f9c194e2d0a9d0 - Cache
    llm/3.0.74.2887@llm/prod:3981be4b18fe12bd3e6b15efb666da2494a3cf6d - Cache
    odbc/2.3.7:7d2ac76effb4a725f95b5eac45cf5694f8d5c1f2 - Missing
    openssl/1.1.1g:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3 - Cache
    pistache/2f04981.1@idbdevbuild/stable:ef58664316b93dde9b463dc888227d2113276c95 - Missing
    protobuf/3.9.1@bincrafters/stable:435f6d82215d51c7350154060d2d6634250003de - Cache
    protoc_installer/3.9.1@bincrafters/stable:c0c1ef10e3d0ded44179e28b669d6aed0277ca6a - Cache
    spdlog/0.16.3.1@idbdevbuild/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    tbb/2020.1:435f6d82215d51c7350154060d2d6634250003de - Missing
    zlib/1.2.8:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3 - Cache
    zmq/4.3.2@bincrafters/stable:acc65ea587795417f1e4875725a2d13dcc42a806 - Cache

Installing (downloading, building) binaries...
ERROR: Missing binary: boost/1.72.0:0d14d262cdde8dde978c9dee82e8fb4c0ddf70c1
ERROR: Missing binary: caf/0.17.3.1@bincrafters/stable:6343484dfa90a2e1ec36a9397a8ce2a2ca53da04
ERROR: Missing binary: libcurl/7.69.1:5bf21f754405cd5cdae4509e0ed60c17fc8ca929
ERROR: Missing binary: libiconv/1.15:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3
ERROR: Missing binary: odbc/2.3.7:7d2ac76effb4a725f95b5eac45cf5694f8d5c1f2
ERROR: Missing binary: pistache/2f04981.1@idbdevbuild/stable:ef58664316b93dde9b463dc888227d2113276c95
ERROR: Missing binary: tbb/2020.1:435f6d82215d51c7350154060d2d6634250003de

libiconv/1.15: WARN: Can't find a 'libiconv/1.15' package for the specified settings, options and dependencies:
- Settings: arch=x86_64, build_type=Debug, compiler=gcc, compiler.version=5, os=Linux
- Options: fPIC=True, shared=False
- Dependencies:
- Requirements:
- Package ID: b1b85bfdc2206d81b3ed6a49e78498182bb98ef3

ERROR: Missing prebuilt package for 'libiconv/1.15'
Try to build it from sources with "--build libiconv"
Or read "http://docs.conan.io/en/latest/faq/troubleshooting.html#error-missing-prebuilt-package"

$ conan install .. --build libiconv
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Debug
compiler=gcc
compiler.libcxx=libstdc++
compiler.version=5
os=Linux
os_build=Linux
[options]
[build_requires]
[env]

WARN: boost/1.72.0: requirement zlib/1.2.11 overridden by your conanfile to zlib/1.2.8
WARN: cppzmq/4.3.0@bincrafters/stable: requirement zmq/4.2.5@bincrafters/stable overridden by your conanfile to zmq/4.3.2@bincrafters/stable
WARN: libcurl/7.69.1: requirement openssl/1.1.1f overridden by your conanfile to openssl/1.1.1g
WARN: libcurl/7.69.1: requirement zlib/1.2.11 overridden by your conanfile to zlib/1.2.8
tbb/2020.1: WARN: Intel-TBB strongly discourages usage of static linkage
libiconv/1.15: Forced build from source
boost/1.72.0: WARN: Package is corrupted, removing folder: /userhome/epederson/.conan/data/boost/1.72.0/_/_/package/0d14d262cdde8dde978c9dee82e8fb4c0ddf70c1
conanfile.py: Installing package
Requirements
    boost/1.72.0 from 'conan-idb-third-party' - Cache
    bzip2/1.0.8 from local cache - Cache
    caf/0.17.3.1@bincrafters/stable from 'conan-idb-third-party' - Cache
    catch2/2.2.2@bincrafters/stable from 'conan-idb-third-party' - Cache
    cppzmq/4.3.0@bincrafters/stable from 'conan-idb-third-party' - Cache
    jsoncpp/1.9.2 from 'conan-idb-third-party' - Cache
    libcurl/7.69.1 from 'conan-idb-third-party' - Cache
    libiconv/1.15 from 'conan-idb-third-party' - Cache
    libsodium/1.0.18 from 'conan-idb-third-party' - Cache
    llm/3.0.74.2887@llm/prod from 'ifi-conan-local' - Cache
    odbc/2.3.7 from 'conan-idb-third-party' - Cache
    openssl/1.1.1g from 'conan-idb-third-party' - Cache
    pistache/2f04981.1@idbdevbuild/stable from 'conan-idb-third-party' - Cache
    protobuf/3.9.1@bincrafters/stable from 'conan-idb-third-party' - Cache
    protoc_installer/3.9.1@bincrafters/stable from 'conan-idb-third-party' - Cache
    spdlog/0.16.3.1@idbdevbuild/stable from 'conan-idb-third-party' - Cache
    tbb/2020.1 from 'conan-idb-third-party' - Cache
    zlib/1.2.8 from 'conan-idb-third-party' - Cache
    zmq/4.3.2@bincrafters/stable from 'conan-idb-third-party' - Cache
Packages
    boost/1.72.0:0d14d262cdde8dde978c9dee82e8fb4c0ddf70c1 - Missing
    bzip2/1.0.8:a5fb342591cd091faa8b36194fe6a476ae195f44 - Cache
    caf/0.17.3.1@bincrafters/stable:6343484dfa90a2e1ec36a9397a8ce2a2ca53da04 - Missing
    catch2/2.2.2@bincrafters/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    cppzmq/4.3.0@bincrafters/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    jsoncpp/1.9.2:435f6d82215d51c7350154060d2d6634250003de - Cache
    libcurl/7.69.1:5bf21f754405cd5cdae4509e0ed60c17fc8ca929 - Missing
    libiconv/1.15:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3 - Build
    libsodium/1.0.18:6236a09dcd18d9f4591a2476c0f9c194e2d0a9d0 - Cache
    llm/3.0.74.2887@llm/prod:3981be4b18fe12bd3e6b15efb666da2494a3cf6d - Cache
    odbc/2.3.7:7d2ac76effb4a725f95b5eac45cf5694f8d5c1f2 - Missing
    openssl/1.1.1g:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3 - Cache
    pistache/2f04981.1@idbdevbuild/stable:ef58664316b93dde9b463dc888227d2113276c95 - Missing
    protobuf/3.9.1@bincrafters/stable:435f6d82215d51c7350154060d2d6634250003de - Cache
    protoc_installer/3.9.1@bincrafters/stable:c0c1ef10e3d0ded44179e28b669d6aed0277ca6a - Cache
    spdlog/0.16.3.1@idbdevbuild/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    tbb/2020.1:435f6d82215d51c7350154060d2d6634250003de - Missing
    zlib/1.2.8:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3 - Cache
    zmq/4.3.2@bincrafters/stable:acc65ea587795417f1e4875725a2d13dcc42a806 - Cache

Installing (downloading, building) binaries...
ERROR: Missing binary: boost/1.72.0:0d14d262cdde8dde978c9dee82e8fb4c0ddf70c1
ERROR: Missing binary: caf/0.17.3.1@bincrafters/stable:6343484dfa90a2e1ec36a9397a8ce2a2ca53da04
ERROR: Missing binary: libcurl/7.69.1:5bf21f754405cd5cdae4509e0ed60c17fc8ca929
ERROR: Missing binary: odbc/2.3.7:7d2ac76effb4a725f95b5eac45cf5694f8d5c1f2
ERROR: Missing binary: pistache/2f04981.1@idbdevbuild/stable:ef58664316b93dde9b463dc888227d2113276c95
ERROR: Missing binary: tbb/2020.1:435f6d82215d51c7350154060d2d6634250003de

tbb/2020.1: WARN: Can't find a 'tbb/2020.1' package for the specified settings, options and dependencies:
- Settings: arch=x86_64, build_type=Debug, compiler=gcc, compiler.libcxx=libstdc++, compiler.version=5, os=Linux
- Options: fPIC=True, shared=False, tbbmalloc=True, tbbproxy=False
- Dependencies:
- Requirements:
- Package ID: 435f6d82215d51c7350154060d2d6634250003de

ERROR: Missing prebuilt package for 'tbb/2020.1'
Try to build it from sources with "--build tbb"
Or read "http://docs.conan.io/en/latest/faq/troubleshooting.html#error-missing-prebuilt-package"

$ conan install .. --build tbb
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Debug
compiler=gcc
compiler.libcxx=libstdc++
compiler.version=5
os=Linux
os_build=Linux
[options]
[build_requires]
[env]

WARN: boost/1.72.0: requirement zlib/1.2.11 overridden by your conanfile to zlib/1.2.8
WARN: cppzmq/4.3.0@bincrafters/stable: requirement zmq/4.2.5@bincrafters/stable overridden by your conanfile to zmq/4.3.2@bincrafters/stable
WARN: libcurl/7.69.1: requirement openssl/1.1.1f overridden by your conanfile to openssl/1.1.1g
WARN: libcurl/7.69.1: requirement zlib/1.2.11 overridden by your conanfile to zlib/1.2.8
tbb/2020.1: WARN: Intel-TBB strongly discourages usage of static linkage
tbb/2020.1: Forced build from source
boost/1.72.0: WARN: Package is corrupted, removing folder: /userhome/epederson/.conan/data/boost/1.72.0/_/_/package/0d14d262cdde8dde978c9dee82e8fb4c0ddf70c1
conanfile.py: Installing package
Requirements
    boost/1.72.0 from 'conan-idb-third-party' - Cache
    bzip2/1.0.8 from local cache - Cache
    caf/0.17.3.1@bincrafters/stable from 'conan-idb-third-party' - Cache
    catch2/2.2.2@bincrafters/stable from 'conan-idb-third-party' - Cache
    cppzmq/4.3.0@bincrafters/stable from 'conan-idb-third-party' - Cache
    jsoncpp/1.9.2 from 'conan-idb-third-party' - Cache
    libcurl/7.69.1 from 'conan-idb-third-party' - Cache
    libiconv/1.15 from 'conan-idb-third-party' - Cache
    libsodium/1.0.18 from 'conan-idb-third-party' - Cache
    llm/3.0.74.2887@llm/prod from 'ifi-conan-local' - Cache
    odbc/2.3.7 from 'conan-idb-third-party' - Cache
    openssl/1.1.1g from 'conan-idb-third-party' - Cache
    pistache/2f04981.1@idbdevbuild/stable from 'conan-idb-third-party' - Cache
    protobuf/3.9.1@bincrafters/stable from 'conan-idb-third-party' - Cache
    protoc_installer/3.9.1@bincrafters/stable from 'conan-idb-third-party' - Cache
    spdlog/0.16.3.1@idbdevbuild/stable from 'conan-idb-third-party' - Cache
    tbb/2020.1 from 'conan-idb-third-party' - Cache
    zlib/1.2.8 from 'conan-idb-third-party' - Cache
    zmq/4.3.2@bincrafters/stable from 'conan-idb-third-party' - Cache
Packages
    boost/1.72.0:0d14d262cdde8dde978c9dee82e8fb4c0ddf70c1 - Missing
    bzip2/1.0.8:a5fb342591cd091faa8b36194fe6a476ae195f44 - Cache
    caf/0.17.3.1@bincrafters/stable:6343484dfa90a2e1ec36a9397a8ce2a2ca53da04 - Missing
    catch2/2.2.2@bincrafters/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    cppzmq/4.3.0@bincrafters/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    jsoncpp/1.9.2:435f6d82215d51c7350154060d2d6634250003de - Cache
    libcurl/7.69.1:5bf21f754405cd5cdae4509e0ed60c17fc8ca929 - Missing
    libiconv/1.15:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3 - Missing
    libsodium/1.0.18:6236a09dcd18d9f4591a2476c0f9c194e2d0a9d0 - Cache
    llm/3.0.74.2887@llm/prod:3981be4b18fe12bd3e6b15efb666da2494a3cf6d - Cache
    odbc/2.3.7:7d2ac76effb4a725f95b5eac45cf5694f8d5c1f2 - Missing
    openssl/1.1.1g:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3 - Cache
    pistache/2f04981.1@idbdevbuild/stable:ef58664316b93dde9b463dc888227d2113276c95 - Missing
    protobuf/3.9.1@bincrafters/stable:435f6d82215d51c7350154060d2d6634250003de - Cache
    protoc_installer/3.9.1@bincrafters/stable:c0c1ef10e3d0ded44179e28b669d6aed0277ca6a - Cache
    spdlog/0.16.3.1@idbdevbuild/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    tbb/2020.1:435f6d82215d51c7350154060d2d6634250003de - Build
    zlib/1.2.8:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3 - Cache
    zmq/4.3.2@bincrafters/stable:acc65ea587795417f1e4875725a2d13dcc42a806 - Cache

Installing (downloading, building) binaries...
ERROR: Missing binary: boost/1.72.0:0d14d262cdde8dde978c9dee82e8fb4c0ddf70c1
ERROR: Missing binary: caf/0.17.3.1@bincrafters/stable:6343484dfa90a2e1ec36a9397a8ce2a2ca53da04
ERROR: Missing binary: libcurl/7.69.1:5bf21f754405cd5cdae4509e0ed60c17fc8ca929
ERROR: Missing binary: libiconv/1.15:b1b85bfdc2206d81b3ed6a49e78498182bb98ef3
ERROR: Missing binary: odbc/2.3.7:7d2ac76effb4a725f95b5eac45cf5694f8d5c1f2
ERROR: Missing binary: pistache/2f04981.1@idbdevbuild/stable:ef58664316b93dde9b463dc888227d2113276c95

libiconv/1.15: WARN: Can't find a 'libiconv/1.15' package for the specified settings, options and dependencies:
- Settings: arch=x86_64, build_type=Debug, compiler=gcc, compiler.version=5, os=Linux
- Options: fPIC=True, shared=False
- Dependencies:
- Requirements:
- Package ID: b1b85bfdc2206d81b3ed6a49e78498182bb98ef3

ERROR: Missing prebuilt package for 'libiconv/1.15'
Try to build it from sources with "--build libiconv"
Or read "http://docs.conan.io/en/latest/faq/troubleshooting.html#error-missing-prebuilt-package"
feature
Source
picture sourcedelica

Most helpful comment

yes, maybe --build xxxx --build yyy if len(missing) <= 5 else --build missing

picture jgsogo on 7 May 2020
👍2 ❤1

All 6 comments

I would say this is more a feature than a bug.

Conan will not do a conan create or conan install with --build=xxx to later fail because there is some other missing package binary. Actually, it will not start downloading or do anything as long as it is known that the build will fail later with a "binary missing" error. This is according to a "fail fast" design, specially for Continuous Integration environments. You can have a CI job busy for a very long time downloading tons of packages, then building others, to finally completely fail your build because there is still some other missing binary that you did not specify to --build=package. For developers, indeed, if you are going to be issuing the command again and again each time with a missing binary package, I would say that --build=missing is indeed the way to go.

Lets see what the rest of the team thinks too.

memsharded picture memsharded on 6 May 2020

I also agree that this behavior is expected although it can be a little bit confusing because of the output of the command that one time is telling you that you must build one library and then others depending on the argument --build.

This could be something we could take into account for the CLI 2.0 regarding the install command explaining better in the output why conan is acting like this.

Some thoughts:

  • Maybe the output should be something like: _"You have lib1, lib2, lib3 missing, although you install lib2 you are still missing libraries and I will fail so use --build missing"_
  • Could it be a possibility to accept a list of libraries? Then we could use --build lib1 lib2 lib3 instead of --build missing so we could have "finer control" even if it could still fail because of a missing lib4 library. [EDIT] --> You can use a list with: --build lib1 --build lib2 etc...
czoido picture czoido on 7 May 2020

Having a look to the output, and this is something I've experienced myself:

$ conan install ..
...
ERROR: Missing prebuilt package for 'libiconv/1.15'
Try to build it from sources with "--build libiconv"

$ conan install .. --build libiconv
...
ERROR: Missing prebuilt package for 'tbb/2020.1'
Try to build it from sources with "--build tbb"

$ conan install .. --build tbb
...
ERROR: Missing prebuilt package for 'libiconv/1.15'
Try to build it from sources with "--build libiconv"

...endless loop 馃し

It happens the same when a missing binary has a missing build require, and it is quite frustrating from the UX perspective.

The easiest way to help with this is to list all the packages that need to be built:

ERROR: Missing prebuilt package for 'libiconv/1.15', `tbb/2020.1`[...]
Try to build it from sources with "--build libiconv --build tbb [...]"
jgsogo picture jgsogo on 7 May 2020
👍2

Yes, I kind of agree, I think changing the instructions would help. But what if the dependency graph is big? are we going to display lets say, 50 --build=xxx arguments? Maybe change it to --build=missing always?

memsharded picture memsharded on 7 May 2020

yes, maybe --build xxxx --build yyy if len(missing) <= 5 else --build missing

jgsogo picture jgsogo on 7 May 2020
👍2 ❤1

Same thing happened to me before, so I would also appreciate to list everything that is missing!

KerstinKeller picture KerstinKeller on 12 May 2020
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

mpdelbuono picture mpdelbuono  路  3Comments
kwint picture kwint  路  3Comments
db4 picture db4  路  3Comments
claasd picture claasd  路  3Comments
zlalanne picture zlalanne  路  3Comments