Conan-center-index: [boost] boost/1.71: bootstrap fails on ubuntu

[firewave]

Package and Environment Details

• Package Name/Version: boost/1.71
• Operating System+version: Linux Ubuntu 16.04/19.04
• Compiler+version: GCC 5.4.0/CLang 8
• Conan version: conan 1.18.0
• Python version: Python 3.6.x/3.7.x

Steps to reproduce

Starting with 1.71 the build now fail on all our ubuntu-based systems. The same systems can build 1.70 just fine. CentOS 7 is fine as well.

The command:
conan create /tmp/conanfile.py local/testing -o boost:bzip2=False -o boost:namespace=boost_ns -o boost:namespace_alias=True -o boost:magic_autolink=True --build outdated

On a side note - as seen in the log two of the patches are no longer necessary and should be removed.

Logs

Click to expand log

boost/1.71.0@local/testing: Calling build()
boost/1.71.0@local/testing: ./bootstrap.sh -with-toolset=cc
Bootstrapping the build engine with toolset cc... 

Failed to bootstrap the build engine
Consult 'bootstrap.log' for more details
boost/1.71.0@local/testing: 
WARN: patches/bcp_namespace_issues.patch: already patched  b'Jamroot'
WARN: patches/boost_core_qnx_cxx_provide___cxa_get_globals.patch: already patched  b'boost/core/uncaught_exceptions.hpp'
boost/1.71.0@local/testing: WARN: Error 256 while executing ./bootstrap.sh -with-toolset=cc
boost/1.71.0@local/testing: WARN: 
Unknown toolset: cc

You can specify the toolset as the argument, i.e.:
    ./build.sh gcc

Toolsets supported by this script are:
    acc, clang, como, gcc, intel-darwin, intel-linux, kcc, kylix, mipspro,
    pathscale, pgi, qcc, sun, sunpro, tru64cxx, vacpp

For any toolset you can override the path to the compiler with the CXX
environment variable. You can also use additional flags for the compiler
with the CXXFLAGS environment variable.

A special toolset; cxx, is available which is used as a fallback when a more
specific toolset is not found and the cxx command is detected. The 'cxx'
toolset will use the CXX, CXXFLAGS, and LIBS environment variables, if present.

Similarly, the cross-cxx toolset is available for cross-compiling by using the
BUILD_CXX, BUILD_CXXFLAGS, and BUILD_LDFLAGS environment variables to compile
binaries that will be executed on the build system. This allows CXX etc. to be
set for cross-compilers to be propagated to subprocesses.



boost/1.71.0@local/testing: ERROR: Package 'c89b33558f87fb078774818601cdb426f75abbe0' build failed
boost/1.71.0@local/testing: WARN: Build folder /home/parstream-ci.gen/.conan/data/boost/1.71.0/local/testing/build/c89b33558f87fb078774818601cdb426f75abbe0
ERROR: boost/1.71.0@local/testing: Error in build() method, line 401
    self._bootstrap()
while calling '_bootstrap', line 812
    self.run(cmd)
    ConanException: Error 256 while executing ./bootstrap.sh -with-toolset=cc

[SSE4]

/cc @grafikrobot
it seems like cc toolset was removed in 1.71, and we should use cxx (or cross-cxx) instead

[firewave]

I think it means, that it cannot find the cc binary because changing it causes this error:

boost/1.71.0@local/testing: ./bootstrap.sh -with-toolset=cxx
Bootstrapping the build engine with toolset cxx... 

Failed to bootstrap the build engine
Consult 'bootstrap.log' for more details
boost/1.71.0@local/testing: 
WARN: patches/bcp_namespace_issues.patch: already patched  b'Jamroot'
WARN: patches/boost_core_qnx_cxx_provide___cxa_get_globals.patch: already patched  b'boost/core/uncaught_exceptions.hpp'
boost/1.71.0@local/testing: WARN: Error 256 while executing ./bootstrap.sh -with-toolset=cxx
boost/1.71.0@local/testing: WARN: ###
###
### Using 'cxx' toolset.
###
###
cxx --version
./build.sh: 17: ./build.sh: cxx: not found

In the CentOS build it is using gcc

boost/1.71.0@local/testing: ./bootstrap.sh -with-toolset=gcc
Bootstrapping the build engine with toolset gcc... 

I think the actual problem is, that only g++ is installed on the ubuntu system - it's strange, that it worked with 1.70.

[SSE4]

@firewave
so, do I understand it correctly, you have g++ in PATH, but no gcc?
either way, we have to fix:

• use cxx toolset instead of cc for 1.71 and later
• check for clang++/g++ instead of clang/gcc (as boost uses C++ compiler)

[firewave]

This is not fixed.

boost/1.71.0@local/testing: WARN: Error 1 while executing ./bootstrap.sh -with-toolset=cxx
boost/1.71.0@local/testing: WARN: ###
###
### Using 'cxx' toolset.
###
###
cxx --version
./build.sh: 17: cxx: not found

boost/1.71.0@local/testing: ERROR: Package 'adb1ec51d6f4e61cf9fb0891ef0bd45babd829a7' build failed
boost/1.71.0@local/testing: WARN: Build folder /home/parstream-ci.gen/.conan/data/boost/1.71.0/local/testing/build/adb1ec51d6f4e61cf9fb0891ef0bd45babd829a7
ERROR: boost/1.71.0@local/testing: Error in build() method, line 403
    self._bootstrap()
while calling '_bootstrap', line 818
    self.run(cmd)
    ConanException: Error 1 while executing ./bootstrap.sh -with-toolset=cxx

As mentioned in a previous comment --wth-toolset= actually expects an existing binary.

[firewave]

The compiler settings I am using are CC=clang-9 and CXX=clang++-9.
I am using a ubuntu 19.04/19.10 docker image (ubuntu:disco and ubuntu:eoan) and only installed the clang-9 package .

The whole list of additionally installed packages: make g++ git wget python3-dev python3-setuptools python3-wheel devscripts zip clang-9 clang-tidy-9, gcovr python3-venv.

apt-get autoremove was also run on the image.

[mrks]

This is the only google result for ./build.sh: 17: ./build.sh: cxx: not found. I experienced the same without conan. For me, it was caused by running bootstrap.sh with sudo (to avoid the prefix). That caused CC and CXX not to be passed.

Running CC=gcc-9 CXX=g++-9 ./bootstrap.sh as root worked.

Also, you might want to try sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-9 90 --slave /usr/bin/clang++ clang++ /usr/bin/clang++-9.

[firewave]

This also fails with 1.70.0 when using the latest recipe.

[firewave]

Working:

boost/1.70.0@local/testing: ./bootstrap.sh -with-toolset=cc
Bootstrapping the build engine with toolset cc... engine/bin.linuxx86_64/b2

Bootstrapping is done. To build and install, run:

    ./b2 install --prefix=<DIR>

warning: No toolsets are configured.
warning: Configuring default toolset "gcc".
warning: If the default is wrong, your build may not work correctly.
warning: Use the "toolset=xxxxx" option to override our guess.
warning: For more configuration options, please consult
warning: http://boost.org/boost-build2/doc/html/bbv2/advanced/configuration.html

Not working (latest recipe):

boost/1.70.0@local/testing: ./bootstrap.sh -with-toolset=cc
Bootstrapping the build engine with toolset cc... engine/bin.linuxx86_64/b2

Bootstrapping is done. To build and install, run:

    ./b2 install --prefix=<DIR>

sh: 1: clang++: not found

[firewave]

I was able to get it to work with 1.70.0 by removing the toolset parameter from
https://github.com/conan-io/conan-center-index/blob/8a07dd95d9479f8998f77fdcd5d68a334b763778/recipes/boost/all/conanfile.py#L351

But this will obviously break #192 again...

[firewave]

Regrading 1.71.0 it seems like the cxx toolset is not working properly as it is being used as a command:

boost/1.71.0@local/testing: WARN: Error 1 while executing ./bootstrap.sh -with-toolset=cxx
boost/1.71.0@local/testing: WARN: ###
###
### Using 'cxx' toolset.
###
###
cxx --version
./build.sh: 17: cxx: not found

[firewave]

It indeed looks like a bug in the boost.

In 1.70.0 it was using the CC environment variable when it was set in case of the cc toolset:
https://github.com/boostorg/build/blob/32787ecf419a5f3d4af5d14955fcd166ae11c894/src/engine/build.sh#L270

In 1.71.0 it will always overwrite the CXX variable in case of the cxx toolset causing an invocation of cxx as command:
https://github.com/boostorg/build/blob/04efaac148671479b514db882b1c520dcaafe89e/src/engine/build.sh#L354

[firewave]

CXX is not set when it executes the code in build.sh.

It is set in my conan profile:

conan profile update env.CXX=clang++-9 default

All other recipes are working and use the set compiler.

[uilianries]

Indeed all our Docker images has CXX and CC configured for Clang profiles to avoid this kind of situation.

[firewave]

I added logging to the code which invokes the bootstrap and CXX is properly set. So this issue lies within boost.

[firewave]

Okay - look one line further I feel stupid now:

https://github.com/conan-io/conan-center-index/blob/71641ab6bb71d0ed48cae1fc1b85199dc07b2e1a/recipes/boost/all/conanfile.py#L832

Removing CXX from the environment breaks -with-toolset=cxx as that means "use CXX as toolset" - which it currently doesn't seem to use because of a bug introduced when -with-toolset replace cc with cxx...

The broken code in question is
https://github.com/boostorg/build/blob/ab5ce1e4686715bc83eeb28b809b9268d6cb2f63/src/engine/build.sh#L354

Edit: Turns out the boost is working just fine with -with-toolset=cxx when CXX is actually set.

I also think it is no longer necessary to filter CC and CFLAGS as those are no longer used by the bootstrap.

I think this has been working in the past with -with-toolset=cc since cc is an actual executable unlike cxx.

[blockspacer]

failed with conan profile:

[settings]
# We are building in Ubuntu Linux
os_build=Linux
os=Linux
arch_build=x86_64
arch=x86_64

compiler=clang
compiler.version=6.0
compiler.libcxx=libstdc++11

[env]
CC=/usr/bin/clang-6.0
CXX=/usr/bin/clang++-6.0

self.requires("boost/1.71.0@conan/stable")

export CC=clang-6.0
export CXX=clang++-6.0

$CC --version
$CXX --version

conan install --build=missing --profile clang -o enable_tests=False .

boost/1.71.0@conan/stable: Building your package in /home/denis/.conan/data/boost/1.71.0/conan/stable/build/e685f679210e62cd0cd889c871206164c704de92
boost/1.71.0@conan/stable: Generator txt created conanbuildinfo.txt
boost/1.71.0@conan/stable: Calling build()
boost/1.71.0@conan/stable: ./bootstrap.sh -with-toolset=cc
Bootstrapping the build engine with toolset cc... 

Failed to bootstrap the build engine
Consult 'bootstrap.log' for more details
boost/1.71.0@conan/stable: 
WARN: openssl/1.1.1c: requirement zlib/1.2.11 overridden by clang_folly_conan/v2019.01.14.00@conan/stable to zlib/1.2.11@conan/stable 
boost/1.71.0@conan/stable: WARN: Package is corrupted, removing folder: /home/denis/.conan/data/boost/1.71.0/conan/stable/package/e685f679210e62cd0cd889c871206164c704de92
cmake_platform_detection/master@conan/stable: WARN: Lib folder doesn't exist, can't collect libraries: /home/denis/.conan/data/cmake_platform_detection/master/conan/stable/package/dcd718216d5b2dabdcbc7cb602316ef046ffda71/lib
boost/1.71.0@conan/stable: WARN: Build folder is dirty, removing it: /home/denis/.conan/data/boost/1.71.0/conan/stable/build/e685f679210e62cd0cd889c871206164c704de92
boost/1.71.0@conan/stable: WARN: Error 1 while executing ./bootstrap.sh -with-toolset=cc
boost/1.71.0@conan/stable: WARN: 
Unknown toolset: cc

You can specify the toolset as the argument, i.e.:
    ./build.sh gcc

Toolsets supported by this script are:
    acc, clang, como, gcc, intel-darwin, intel-linux, kcc, kylix, mipspro,
    pathscale, pgi, qcc, sun, sunpro, tru64cxx, vacpp

For any toolset you can override the path to the compiler with the CXX
environment variable. You can also use additional flags for the compiler
with the CXXFLAGS environment variable.

A special toolset; cxx, is available which is used as a fallback when a more
specific toolset is not found and the cxx command is detected. The 'cxx'
toolset will use the CXX, CXXFLAGS, and LIBS environment variables, if present.

Similarly, the cross-cxx toolset is available for cross-compiling by using the
BUILD_CXX, BUILD_CXXFLAGS, and BUILD_LDFLAGS environment variables to compile
binaries that will be executed on the build system. This allows CXX etc. to be
set for cross-compilers to be propagated to subprocesses.


boost/1.71.0@conan/stable: ERROR: Package 'e685f679210e62cd0cd889c871206164c704de92' build failed
boost/1.71.0@conan/stable: WARN: Build folder /home/denis/.conan/data/boost/1.71.0/conan/stable/build/e685f679210e62cd0cd889c871206164c704de92
ERROR: boost/1.71.0@conan/stable: Error in build() method, line 415
    self._bootstrap()
while calling '_bootstrap', line 834
    self.run(cmd)
    ConanException: Error 1 while executing ./bootstrap.sh -with-toolset=cc
Elapsed time: 15 s. (time), 0.002071 s. (clock)

UPD: for some reason i'm not getting error above with CONAN_REVISIONS_ENABLED=1 CONAN_VERBOSE_TRACEBACK=1 CONAN_PRINT_RUN_COMMANDS=1 CONAN_LOGGING_LEVEL=10 GIT_SSL_NO_VERIFY=true conan install -s build_type=Debug --build=missing --profile clang -o enable_tests=False . also i tried to change CC=/usr/bin/clang-6.0 to CC=clang-6.0. Not sure why it helped

[SSE4]

the code that was failing has been completely removed in #1016, so these errors aren't going to happen any more - we're not doing bootstrap during the boost build. there is another bug (#1122) and pending PR #1124 that fixes b2 invocation on certain systems. I'll keep it open for a while, but once #1124 merged, pls retest and let me know (or just test changes locally).