Xgboost: pip xgboost in virtualenv on Mac and got error code1

[wangli0519]

For bugs or installation issues, please provide the following information.
The more information you provide, the more easily we will be able to offer
help and advice.

Environment info

Operating System: Virtualenv on Mac Sierra

Compiler:

Package used (python/R/jvm/C++): Python 2.7

xgboost version used:

If installing from source, please provide

1. The commit hash (git rev-parse HEAD)
2. Logs will be helpful (If logs are large, please upload as attachment).

`Collecting xgboost
Using cached xgboost-0.6a2.tar.gz
Complete output from command python setup.py egg_info:
rm -f -rf build build_plugin lib bin _~ /_~ _/_/_~ */_/_/_~ _/_.o _/_/_.o */_/_/_.o xgboost
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
/bin/sh: clang-omp++: command not found
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
make: * [build/learner.o] Error 127
make: ** Waiting for unfinished jobs....
/bin/sh: clang-omp++: command not found
make: ** [build/logging.o] Error 127
-----------------------------
Building multi-thread xgboost failed
Start to build single-thread xgboost
rm -f -rf build build_plugin lib bin _~ */_~ _/_/_~ */_/_/_~ _/_.o _/_/_.o */_/_/_.o xgboost
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
/bin/sh: clang-omp++: command not found
make: ** [build/learner.o] Error 127
make: ** Waiting for unfinished jobs....
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
/bin/sh: clang-omp++: command not found
make: ** [build/logging.o] Error 127
Successfully build single-thread xgboost
If you want multi-threaded version
See additional instructions in doc/build.md
Traceback (most recent call last):
File "", line 1, in
File "/private/var/folders/kd/chcp2c9j5vz93c4fzft0dmnm0000gn/T/pip-build-Sbc1U5/xgboost/setup.py", line 29, in
LIB_PATH = libpath'find_lib_path'
File "/private/var/folders/kd/chcp2c9j5vz93c4fzft0dmnm0000gn/T/pip-build-Sbc1U5/xgboost/xgboost/libpath.py", line 45, in find_lib_path
'List of candidates:n' + ('n'.join(dll_path)))
**builtin.XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
List of candidates:
/private/var/folders/kd/chcp2c9j5vz93c4fzft0dmnm0000gn/T/pip-build-Sbc1U5/xgboost/xgboost/libxgboost.so
/private/var/folders/kd/chcp2c9j5vz93c4fzft0dmnm0000gn/T/pip-build-Sbc1U5/xgboost/xgboost/../../lib/libxgboost.so
/private/var/folders/kd/chcp2c9j5vz93c4fzft0dmnm0000gn/T/pip-build-Sbc1U5/xgboost/xgboost/./lib/libxgboost.so

Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/kd/chcp2c9j5vz93c4fzft0dmnm0000gn/T/pip-build-Sbc1U5/xgboost/

If you are using python package, please provide

1. The python version and distribution
2. The command to install xgboost if you are not installing from source
   pip install xgboost
   If you are using R package, please provide
3. The R sessionInfo()
4. The command to install xgboost if you are not installing from source

Steps to reproduce

1.
2.
3.

What have you tried?

1.
2.
3.

[phunterlau]

the log shows you don't have either gcc-5 or clang-omp++. Do you want to brew install gcc or brew install --with-clang llvm?

[wangli0519]

Hi, when I tried to brew install gcc, it shows: Warning: gcc-6.2.0 already installed

When I tried to change gcc in the Makefile as indicated by other post, I find:
--use customized config file
ifndef CC
export CC = $(if $(shell which gcc-6),gcc-6,gcc)
endif
ifndef CXX
export CXX = $(if $(shell which g++-6),g++-6,g++)
endif

--on Mac OS X, force brew gcc-6, since the Xcode c++ fails anyway
--it is useful for pip install compiling-on-the-fly
OS := $(shell uname)
ifeq ($(OS), Darwin)
export CC = $(if $(shell which gcc-6),gcc-6,clang)
export CXX = $(if $(shell which g++-6),g++-6,clang++)
endif

I also tried to make manually, but after make -j4, I got:
g++-6 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
g++-6 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
g++-6 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/c_api/c_api.o src/c_api/c_api.cc >build/c_api/c_api.d
g++-6 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/c_api/c_api_error.o src/c_api/c_api_error.cc >build/c_api/c_api_error.d
g++-6 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp src/c_api/c_api_error.cc -o build/c_api/c_api_error.o
In file included from src/c_api/./c_api_error.h:9:0,
from src/c_api/c_api_error.cc:6:
dmlc-core/include/dmlc/base.h:153:23: fatal error: sys/types.h: No such file or directory
#include
^
compilation terminated.
make: ** [build/c_api/c_api_error.o] Error 1
make: ** Waiting for unfinished jobs....
g++-6 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp src/logging.cc -o build/logging.o
In file included from dmlc-core/include/dmlc/logging.h:10:0,
from include/xgboost/logging.h:11,
from src/logging.cc:7:
/usr/local/Cellar/gcc/6.2.0/include/c++/6.2.0/cstdio:42:19: fatal error: stdio.h: No such file or directory
#include
^
compilation terminated.
make: ** [build/logging.o] Error 1
g++-6 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp src/c_api/c_api.cc -o build/c_api/c_api.o
In file included from include/xgboost/data.h:10:0,
from src/c_api/c_api.cc:3:
dmlc-core/include/dmlc/base.h:153:23: fatal error: sys/types.h: No such file or directory
#include
^
compilation terminated.
make: ** [build/c_api/c_api.o] Error 1
g++-6 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp src/learner.cc -o build/learner.o
In file included from dmlc-core/include/dmlc/logging.h:10:0,
from include/xgboost/logging.h:11,
from src/learner.cc:7:
/usr/local/Cellar/gcc/6.2.0/include/c++/6.2.0/cstdio:42:19: fatal error: stdio.h: No such file or directory
#include
^
compilation terminated.
make: *** [build/learner.o] Error 1

Sorry for the trouble to you and really appreciate your help.

[phunterlau]

@wangli0519 I think you need to run xcode-select --install otherwise the system's include path is not found.

[wangli0519]

@phunterlau Thank you for your reply. I got some warnings from the system and had to fix some minor issues, but basically it worked as you suggested. Thanks so much for your help.

[dice89]

@wangli0519 Can you please provide your solution, I would really appreciate it! Having the same troubles here.

[phunterlau]

@dice89 I think @wangli0519 had xcode-select --install since Mac asks for licence agreement firstly for xcode.

[dice89]

@phunterlau My issue was caused by different build scripts for the current HEAD from github and the version on pypi on 0.6a2 . For the pypi package you need somehow clang-omp++ as a compiler. In order to install this old package you have to run:

brew tap homebrew/boneyard
brew install clang-omp

Then pip install xgboost==0.6a2 runs without problems. Looking forward to the next version on pypi where you only need gcc-6 on OSx.

[phunterlau]

@dice89 that is true: at the time of building pip version, gcc from brew by default gives version 5. In the next round, I will update to version 6 and discard the outdated version 5.

[wangli0519]

@dice89 Hi, sorry that I just came back from a business trip. I believe @phunterlau has answered your question as he is an expert on this area. @dice89 Really sorry for my late response.

[xiaohongsimon]

same problem here, while "xcode-select --install" does not work, as "xcode-select: error: command line tools are already installed, use "Software Update" to install updates";

g++-6 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
g++-6 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
g++-6 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/c_api/c_api.o src/c_api/c_api.cc >build/c_api/c_api.d
g++-6 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/c_api/c_api_error.o src/c_api/c_api_error.cc >build/c_api/c_api_error.d
g++-6 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp src/c_api/c_api_error.cc -o build/c_api/c_api_error.o
g++-6 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp src/logging.cc -o build/logging.o
In file included from src/c_api/./c_api_error.h:9:0,
from src/c_api/c_api_error.cc:6:
dmlc-core/include/dmlc/base.h:153:23: fatal error: sys/types.h: No such file or directory
#include
^
compilation terminated.
In file included from dmlc-core/include/dmlc/logging.h:10:0,
from include/xgboost/logging.h:11,
from src/logging.cc:7:
/usr/local/Cellar/gcc/6.2.0/include/c++/6.2.0/cstdio:42:19: fatal error: stdio.h: No such file or directory
#include
^
compilation terminated.
make: ** [build/c_api/c_api_error.o] Error 1
make: ** Waiting for unfinished jobs....
make: ** [build/logging.o] Error 1
g++-6 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp src/c_api/c_api.cc -o build/c_api/c_api.o
In file included from include/xgboost/data.h:10:0,
from src/c_api/c_api.cc:3:
dmlc-core/include/dmlc/base.h:153:23: fatal error: sys/types.h: No such file or directory
#include
^
compilation terminated.
make: ** [build/c_api/c_api.o] Error 1
g++-6 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp src/learner.cc -o build/learner.o
In file included from dmlc-core/include/dmlc/logging.h:10:0,
from include/xgboost/logging.h:11,
from src/learner.cc:7:
/usr/local/Cellar/gcc/6.2.0/include/c++/6.2.0/cstdio:42:19: fatal error: stdio.h: No such file or directory
#include
^
compilation terminated.
make: *** [build/learner.o] Error 1

[xiaohongsimon]

i uninstalled gcc6, then it's OK now, guess something is wrong when i updated my mac system

[tingli81]

Hi, I think I have the same problem as above. I'm trying to install XG Boost using sudo pip install xgboost in a virtualenv.

But I keep getting an error. I tried adding xcode and gcc and clang as above. This is what I get.

~/Documents/Machine_Learning » sudo pip install xgboost ting@Tings-MacBook-Pro

The directory '/Users/ting/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/ting/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting xgboost
Downloading xgboost-0.6a2.tar.gz (1.2MB)
100% |████████████████████████████████| 1.2MB 944kB/s
Complete output from command python setup.py egg_info:
rm -f -rf build build_plugin lib bin ~ */~ //~ *///~ /.o //.o *///.o xgboost
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
/bin/sh: clang-omp++: command not found
make: * [build/learner.o] Error 127
make: Waiting for unfinished jobs....
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
/bin/sh: clang-omp++: command not found
make: [build/logging.o] Error 127
-----------------------------
Building multi-thread xgboost failed
Start to build single-thread xgboost
rm -f -rf build build_plugin lib bin ~ */~ //~ *///~ /.o //.o *///.o xgboost
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
/bin/sh: clang-omp++: command not found
make: [build/learner.o] Error 127
make: Waiting for unfinished jobs....
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
/bin/sh: clang-omp++: command not found
make: * [build/logging.o] Error 127
Successfully build single-thread xgboost
If you want multi-threaded version
See additional instructions in doc/build.md
Traceback (most recent call last):
File "", line 1, in
File "/private/tmp/pip-build-a2yx42fo/xgboost/setup.py", line 29, in
LIB_PATH = libpath'find_lib_path'
File "/private/tmp/pip-build-a2yx42fo/xgboost/xgboost/libpath.py", line 45, in find_lib_path
'List of candidates:n' + ('n'.join(dll_path)))
XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
List of candidates:
/private/tmp/pip-build-a2yx42fo/xgboost/xgboost/libxgboost.so
/private/tmp/pip-build-a2yx42fo/xgboost/xgboost/../../lib/libxgboost.so
/private/tmp/pip-build-a2yx42fo/xgboost/xgboost/./lib/libxgboost.so

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /private/tmp/pip-build-a2yx42fo/xgboost/