xgboost 🚀 - pip install failure

Does not seem to work for me

ubuntu@ip-172-31-11-169:~$ sudo apt-get update
... yadda yadda
ubuntu@ip-172-31-11-169:~$ sudo apt-get install python-pip
Reading package lists... Done
Building dependency tree
Reading state information... Done
python-pip is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 74 not upgraded.
ubuntu@ip-172-31-11-169:~$ sudo pip install xgboost
Downloading/unpacking xgboost
Could not find a version that satisfies the requirement xgboost (from versions: 0.4a12, 0.4a13)
Cleaning up...
No distributions matching the version for xgboost
Storing debug log for failure in /home/ubuntu/.pip/pip.log
ubuntu@ip-172-31-11-169:~$

grfiv on 5 Sep 2015

The stackoverflow suggestion of --pre produced this:

ubuntu@ip-172-31-11-169:~$ sudo pip install --pre xgboost
Downloading/unpacking xgboost
Downloading xgboost-0.4a13.tar.gz (690kB): 690kB downloaded
Running setup.py (path:/tmp/pip_build_root/xgboost/setup.py) egg_info for package xgboost
xgboost/build-python.sh: 13: xgboost/build-python.sh: pushd: not found
make: * No rule to make target python'. Stop. ----------------------------- Building multi-thread xgboost failed Start to build single-thread xgboost make: *** No rule to make targetclean'. Stop.
make: ** No rule to make target `python'. Stop.
Successfully build single-thread xgboost
If you want multi-threaded version
See additional instructions in doc/build.md
xgboost/build-python.sh: 26: xgboost/build-python.sh: popd: not found
(None, None)
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip_build_root/xgboost/setup.py", line 20, in
import xgboost
File "./xgboost/init.py", line 8, in
from .core import DMatrix, Booster
File "./xgboost/core.py", line 78, in
_LIB = _load_lib()
File "./xgboost/core.py", line 72, in _load_lib
lib = ctypes.cdll.LoadLibrary(lib_path[0])
File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
return self._dlltype(name)
File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in *init
self._handle = _dlopen(self._name, mode)
OSError: /tmp/pip_build_root/xgboost/xgboost/./wrapper/libxgboostwrapper.so: invalid ELF header
Complete output from command python setup.py egg_info:
xgboost/build-python.sh: 13: xgboost/build-python.sh: pushd: not found

make: *** No rule to make target `python'. Stop.

Building multi-thread xgboost failed

Start to build single-thread xgboost

make: *** No rule to make target `clean'. Stop.

make: *** No rule to make target `python'. Stop.

Successfully build single-thread xgboost

If you want multi-threaded version

See additional instructions in doc/build.md

xgboost/build-python.sh: 26: xgboost/build-python.sh: popd: not found

(None, None)

Traceback (most recent call last):

File "", line 17, in

File "/tmp/pip_build_root/xgboost/setup.py", line 20, in

import xgboost

File "./xgboost/init.py", line 8, in

from .core import DMatrix, Booster

File "./xgboost/core.py", line 78, in

_LIB = _load_lib()

File "./xgboost/core.py", line 72, in _load_lib

lib = ctypes.cdll.LoadLibrary(lib_path[0])

File "/usr/lib/python2.7/ctypes/init.py", line 443, in LoadLibrary

return self._dlltype(name)

File "/usr/lib/python2.7/ctypes/init.py", line 365, in init

self._handle = _dlopen(self._name, mode)

OSError: /tmp/pip_build_root/xgboost/xgboost/./wrapper/libxgboostwrapper.so: invalid ELF header

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/xgboost
Storing debug log for failure in /home/ubuntu/.pip/pip.log
ubuntu@ip-172-31-11-169:~$

grfiv on 5 Sep 2015

@cliveseldon @grfiv the versioning of pip installation has some confusions, since the official version 0.4 has been there for a long time, and the pip installation contains some recent update from the source code, so I have named the version with 0.4a12 0.4a13 something. It will be corrected later from the next official release.

A good solution for updating pip installation can be, deleting the xgboost directory and the xgboost-0.4a13.dist-info directory both from the site package directory, e.g. /User/phunterlau/python2.7/site-packages/ (the installation location can be found from print xgboost.__file__ ), and redo 'pip install xgboost' . Thanks.

phunterlau on 18 Sep 2015

Hello,
I have the same problem on my ubuntu box:

To run :

sudo -H pip install --pre xgboost
python -c "import xgboost"

Fails as follows:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/xgboost/__init__.py", line 11, in <module>
    from .core import DMatrix, Booster
  File "/usr/local/lib/python2.7/dist-packages/xgboost/core.py", line 92, in <module>
    _LIB = _load_lib()
  File "/usr/local/lib/python2.7/dist-packages/xgboost/core.py", line 86, in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python2.7/dist-packages/xgboost/./wrapper/libxgboostwrapper.so: invalid ELF header

I've also tried without --pre, it fails identically.

Any clue ?
Thanks,
F.

Fkawala on 23 Sep 2015

I used pip install xgboost to install xgboost-0.4a14 on a mac OSX. When I tried loading it I got this error.

OSError Traceback (most recent call last)
in ()
----> 1 import xgboost as xgb

/Users/azaitzeff/anaconda/lib/python2.7/site-packages/xgboost/init.py in ()
9 import os
10
---> 11 from .core import DMatrix, Booster
12 from .training import train, cv
13 from .sklearn import XGBModel, XGBClassifier, XGBRegressor

/Users/azaitzeff/anaconda/lib/python2.7/site-packages/xgboost/core.py in ()
90
91 # load the XGBoost library globally
---> 92 _LIB = _load_lib()
93
94 def _check_call(ret):

/Users/azaitzeff/anaconda/lib/python2.7/site-packages/xgboost/core.py in _load_lib()
84 if len(lib_path) == 0:
85 return None
---> 86 lib = ctypes.cdll.LoadLibrary(lib_path[0])
87 lib.XGBGetLastError.restype = ctypes.c_char_p
88 return lib

/Users/azaitzeff/anaconda/lib/python2.7/ctypes/init.pyc in LoadLibrary(self, name)
441
442 def LoadLibrary(self, name):
--> 443 return self._dlltype(name)
444
445 cdll = LibraryLoader(CDLL)

/Users/azaitzeff/anaconda/lib/python2.7/ctypes/init.pyc in init(self, name, mode, handle, use_errno, use_last_error)
363
364 if handle is None:
--> 365 self._handle = _dlopen(self._name, mode)
366 else:
367 self._handle = handle

OSError: dlopen(/Users/azaitzeff/anaconda/lib/python2.7/site-packages/xgboost/./wrapper/libxgboostwrapper.so, 6): Library not loaded: /usr/local/lib/gcc/4.9/libgomp.1.dylib
Referenced from: /Users/azaitzeff/anaconda/lib/python2.7/site-packages/xgboost/./wrapper/libxgboostwrapper.so
Reason: image not found

Any help. Thanks
AZ

AZaitzeff on 23 Sep 2015

@AZaitzeff @Fkawala Thanks. Can you confirm if you have gcc installed on your ubuntu or Mac? With your information, I see the problem of precompiled lib: if there is no gcc installed on the system, the build script in pip would fail. However, in the pip installation package, I included a precompiled lib libxgboostwrapper.so using gcc 4.9 from brew in Mac, so if the build script failed, xgboost will search for this precompiled lib. Linux doesn't recognize this Mac lib, so it has ELF code error; For mac with no OpenMP, it will trigger the link error. Now I see it causes some confusions and it might not be a good idea. I will update the pip installation, and display the message if there is no gcc. P.S. We will have precompiled and static linked lib in the next official release, which is a better solution of shipping precompiled code.

phunterlau on 23 Sep 2015

@phunterlau Thanks. I do not have OpenMP on my Mac. So that is what was causing the error.

AZaitzeff on 23 Sep 2015

@phunterlau ok, I'll set up a proper compilation environment and test the pip installation again.

Thanks for the quick reply.

Fkawala on 23 Sep 2015

So I ran brew install clang-omp and it installed. Deleted the xgboost directory and the xgboost-0.4a13.dist-info and reran pip install xgboost. Closed my terminal windows and reopened them. But I am still getting the same error when trying to import xgboost. Any thoughts?

AZaitzeff on 23 Sep 2015

@AZaitzeff actually clang-omp is not recommended, since clang from Xcode doesn't support it so clang-omp may give some strange result. The mac pip installation package was created and tested on gcc 4.9 from brew with OpenMP. Do you want to try with brew install gcc49? Now we have gcc 5 from brew too, but I haven't had a chance for fully test it. I will do it in the next pip upload.

phunterlau on 23 Sep 2015

Thanks that worked great.

AZaitzeff on 24 Sep 2015

@phunterlau the build (using ./xgboost/build-python.sh) works like a charm. However, the pip installation results in the pre-compiled lib. I modified setup.py to show why the compilation is not attempted. With PIP 7.1.2 (python 2.7), to run:

pip install --user -e ./tweaked-xgboost-0.4a13/

gives:

'pip' in __file__: False
not os.name == 'nt': True

Fkawala on 24 Sep 2015

According to http://stackoverflow.com/questions/9959279/detect-pip-in-setup-py [1] the 'pip' in __file__ test does not work with Py 2.7 2014-10-27 nor with custom installation (as I used in my previous comment). As proposed in [1] 'pip' in os.environ.get('_') is a more standard way to test if pip is runs the setup.py.

Fkawala on 24 Sep 2015

@Fkawala Thanks. 'pip' in __file__ was an old trick from long time ago, nice to know the new one. I will update the pip installation and later on send a PR (with some other patches).

phunterlau on 24 Sep 2015

@phunterlau I have similar problem, didn't understand how to fix this ELF error. I'm working on ubuntu and can't import xgboost package after installing it with pip. You wrote about gcc, I installed gcc-4.9, but still don't work, what I'm missing?

Here is my error:

import xgboost

OSError Traceback (most recent call last)
in ()
----> 1 import xgboost
/home/antklen/anaconda/lib/python2.7/site-packages/xgboost/init.py in ()
9 import os
10
---> 11 from .core import DMatrix, Booster
12 from .training import train, cv
13 from .sklearn import XGBModel, XGBClassifier, XGBRegressor
/home/antklen/anaconda/lib/python2.7/site-packages/xgboost/core.py in ()
90
91 # load the XGBoost library globally
---> 92 _LIB = _load_lib()
93
94 def _check_call(ret):
/home/antklen/anaconda/lib/python2.7/site-packages/xgboost/core.py in _load_lib()
84 if len(lib_path) == 0:
85 return None
---> 86 lib = ctypes.cdll.LoadLibrary(lib_path[0])
87 lib.XGBGetLastError.restype = ctypes.c_char_p
88 return lib
/home/antklen/anaconda/lib/python2.7/ctypes/__init__.pyc in LoadLibrary(self, name)
441
442 def LoadLibrary(self, name):
--> 443 return self._dlltype(name)
444
445 cdll = LibraryLoader(CDLL)
/home/antklen/anaconda/lib/python2.7/ctypes/__init__.pyc in init(self, name, mode, handle, use_errno, use_last_error)
363
364 if handle is None:
--> 365 self._handle = _dlopen(self._name, mode)
366 else:
367 self._handle = handle
OSError: /home/antklen/anaconda/lib/python2.7/site-packages/xgboost/./wrapper/libxgboostwrapper.so: invalid ELF header

antklen on 11 Oct 2015

@antklen in windows you need to install setuptools first. i'm not sure if that's the case for ubuntu, but maybe worth trying?

smargs on 11 Oct 2015

@antklen it is because the current pip installation is a on-the-fly compiler script and it ships with a Mac version of .so file, in case some mac users didn't install g++ for compilation. It turned out not a good idea and caused some confusions for Linux users.

The solution is that, please install gcc and g++ apt-get install g++, so it will pickup the compiling script.

And, if your pip is newer than 0.7.1.2, it may have another problem of checking if pip is running. I will fix and ship it in the next release. If you have g++ but still the same error, please install from setup.py manually by following the installation document.

phunterlau on 12 Oct 2015

@smargsongithub @phunterlau Thank you for replies!
apt-get install g++ also didn't work for me, but I succeed with manual installation as described here https://www.kaggle.com/c/springleaf-marketing-response/forums/t/16320/cloud-vs-local-machine-pros-and-cons
I had some issues beacause I'm using Anaconda, so my final recipe is

sudo apt-get install make
sudo apt-get update
sudo apt-get install gcc
sudo apt-get install g++
sudo apt-get install git
sudo git clone https://github.com/dmlc/xgboost
cd xgboost
sudo ./build.sh
cd python-package
sudo /home/username/anaconda/bin/python setup.py install

antklen on 12 Oct 2015

👍9 😄1

@antklen oh, you need sudo when install g++ :-)

phunterlau on 13 Oct 2015

@Fkawala I am trying 'pip' in os.environ.get('_') for the updated pip, but os.environ.get('_') always returns None and this if statement gives error. Do we have other method for detecting if pip in the command line? Otherwise, I would create a separate setup.py just for pip and trigger the compiler there. Thanks.

phunterlau on 18 Oct 2015

@phunterlau I am getting the following traceback from pip install xgboost, any thoughts? (running a Mac)

Traceback (most recent call last):
File "", line 20, in
File "/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/setup.py", line 38, in
LIB_PATH = libpath'find_lib_path'
File "/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/xgboost/libpath.py", line 47, in find_lib_path
'List of candidates:n' + ('n'.join(dll_path)))
builtin.XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you run build.sh in root path?
List of candidates:
/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/xgboost/libxgboostwrapper.so
/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/xgboost/../../wrapper/libxgboostwrapper.so
/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/xgboost/./wrapper/libxgboostwrapper.so
/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/xgboost/../../../xgboost/wrapper/libxgboostwrapper.so

Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost

pcohen89 on 24 Oct 2015

@pcohen89 this python package ships with C++ code and compiles on-the-fly, so you might need a complier. Do you have your compiler installed, e.g. gcc-5 or clang LLVM from Xcode? Mac doesn't install xcode command line tools by default and you can follow this instruction http://railsapps.github.io/xcode-command-line-tools.html or, another better choice is installing homebrew http://brew.sh/ and brew install g++-5. The later option (g++-5) gives multi-thread OpenMP support.

After installing compilers, please remove the old xgboost installation, and redo pip install xgboost

phunterlau on 25 Oct 2015

@antklen @phunterlau Tried following the steps mentioned by @antklen . Now, I am able to run import xgboost on terminal. However on anacondas ipython notebook it gives me the below error. Any help please. I am using ubuntu.

XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you run build.sh in root path?
List of candidates:
/home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/xgboost/libxgboostwrapper.so
/home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/xgboost/../../wrapper/libxgboostwrapper.so
/home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/xgboost/./wrapper/libxgboostwrapper.so

jeechu on 26 Oct 2015

@Jeechu that means xgboost was not compiled, have you:

removed the old xgboost installation?
installed xcode (mac), or gcc-5 (Mac uses brew, linux use apt-get or yum)?

pip install xgboost -v can tell more about error message.

phunterlau on 26 Oct 2015

@phunterlau me and @Jeechu we have removed the old xgboost installation and installed gcc,g++. XGBoost is running fine via terminal, but using anaconda ipython notebook it is giving that error.

Is there something wrong with the installation path?

israjsingh on 26 Oct 2015

@rajdeltarobo @Jeechu what is the python version of your ipython notebook? and, what is your $PYTHONPATH variables in the .bashrc (or other env files from your shells)?

Seems like your xgboost is search for /home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/ which is not a pip installation path (pip installation has its additional version number e.g. xgboost-0.4a24), so the ipython notebook tries searching your xgboost from python setup.py install from github, thus the problem may come from $PYTHONPATH

phunterlau on 26 Oct 2015

@phunterlau $PYTHONPATH is not defined, its returning a blank. However, below is the value of path

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/simpl/anaconda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

The python path is

➜  ~  which python  
/usr/bin/python

The python version of ipython notebook is 2.7

israjsingh on 26 Oct 2015

How about adding something like this to your PYTHONPATH:
/Users/simpl/anaconda/lib/python2.7/site-packages/

if you want to use anaconda's python, just stay with it for everything, otherwise use virtualenv instead of having multiple python installations. It is very confusing if there are multiple python installations on the the same machine.

phunterlau on 26 Oct 2015

@phunterlau Not working. I am using ubuntu so added

/home/user/anaconda/lib/python2.7/site-packages/

Although, I can see this at the end of Bashrc file

# added by Anaconda 2.3.0 installer
export PATH="/home/user/anaconda/bin:$PATH"

israjsingh on 26 Oct 2015

@phunterlau I had installed gcc and thought that was sufficient. Got Xcode and it all worked. Thank you all for your support on Xgboost, it's awesome and you all are super helpful.

pcohen89 on 26 Oct 2015

@pcohen89 nice to know xcode helped. You might find xcode clang only support single thread xgboost, so you might want to try brew install g++-5 for multithread support. anyway, a working copy is a good copy.

@rajdeltarobo PATH is different from PYTHONPATH, where PYTHONPATH allows python to find the customized package installation. Please add it and source your bashrc.

phunterlau on 26 Oct 2015

copy /wrapper/libxgboostwrapper.so to /home/username/anaconda3/lib/python3.4/site-packages/xgboost-0.4-py3.4.egg/xgboost/ solved the problem for me (i am using Ubuntu 14.04)

my problem was :

XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you run build.sh in root path?
List of candidates:
/home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/xgboost/libxgboostwrapper.so
/home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/xgboost/../../wrapper/libxgboostwrapper.so
/home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/xgboost/./wrapper/libxgboostwrapper.so

davutpolat on 29 Oct 2015

@davutpolat the error from your message comes from a recent change for conflict-resolving for pip installation, but turned out that change was not a good idea, so we rolled it back (PR #572). In future, this copy is not needed and we will have another separated setup configuration only for pip installation with compiling on the fly. Sorry for the inconvenience for the last couple of days.

phunterlau on 29 Oct 2015

I'm attempting to install xgboost through pip but fail with the same error as @davutpolat had. I wanted to use his solution, but I have no idea as where to find /wrapper/libxgboostwrapper.so.

Claygirl on 25 Dec 2015

@claygirl do you have gcc or clang in your system? please also tell your system env and error message

phunterlau on 26 Dec 2015

Solution. I had same OSError: /usr/local/lib/libxgboostwrapper.so: cannot open shared object file: No such file or directory. I copied the
site-packages/xgboost-0.4-py2.7.egg/xgboost/./wrapper/libxgboostwrapper.so to /usr/local/lib/ and seems working.
But not happy with it as I have both 2.7 and 3.4. ..Any new resolution?

dsivakumar on 6 Jan 2016

@dsivakumar site-packages/xgboost-0.4-py2.7.egg/xgboost/./wrapper is the expected location for libxgboostwrapper.so if install with pip, and xgboost/lib_path.py has included this path so import xgboost can easily find it. If you want both 2.7 and 3.4, please use virtualenv and install with pip separately, since pip depends on python versions.

phunterlau on 6 Jan 2016

@phunterlau Thanks, that seems the only solution.

dsivakumar on 7 Jan 2016

I tried several times but lost hope to install xgboost on my windows OS PC,
but now may be you will help what to do
it is the error
C:UsersSander>pip install xgboost
Collecting xgboost
Downloading xgboost-0.4a30.tar.gz (753kB)
100% |################################| 753kB 397kB/s
No files/directories in
c:userssanderappdatalocaltemppip-build-gqsijfxgbo
ostpip-egg-info (from PKG-INFO)
I use anaconda python installation
I remember in one of kaggle competition, somebody failed install xgboos and
chooses another technique and he got 2 place...

On Thu, Jan 7, 2016 at 3:31 AM, dsivakumar [email protected] wrote:

@phunterlau https://github.com/phunterlau Thanks, that seems the only
solution.

—
Reply to this email directly or view it on GitHub
https://github.com/dmlc/xgboost/issues/463#issuecomment-169595600.

Sandy4321 on 7 Jan 2016

😄2

@Sandy4321 Windows doesn't support pip installation now because it was not set for VS command line compiler, as mentioned in the installation guide and trouble shooting. Please use the VS solution file in windows/ to compile and install. Hope it can help you get 2nd place too :-)

phunterlau on 7 Jan 2016

I'm trying to installe Xgboost on Mac 10.10.5 without Xcode. I firstly installed GCC 5.3 with openmp support by brew install gcc --without-multilib. Secondly, I build xgboost by

git clone --recursive https://github.com/dmlc/xgboost
cd xgboost
export CC=/usr/local/bin/gcc-5
export CXX=/usr/local/bin/g++-5
cp make/config.mk ./config.mk
make -j4

Then error appears, as below,
make: ** [lib/libxgboost.a] Error 1
make: ** Waiting for unfinished jobs....
clangclang: : warning: warning: argument unused during compilation: '-pthread'
argument unused during compilation: '-pthread'
ld: library not found for -lgomp
ld: library not found for -lgomp
clang: error: linker command failed with exit code 1 (use -v to see invocation)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: ** [lib/libxgboost.so] Error 1
make: ** [xgboost] Error 1

I ignored the error and kept on going. I typed in codes as below
sudo ./build.sh

Then it said
clang: warning: argument unused during compilation: '-pthread'
Successfully build single-thread xgboost

Would you pls tell me why it still a single-thread Xgboost although I have installed GCC 5 with OpenMP.
What should I do to change the Xgboost I had installed into a multi-threaded version?

LiyanJin on 2 Apr 2016

the recent makefile works better with clang-omp.

phunterlau on 2 Apr 2016

你好，非常感谢你的解答。
你的意思是现在xgboost不用gcc，而用clang来做编译器也能支持multi threaded，是吗？由于MAC 10.9以上版本自带了clang，所以我是否还需要从homebrew上下载clang-omp
我是新手，之前是根据这个网站上的方法在安装xgboost，现在是否有更新的安装指南，能告诉我具体如何配置吗？
另外，改用clang-omp后，是否要先删除已经安装好的xgboost，然后再重装一次xgboost呢？
谢谢！

At 2016-04-03 01:34:37, "Hongliang Liu" [email protected] wrote:

the recent makefile works better with clang-omp.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

LiyanJin on 3 Apr 2016

Thx for your quickly reply. According to your suggestion ,I have downloaded clang-omp via homebrew.
Then, what's next step for me? Whether should I remove the current xgboost and reinstall it instead.

LiyanJin on 3 Apr 2016

please change the default compiler in config.mk to clang-omp. Please follow the build instruction Mac OS X part.

phunterlau on 3 Apr 2016

Thx for ur instruction. And I changed the default compiler in config.mk to clang-omp like this
` export CC = /usr/local/bin/clang-omp

export CXX = /usr/local/bin/clang-omp++

ADD_CFLAGS = -fopenmp`

Then I compile by
make -j8

However, error appears again:
make: ** [lib/libxgboost.a] Error 1
make: ** Waiting for unfinished jobs....
clang-3.5clang-3.5: : errorerror: : no such file or directory: 'rabit/lib/librabit.a'no such file or directory: 'rabit/lib/librabit.a'

make: ** [lib/libxgboost.so] Error 1
make: ** [xgboost] Error 1

Is there anything wrong for me?

LiyanJin on 4 Apr 2016

And I run
sudo ./build.sh
error appears as below

include/dmlc/omp.h:13:9: warning: Warning: OpenMP is not available, project will
be compiled into single-thread code. Use OpenMP-enabled compiler to get
benefit of multi-threading. [-W#pragma-messages]

pragma message("Warning: OpenMP is not available, "

LiyanJin on 4 Apr 2016

please try reinstalling clang-omp by brew remove clang-omp; brew install clang-omp; only change these two lines (no ADD_FLAG) in config.mk, and make sure to copy it to the same directory as Makefile is.

export CC = clang-omp
export CXX = clang-omp++

and run make

phunterlau on 7 Apr 2016

👍4

Using Windows 7. Anaconda from yhat Rodeo.

Trying to install xgboost and getting the following error. Please help.

`! pip install xgboost

Collecting xgboost
Using cached xgboost-0.4a30.tar.gz
No files/directories in C:UsersUSERAppDataLocalTemppip-build-dkmqc8n1xgboostpip-egg-info (from PKG-INFO)`

boral on 11 May 2016

👍1

@phunterlau Thx for your suggestion. I tried your instruction and getting the following error. Please give some suggestion.
I reinstalled clang-imp and changed two lines (no ADD_FLAG) in config.mk in the same directory as Makefile is

export CC = clang-omp
export CXX = clang-omp++

Then I run sudo make -j4 and got the following error.

ar crv lib/libxgboost.a 
ar: no archive members specified
usage:  ar -d [-TLsv] archive file ...
    ar -m [-TLsv] archive file ...
    ar -m [-abiTLsv] position archive file ...
    ar -p [-TLsv] archive [file ...]
    ar -q [-cTLsv] archive file ...
    ar -r [-cuTLsv] archive file ...
    ar -r [-abciuTLsv] position archive file ...
    ar -t [-TLsv] archive [file ...]
    ar -x [-ouTLsv] archive [file ...]
make: *** [lib/libxgboost.a] Error 1
make: *** Waiting for unfinished jobs....
clang-3.5: warning: argument unused during compilation: '-pthread'
clang-3.5: warning: argument unused during compilation: '-pthread'

Then I run sudo ./build.sh and get the following error

ar crv lib/libxgboost.a 
ar: no archive members specified
usage:  ar -d [-TLsv] archive file ...
    ar -m [-TLsv] archive file ...
    ar -m [-abiTLsv] position archive file ...
    ar -p [-TLsv] archive [file ...]
    ar -q [-cTLsv] archive file ...
    ar -r [-cuTLsv] archive file ...
    ar -r [-abciuTLsv] position archive file ...
    ar -t [-TLsv] archive [file ...]
    ar -x [-ouTLsv] archive [file ...]
make: *** [lib/libxgboost.a] Error 1
-----------------------------
Building multi-thread xgboost failed

LiyanJin on 15 May 2016

Hi everyone,
To install Xgboost on Windows using Python follow the instructions in the following thread.

http://stackoverflow.com/questions/33749735/how-to-install-xgboost-package-in-python-windows-platform

boral on 15 May 2016

👍1

@boral yep, the current pip install doesn't support window platform. glad that you have figured out

phunterlau on 15 May 2016

@liyanjin you can just do 'make xgboost' for the core part. to have ar, you might need to install ar for mac

phunterlau on 15 May 2016

I am facing similar issue. My error is quite

OSError: dlopen(/Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so, 6): Library not loaded: @rpath/./libgomp.1.dylib
  Referenced from: /Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so
  Reason: image not found

I checked I have gcc installed.
$ brew install gcc49
Warning: homebrew/versions/gcc49-4.9.3 already installed

I am on mac os I installed xgboost through anaconda

lalmanisha on 29 Jun 2016

@lalmanisha I believe you are using Mac. The current best practice on Mac is using clang-omp instead of gcc. Can you try installing clang-omp from brew and change the CC and CXX in the config.mk to clang-omp?

phunterlau on 29 Jun 2016

Thanks for your reply. I still get the same error message. Here is the
`--------------------------------------------------------------------------
OSError Traceback (most recent call last)
in ()
14 # Any results you write to the current directory are saved as output.
15
---> 16 import xgboost as xgb
17 from sklearn.linear_model import LogisticRegression
18 import pandas as pd

/Users/mlal/anaconda/lib/python2.7/site-packages/xgboost.py in ()
80
81 # load the XGBoost library globally
---> 82 xglib = load_xglib()
83
84

/Users/mlal/anaconda/lib/python2.7/site-packages/xgboost.py in load_xglib()
57 if len(dll_path) == 0:
58 raise XGBoostLibraryNotFound('cannot find find the files in the candicate path ' + str(dll_path))
---> 59 lib = ctypes.cdll.LoadLibrary(lib_path[0])
60
61 # DMatrix functions

/Users/mlal/anaconda/lib/python2.7/ctypes/init.pyc in LoadLibrary(self, name)
441
442 def LoadLibrary(self, name):
--> 443 return self._dlltype(name)
444
445 cdll = LibraryLoader(CDLL)

/Users/mlal/anaconda/lib/python2.7/ctypes/init.pyc in init(self, name, mode, handle, use_errno, use_last_error)
363
364 if handle is None:
--> 365 self._handle = _dlopen(self._name, mode)
366 else:
367 self._handle = handle

OSError: dlopen(/Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so, 6): Library not loaded: @rpath/./libgomp.1.dylib
Referenced from: /Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so
Reason: image not found
`
i did change config.mk file as below

choice of compiler
export CC = clang-omp
export CXX = clang-omp
export MPICXX = mpicxx

lalmanisha on 29 Jun 2016

sudo make -j4
ar crv lib/libxgboost.a
ar: no archive members specified
usage: ar -d [-TLsv] archive file ...
ar -m [-TLsv] archive file ...
ar -m [-abiTLsv] position archive file ...
ar -p [-TLsv] archive [file ...]
ar -q [-cTLsv] archive file ...
ar -r [-cuTLsv] archive file ...
ar -r [-abciuTLsv] position archive file ...
ar -t [-TLsv] archive [file ...]
ar -x [-ouTLsv] archive [file ...]
make: ** [lib/libxgboost.a] Error 1
make: ** Waiting for unfinished jobs....
clang: clang: error: error: unsupported option '-fopenmp'unsupported option '-fopenmp'

make: ** [lib/libxgboost.so] Error 1
make: ** [xgboost] Error 1
When i ran
sudo ./build.sh

maker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit_empty.a -pthread -lm
clang: warning: argument unused during compilation: '-pthread'
Successfully build single-thread xgboost
If you want multi-threaded version
See additional instructions in doc/build.md

I still get same error
dlopen(/Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so, 6): Library not loaded: @rpath/./libgomp.1.dylib
Referenced from: /Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so
Reason: image not found

lalmanisha on 29 Jun 2016

ok, I guess combination of above some help and build.md helped me make xgboost work. Thank you all !

lalmanisha on 29 Jun 2016

@lalmanisha since you only need the python package, you can just make xgboost which removes other unnecessary packages like the parallel and yarn version which needs ar. It should have successful make. And glad that you have figured it out.

phunterlau on 30 Jun 2016

Try to install Xgboost with git clone on MAC. I follow all instruction from @LiyanJin until this answer, I downloaded clang-omp via homebrew to use multi-threading.

config.mk :

export CC = clang-omp
export CXX = clang-omp++
export MPICXX = mpicxx

Error :

clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude -fopenmp  -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
In file included from src/learner.cc:7:
In file included from include/xgboost/logging.h:11:
In file included from dmlc-core/include/dmlc/logging.h:10:
/usr/local/Cellar/clang-omp/2015-04-01/libexec/bin/../include/c++/v1/cstdio:100:10: fatal error: 'stdio.h' file not found
#include <stdio.h>
         ^
1 error generated.
make: *** [build/learner.o] Error 1

pip install works well, but I would like to have the lastest features in python package.

Some tips ?

Edit : Find the solution. Just xcode-select --install and make. Hope it will help other.

armgilles on 2 Aug 2016

👍3

@armgilles that is right: by default mac hasn't install the compilers with headers while clang-omp needs the local installed Xcode, installing it by xcode-select --install everything after upgrade to a new system can solve the problem.

phunterlau on 2 Aug 2016

👍1

I had to build it from scratch:

git clone --recursive https://github.com/dmlc/xgboost.git  
cd xgboost  
./build.sh
pip install -e python-package

hnykda on 21 Oct 2016

👍28 🎉13 ❤7 😄3

hnykda suggestion worked for me on Ubuntu 16.0.4

wuatanabe on 29 Jan 2017

@phunterlau
I got the same error.
ar crv lib/libxgboost.a
ar: no archive members specified
usage: ar -d [-TLsv] archive file ...
ar -m [-TLsv] archive file ...
ar -m [-abiTLsv] position archive file ...
ar -p [-TLsv] archive [file ...]
ar -q [-cTLsv] archive file ...
ar -r [-cuTLsv] archive file ...
ar -r [-abciuTLsv] position archive file ...
ar -t [-TLsv] archive [file ...]
ar -x [-ouTLsv] archive [file ...]
make: * [lib/libxgboost.a] Error 1

i have installed ar. i am wondering what the "ar crv" do,if i can use other command to replace.
you have mentioned "@liyanjin you can just do 'make xgboost' for the core part. to have ar, you might need to install ar for mac" , I don't understand what "do 'make xgboost' for the core part" means. hope for your advices

ghost on 7 Feb 2017

I have installed the package on my MAC (macOS 10.12.3) successfully by:

brew install --with-clang llvm
brew install cmake
brew install gcc --without-multilib

git clone --recursive https://github.com/dmlc/xgboost
cd xgboost; cp make/config.mk ./config.mk; make -j4

cd python-package/ ; python setup.py install

GnoDGnaF on 11 Mar 2017

@hnykda Many times, the working solution is the seemingly simple one. Thanks, man.

Xu-Tian on 16 Apr 2017

The solution that I found worked for me was navigating to
/home/username/anaconda3/lib/python3.5/site-packages/xgboost
and running ./build-python.sh

WillKoehrsen on 23 Jun 2017

👍2

It seems there are a lot of issues when installing xgboost. i tried to install onto Mac, and failed too. Could osmeone tell me what is the final solution ?

tcglarry on 10 Jul 2017

what worked for me was adding the following on linux ... credit (everyone above)

git clone --recursive https://github.com/dmlc/xgboost.git  
cd xgboost  
./build.sh
pip install -e python-package 
cd python-package
python setup.py install