Xgboost: Error installing xgboost

[legohyl]

Hey, I'm having an error when trying to use the "make" cmd. I've tried both mingw-64 as well as the TDM-GCC compilers but no luck. Here's the error log:

$ cp make/mingw64.mk config.mk; make -j4
g++ -m64 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -DDMLC_ENABLE_STD_THREAD=0 -Idmlc-core/include -Irabit/include -fopenmp -MM -MT build/c_api/c_api.o src/c_api/c_api.cc >build/c_api/c_api.d
g++ -m64 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -DDMLC_ENABLE_STD_THREAD=0 -Idmlc-core/include -Irabit/include -fopenmp -MM -MT build/c_api/c_api_error.o src/c_api/c_api_error.cc >build/c_api/c_api_error.d
g++ -m64 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -DDMLC_ENABLE_STD_THREAD=0 -Idmlc-core/include -Irabit/include -fopenmp -MM -MT build/data/sparse_page_raw_format.o src/data/sparse_page_raw_format.cc >build/data/sparse_page_raw_format.d
g++ -m64 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -DDMLC_ENABLE_STD_THREAD=0 -Idmlc-core/include -Irabit/include -fopenmp -MM -MT build/data/simple_dmatrix.o src/data/simple_dmatrix.cc >build/data/simple_dmatrix.d
g++ -m64 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -DDMLC_ENABLE_STD_THREAD=0 -Idmlc-core/include -Irabit/include -fopenmp -c src/c_api/c_api.cc -o build/c_api/c_api.o
g++ -m64 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -DDMLC_ENABLE_STD_THREAD=0 -Idmlc-core/include -Irabit/include -fopenmp -c src/data/sparse_page_raw_format.cc -o build/data/sparse_page_raw_format.o
g++ -m64 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -DDMLC_ENABLE_STD_THREAD=0 -Idmlc-core/include -Irabit/include -fopenmp -c src/c_api/c_api_error.cc -o build/c_api/c_api_error.o
g++ -m64 -c -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -DDMLC_ENABLE_STD_THREAD=0 -Idmlc-core/include -Irabit/include -fopenmp -c src/data/simple_dmatrix.cc -o build/data/simple_dmatrix.o
In file included from C:/TDM-GCC-64/x86_64-w64-mingw32/include/locale.h:12:0,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/clocale:42,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/x86_64-w64-mingw32/bits/c++locale.h:41,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/bits/localefwd.h:40,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/string:43,
from dmlc-core/include/dmlc/base.h:154,
from include/xgboost/data.h:10,
from src/data/simple_dmatrix.cc:7:
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:450:83: error: 'FILE* std::fopen(const char_, const char_)' should have been declared inside 'std'
FILE _fopen64(const char * restrict filename,const char * restrict mode);
^
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:450:83: error: 'FILE_ std::fopen(const char_, const char_)' conflicts with a previous declaration
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:449:17: note: previous declaration 'FILE* fopen(const char_, const char_)'
FILE _cdecl fopen(const char * __restrict _Filename,const char * __restrict__ _Mode) __MINGW_ATTRIB_DEPRECATED_SEC_WARN;
^
In file included from C:/TDM-GCC-64/x86_64-w64-mingw32/include/locale.h:12:0,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/clocale:42,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/x86_64-w64-mingw32/bits/c++locale.h:41,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/bits/localefwd.h:40,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/string:43,
from dmlc-core/include/dmlc/base.h:154,
from include/xgboost/data.h:10,
from src/data/sparse_page_raw_format.cc:6:
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:450:83: error: 'FILE_ std::fopen(const char_, const char_)' should have been declared inside 'std'
FILE _fopen64(const char * restrict filename,const char * restrict mode);
^
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:450:83: error: 'FILE_ std::fopen(const char_, const char_)' conflicts with a previous declaration
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:449:17: note: previous declaration 'FILE* fopen(const char_, const char_)'
FILE _cdecl fopen(const char * __restrict _Filename,const char * __restrict__ _Mode) __MINGW_ATTRIB_DEPRECATED_SEC_WARN;
^
In file included from C:/TDM-GCC-64/x86_64-w64-mingw32/include/locale.h:12:0,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/clocale:42,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/x86_64-w64-mingw32/bits/c++locale.h:41,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/bits/localefwd.h:40,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/string:43,
from dmlc-core/include/dmlc/base.h:154,
from src/c_api/./c_api_error.h:9,
from src/c_api/c_api_error.cc:6:
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:450:83: error: 'FILE_ std::fopen(const char_, const char_)' should have been declared inside 'std'
FILE _fopen64(const char * restrict filename,const char * restrict mode);
^
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:450:83: error: 'FILE_ std::fopen(const char_, const char_)' conflicts with a previous declaration
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:449:17: note: previous declaration 'FILE* fopen(const char_, const char_)'
FILE _cdecl fopen(const char * __restrict _Filename,const char * __restrict__ _Mode) __MINGW_ATTRIB_DEPRECATED_SEC_WARN;
^
In file included from C:/TDM-GCC-64/x86_64-w64-mingw32/include/locale.h:12:0,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/clocale:42,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/x86_64-w64-mingw32/bits/c++locale.h:41,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/bits/localefwd.h:40,
from C:/TDM-GCC-64/lib/gcc/x86_64-w64-mingw32/5.1.0/include/c++/string:43,
from dmlc-core/include/dmlc/base.h:154,
from include/xgboost/data.h:10,
from src/c_api/c_api.cc:3:
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:450:83: error: 'FILE_ std::fopen(const char_, const char_)' should have been declared inside 'std'
FILE _fopen64(const char * restrict filename,const char * restrict mode);
^
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:450:83: error: 'FILE_ std::fopen(const char_, const char_)' conflicts with a previous declaration
C:/TDM-GCC-64/x86_64-w64-mingw32/include/stdio.h:449:17: note: previous declaration 'FILE* fopen(const char_, const char_)'
FILE _cdecl fopen(const char * __restrict _Filename,const char * __restrict__ _Mode) __MINGW_ATTRIB_DEPRECATED_SEC_WARN;
^
Makefile:97: recipe for target 'build/c_api/c_api_error.o' failed
mingw32-make: _* [build/c_api/c_api_error.o] Error 1
mingw32-make: * Waiting for unfinished jobs....
Makefile:97: recipe for target 'build/data/simple_dmatrix.o' failed
mingw32-make: ** [build/data/simple_dmatrix.o] Error 1
In file included from dmlc-core/include/dmlc/data.h:16:0,
from include/xgboost/data.h:11,
from src/data/sparse_page_raw_format.cc:6:
dmlc-core/include/dmlc/./registry.h:218:22: warning: 'xgboost::data::make_SparsePageFormat_raw*' defined but not used [-Wunused-variable]
static EntryType & __make_ ## EntryTypeName ## _ ## Name ## __ =
^
src/data/./sparse_batch_page.h:250:3: note: in expansion of macro 'DMLC_REGISTRY_REGISTER'
DMLC_REGISTRY_REGISTER(::xgboost::data::SparsePageFormatReg, SparsePageFormat, Name)
^
src/data/sparse_page_raw_format.cc:93:1: note: in expansion of macro 'XGBOOST_REGISTER_SPARSE_PAGE_FORMAT'
XGBOOST_REGISTER_SPARSE_PAGE_FORMAT(raw)
^
Makefile:97: recipe for target 'build/data/sparse_page_raw_format.o' failed
mingw32-make: ** [build/data/sparse_page_raw_format.o] Error 1
src/c_api/c_api.cc:739:12: warning: 'XGBOOST_LINK_RABIT_C_API_' defined but not used [-Wunused-variable]
static int XGBOOST_LINK_RABIT_C_API_ = RabitLinkTag();
^
Makefile:97: recipe for target 'build/c_api/c_api.o' failed
mingw32-make: *** [build/c_api/c_api.o] Error 1

[fridayily]

I have the same problem!!

[JasonTianCHD]

I have the same problem.

[Cortajarena]

same here!

[Qingmu]

same.......

Windows is really not a good choice to do ml.

[Cortajarena]

@ziqilau @tqchen @JasonTianCHD @yanqingmen
Fixed, just clone repo and then checkout to 9a48a40 until they fix it, something in the last commits must have screwed up the compilation:

git clone --recursive https://github.com/dmlc/xgboost
cd xgboost
git checkout 9a48a40
git submodule init
git submodule update

Cheers!

[legohyl]

I can confirm that this works for the time being. Thanks a lot!

[ezchx]

Yes, after 2 days of messing around, the fix from Cortajarena worked! Thank you so much!!

[Qingmu]

Thanks you!! @Cortajarena

[pe-perry]

Thanks @Cortajarena !

[sandro-felicioni]

Thanks @Cortajarena - you saved my day! :-)

[llmercury]

Thanks @Cortajarena!

[llmercury]

_However, when I imported xgboost in jupyter nootbook,_

import os
mingw_path = "C:TDM-GCC-64bin"
os.environ['PATH'] = mingw_path + ';' + os.environ['PATH']
import xgboost as xgb

_I got error message:_

OSError Traceback (most recent call last)
in ()
----> 1 import xgboost as xgb

C:UsersLAnaconda3libsite-packagesxgboost-0.4-py3.5.eggxgboost__init__.py in ()
9 import os
10
---> 11 from .core import DMatrix, Booster
12 from .training import train, cv
13 from . import rabit # noqa

C:UsersLAnaconda3libsite-packagesxgboost-0.4-py3.5.eggxgboostcore.py in ()
110
111 # load the XGBoost library globally
--> 112 _LIB = _load_lib()
113
114

C:UsersLAnaconda3libsite-packagesxgboost-0.4-py3.5.eggxgboostcore.py in _load_lib()
104 if len(lib_path) == 0:
105 return None
--> 106 lib = ctypes.cdll.LoadLibrary(lib_path[0])
107 lib.XGBGetLastError.restype = ctypes.c_char_p
108 return lib

C:UsersLAnaconda3libctypes__init__.py in LoadLibrary(self, name)
423
424 def LoadLibrary(self, name):
--> 425 return self._dlltype(name)
426
427 cdll = LibraryLoader(CDLL)

C:UsersLAnaconda3libctypes__init__.py in init(self, name, mode, handle, use_errno, use_last_error)
345
346 if handle is None:
--> 347 self._handle = _dlopen(self._name, mode)
348 else:
349 self._handle = handle

OSError: [WinError 1114] A dynamic link library (DLL) initialization routine failed.

Anybody had the same problem? How to fix it?

[lrvenable]

Thank you @Cortajarena for the tip!
Is anyone else getting a subsequent error that the "XGBoost Library cannot be found in the candicate path"?
I am using Anaconda 4.0.0 with Python 3.5.1 on Windows7.
The following commands worked fine:

git clone --recursive https://github.com/dmlc/xgboost
cd xgboost
git checkout 9a48a40
git submodule init
git submodule update

When I switched back to Anaconda Prompt, redirected to python-packages folder and tried:

python setup.py install

I received the error that the library could not be found and it provides a list of candidates.
Ideas on how to fix?

[Cortajarena]

@lrvenable

After doing the steps you named, type:

cd python-package

You will then be inside the python package directory (inside the xgboost folder of the repo you have already cloned).
Then type this in the console:

python setup.py install

Be sure to be using the bit bash instead of the windows command prompt!
Cheers!

[lrvenable]

@Cortajarena you are an absolute savior. many, many thanks!

[Far0n]

that fixed the issue for me (Win7 64Bit, mingw-w64):

...dmlc-coreincludedmlcbase.h line 110++ (commit https://github.com/dmlc/xgboost/commit/75d9be55de7e13a0075b4422c37a74f7b39600b1)

if !defined(GNUC)

define fopen64 std::fopen NOTE: remove "std::"

endif

if (defined MINGW32) || (defined MINGW64)

define fopen64 std::fopen NOTE: remove "std::"

endif

=>

if !defined(GNUC)

define fopen64 fopen

endif

if (defined MINGW32) || (defined MINGW64)

define fopen64 fopen

endif

[khotilov]

The upstream issue was fixed and the reference to dmlc-core was updated #1408 . I would suggest that someone should close this issue, so that people don't keep installing the June 5th version of code.

[zhouzhen1]

I was building the R package on Windows 7. Latest xgboost package from CRAN didn't build.

@Far0n 's solution worked for me.

[pithysr]

@Cortajarena @lrvenable

I get

 $ python setup.py install
 Traceback (most recent call last):
   File "setup.py", line 19, in <module>
     LIB_PATH = libpath['find_lib_path']()
   File "xgboost/libpath.py", line 45, in find_lib_path
     'List of candidates:\n' + ('\n'.join(dll_path)))
 XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
 List of candidates:
 ~/xgboost/xgboost/python-package/xgboost/libxgboost.so
 ~/xgboost/xgboost/python-package/xgboost/../../lib/libxgboost.so
 ~/xgboost/xgboost/python-package/xgboost/./lib/libxgboost.so