Lightgbm: run python setup.py, says:Exception: Please install CMake and all required dependencies first The full version of error log was saved into /Users/hong/LightGBM_compilation.log

[jiangniao26]

My computer is Mac. I followed the Installation Guide step by step:

brew install cmake
brew install gcc
git clone --recursive https://github.com/Microsoft/LightGBM ; cd LightGBM
export CXX=g++-7 CC=gcc-7
mkdir build ; cd build
cmake ..
make -j4

pip install setuptools wheel numpy scipy scikit-learn -U

cd LightGBM/python-package
export CXX=g++-7 CC=gcc-7 # for macOS users only
python setup.py install

But it shows that:

Error Message:

running install
creating compile
creating compile/include
creating compile/include/LightGBM
copying ../include/LightGBM/application.h -> ./compile/include/LightGBM
copying ../include/LightGBM/bin.h -> ./compile/include/LightGBM
copying ../include/LightGBM/boosting.h -> ./compile/include/LightGBM
copying ../include/LightGBM/c_api.h -> ./compile/include/LightGBM
copying ../include/LightGBM/config.h -> ./compile/include/LightGBM
copying ../include/LightGBM/dataset.h -> ./compile/include/LightGBM
copying ../include/LightGBM/dataset_loader.h -> ./compile/include/LightGBM
copying ../include/LightGBM/export.h -> ./compile/include/LightGBM
copying ../include/LightGBM/feature_group.h -> ./compile/include/LightGBM
copying ../include/LightGBM/lightgbm_R.h -> ./compile/include/LightGBM
copying ../include/LightGBM/meta.h -> ./compile/include/LightGBM
copying ../include/LightGBM/metric.h -> ./compile/include/LightGBM
copying ../include/LightGBM/network.h -> ./compile/include/LightGBM
copying ../include/LightGBM/objective_function.h -> ./compile/include/LightGBM
copying ../include/LightGBM/prediction_early_stop.h -> ./compile/include/LightGBM
copying ../include/LightGBM/R_object_helper.h -> ./compile/include/LightGBM
copying ../include/LightGBM/tree.h -> ./compile/include/LightGBM
copying ../include/LightGBM/tree_learner.h -> ./compile/include/LightGBM
creating compile/include/LightGBM/utils
copying ../include/LightGBM/utils/array_args.h -> ./compile/include/LightGBM/utils
copying ../include/LightGBM/utils/common.h -> ./compile/include/LightGBM/utils
copying ../include/LightGBM/utils/file_io.h -> ./compile/include/LightGBM/utils
copying ../include/LightGBM/utils/log.h -> ./compile/include/LightGBM/utils
copying ../include/LightGBM/utils/openmp_wrapper.h -> ./compile/include/LightGBM/utils
copying ../include/LightGBM/utils/pipeline_reader.h -> ./compile/include/LightGBM/utils
copying ../include/LightGBM/utils/random.h -> ./compile/include/LightGBM/utils
copying ../include/LightGBM/utils/text_reader.h -> ./compile/include/LightGBM/utils
copying ../include/LightGBM/utils/threading.h -> ./compile/include/LightGBM/utils
creating compile/src
creating compile/src/application
copying ../src/application/application.cpp -> ./compile/src/application
copying ../src/application/predictor.hpp -> ./compile/src/application
creating compile/src/boosting
copying ../src/boosting/boosting.cpp -> ./compile/src/boosting
copying ../src/boosting/dart.hpp -> ./compile/src/boosting
copying ../src/boosting/gbdt.cpp -> ./compile/src/boosting
copying ../src/boosting/gbdt.h -> ./compile/src/boosting
copying ../src/boosting/gbdt_model_text.cpp -> ./compile/src/boosting
copying ../src/boosting/gbdt_prediction.cpp -> ./compile/src/boosting
copying ../src/boosting/goss.hpp -> ./compile/src/boosting
copying ../src/boosting/prediction_early_stop.cpp -> ./compile/src/boosting
copying ../src/boosting/rf.hpp -> ./compile/src/boosting
copying ../src/boosting/score_updater.hpp -> ./compile/src/boosting
copying ../src/c_api.cpp -> ./compile/src
creating compile/src/io
copying ../src/io/bin.cpp -> ./compile/src/io
copying ../src/io/config.cpp -> ./compile/src/io
copying ../src/io/dataset.cpp -> ./compile/src/io
copying ../src/io/dataset_loader.cpp -> ./compile/src/io
copying ../src/io/dense_bin.hpp -> ./compile/src/io
copying ../src/io/dense_nbits_bin.hpp -> ./compile/src/io
copying ../src/io/file_io.cpp -> ./compile/src/io
copying ../src/io/metadata.cpp -> ./compile/src/io
copying ../src/io/ordered_sparse_bin.hpp -> ./compile/src/io
copying ../src/io/parser.cpp -> ./compile/src/io
copying ../src/io/parser.hpp -> ./compile/src/io
copying ../src/io/sparse_bin.hpp -> ./compile/src/io
copying ../src/io/tree.cpp -> ./compile/src/io
copying ../src/lightgbm_R.cpp -> ./compile/src
copying ../src/main.cpp -> ./compile/src
creating compile/src/metric
copying ../src/metric/binary_metric.hpp -> ./compile/src/metric
copying ../src/metric/dcg_calculator.cpp -> ./compile/src/metric
copying ../src/metric/map_metric.hpp -> ./compile/src/metric
copying ../src/metric/metric.cpp -> ./compile/src/metric
copying ../src/metric/multiclass_metric.hpp -> ./compile/src/metric
copying ../src/metric/rank_metric.hpp -> ./compile/src/metric
copying ../src/metric/regression_metric.hpp -> ./compile/src/metric
copying ../src/metric/xentropy_metric.hpp -> ./compile/src/metric
creating compile/src/network
copying ../src/network/linker_topo.cpp -> ./compile/src/network
copying ../src/network/linkers.h -> ./compile/src/network
copying ../src/network/linkers_mpi.cpp -> ./compile/src/network
copying ../src/network/linkers_socket.cpp -> ./compile/src/network
copying ../src/network/network.cpp -> ./compile/src/network
copying ../src/network/socket_wrapper.hpp -> ./compile/src/network
creating compile/src/objective
copying ../src/objective/binary_objective.hpp -> ./compile/src/objective
copying ../src/objective/multiclass_objective.hpp -> ./compile/src/objective
copying ../src/objective/objective_function.cpp -> ./compile/src/objective
copying ../src/objective/rank_objective.hpp -> ./compile/src/objective
copying ../src/objective/regression_objective.hpp -> ./compile/src/objective
copying ../src/objective/xentropy_objective.hpp -> ./compile/src/objective
creating compile/src/treelearner
copying ../src/treelearner/data_parallel_tree_learner.cpp -> ./compile/src/treelearner
copying ../src/treelearner/data_partition.hpp -> ./compile/src/treelearner
copying ../src/treelearner/feature_histogram.hpp -> ./compile/src/treelearner
copying ../src/treelearner/feature_parallel_tree_learner.cpp -> ./compile/src/treelearner
copying ../src/treelearner/gpu_tree_learner.cpp -> ./compile/src/treelearner
copying ../src/treelearner/gpu_tree_learner.h -> ./compile/src/treelearner
copying ../src/treelearner/leaf_splits.hpp -> ./compile/src/treelearner
creating compile/src/treelearner/ocl
copying ../src/treelearner/ocl/histogram16.cl -> ./compile/src/treelearner/ocl
copying ../src/treelearner/ocl/histogram256.cl -> ./compile/src/treelearner/ocl
copying ../src/treelearner/ocl/histogram64.cl -> ./compile/src/treelearner/ocl
copying ../src/treelearner/parallel_tree_learner.h -> ./compile/src/treelearner
copying ../src/treelearner/serial_tree_learner.cpp -> ./compile/src/treelearner
copying ../src/treelearner/serial_tree_learner.h -> ./compile/src/treelearner
copying ../src/treelearner/split_info.hpp -> ./compile/src/treelearner
copying ../src/treelearner/tree_learner.cpp -> ./compile/src/treelearner
copying ../src/treelearner/voting_parallel_tree_learner.cpp -> ./compile/src/treelearner
copying ../windows/LightGBM.sln -> ./compile/windows
copying ../windows/LightGBM.vcxproj -> ./compile/windows
copying ../CMakeLists.txt -> ./compile/
copying ../LICENSE -> ./
INFO:LightGBM:Starting to compile the library.
INFO:LightGBM:Starting to compile with CMake.
Traceback (most recent call last):
File "setup.py", line 68, in silent_call
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
File "/Users/hong/anaconda/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/hong/anaconda/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['cmake', '../compile/']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "setup.py", line 282, in
'Topic :: Scientific/Engineering :: Artificial Intelligence'])
File "/Users/hong/anaconda/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
return distutils.core.setup(**attrs)
File "/Users/hong/anaconda/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/Users/hong/anaconda/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/Users/hong/anaconda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "setup.py", line 208, in run
opencl_include_dir=self.opencl_include_dir, opencl_library=self.opencl_library)
File "setup.py", line 154, in compile_cpp
silent_call(cmake_cmd, raise_error=True, error_msg='Please install CMake and all required dependencies first')
File "setup.py", line 77, in silent_call
raise Exception("\n".join((error_msg, log_notice)))
Exception: Please install CMake and all required dependencies first
The full version of error log was saved into /Users/hong/LightGBM_compilation.log

And LightGBM_compilation.log shows that:
-- The C compiler identification is AppleClang 9.0.0.9000039
-- The CXX compiler identification is AppleClang 9.0.0.9000039
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:26 (message):
AppleClang wasn't supported. Please see
https://github.com/Microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#macos

-- Configuring incomplete, errors occurred!
See also "/Users/hong/LightGBM/python-package/build_cpp/CMakeFiles/CMakeOutput.log".

I'm sure I have installed cmake by: brew install cmake
how can I deal with this problem?

[guolinke]

Did brew install gcc install gcc 7 ?

[jiangniao26]

@guolinke Yes, after I reinstall by brew install gcc, it shows that,
==> Reinstalling gcc
==> Downloading https://homebrew.bintray.com/bottles/gcc-7.3.0_1.sierra.bottle.t
Already downloaded: /Users/hong/Library/Caches/Homebrew/gcc-7.3.0_1.sierra.bottle.tar.gz
==> Pouring gcc-7.3.0_1.sierra.bottle.tar.gz
🍺 /usr/local/Cellar/gcc/7.3.0_1: 1,486 files, 284MB
But it still doesn't work

[guolinke]

For the log, the error is

CMake Error at CMakeLists.txt:26 (message):
AppleClang wasn't supported. Please see
https://github.com/Microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#macos

So it seems the gcc isn't used by cmake.

Did it throw the error If you run following cmd only ?

brew install cmake
brew install gcc
git clone --recursive https://github.com/Microsoft/LightGBM ; cd LightGBM
export CXX=g++-7 CC=gcc-7
mkdir build ; cd build
cmake ..
make -j4

[jiangniao26]

MacBook-Pro:~ hong$ cd LightGBM
MacBook-Pro:LightGBM hong$ export CXX=g++-7 CC=gcc-7
MacBook-Pro:LightGBM hong$ mkdir build
mkdir: build: File exists
MacBook-Pro:LightGBM hong$ cd build
MacBook-Pro:build hong$ cmake ..
-- Found OpenMP_C: -fopenmp  
-- Found OpenMP_CXX: -fopenmp  
-- Found OpenMP: TRUE   
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/hong/LightGBM/build
MacBook-Pro:build hong$ make -j4
[ 50%] Built target _lightgbm
[100%] Built target lightgbm

No, it didn't throw any error.

[guolinke]

okay, so you can install python package via -p.

brew install cmake
brew install gcc
git clone --recursive https://github.com/Microsoft/LightGBM ; cd LightGBM
export CXX=g++-7 CC=gcc-7
mkdir build ; cd build
cmake ..
make -j4
cd ../python-package
python setup.py install -p

[jiangniao26]

It still doesn't work,

MacBook-Pro:python-package hong$ python setup.py install -p
running install
error: [Errno 13] Permission denied: '/Users/hong/LightGBM_compilation.log'

'''LightGBM_compilation.log''' is the same as before

[guolinke]

error: [Errno 13] Permission denied: '/Users/hong/LightGBM_compilation.log'
you may need to run by root

[jiangniao26]

@guolinke It works, thanks! 真是太谢谢了，请原谅我这个小白。

[guolinke]

you are welcome