Lightgbm: Install R package error ( Windows10 )

[teru111]

Environment info

Windows 10 x64
R version 3.5.1(Path)
Rtools34 (Path)
VS 2017
Cmake(Path)

Running following command in RStudio

devtools::install_github("Laurae2/lgbdl", force = TRUE)
library(lgbdl)
lgb.dl(commit = "master",
compiler = "vs",
repo = "https://github.com/Microsoft/LightGBM",
cores =6,
r35 = true)

Error message

* installing *source* package 'lightgbm' ...
** libs
c:/Rtools/mingw_64/bin/gcc  -I"C:/R/R-35~1.1/include" -DNDEBUG          -O2 -Wall  -std=gnu99 -mtune=generic -c R_init.c -o R_init.o
c:/Rtools/mingw_64/bin/gcc -shared -s -static-libgcc -o lightgbm.dll tmp.def R_init.o -LC:/R/R-35~1.1/bin/x64 -lR
installing via 'install.libs.R' to C:/R/R-3.5.1/library/lightgbm
 eval(ei, envir) error : Copying CMakeLists failed
* removing 'C:/R/R-3.5.1/library/lightgbm'
In R CMD INSTALL
[1] FALSE
Warning message:
In install.packages(file.path(lgb_git_dir, "LightGBM", "R-package",  :
  installation of package ‘C:/Users/Kline/AppData/Local/Temp/Rtmpwb0XlY/LightGBM/R-package’ had non-zero exit status

I found the error message "Copying CMakeLists failed" in the following URL (line 20),
but I don't know how to fix it. Please help me.
https://github.com/Microsoft/LightGBM/blob/master/R-package/src/install.libs.R

# Move in CMakeLists.txt
if (!file.copy("../inst/bin/CMakeLists.txt", "CMakeLists.txt", overwrite = TRUE)){
  stop("Copying CMakeLists failed")
}

[BruceZhaoR]

@teru111
pleage see : https://github.com/Laurae2/lgbdl/issues/4

you'd better use Rtools35, if you use gcc compiler

create a temp.bat with the following codes if you use vs compiler :

C:
cd C:\
git clone --recursive https://github.com/Microsoft/LightGBM
cd LightGBM
git checkout master
mkdir build && cd build
cmake -DCMAKE_GENERATOR_PLATFORM=x64 -DUSE_R35=1  ..
cmake --build . --target _lightgbm  --config Release

run the temp.bat

gcc compiler :

c:
cd C:\
git clone --recursive https://github.com/Microsoft/LightGBM
cd LightGBM
git checkout master
mkdir build && cd build
cmake -G "MinGW Makefiles" -DUSE_R35=1  ..
cmake -G "MinGW Makefiles" -DUSE_R35=1  ..
mingw32-make.exe -j1

1. open the C:\LightGBM\R-package\src\install.libs.R
2. set use_precompile <- TRUE
3. modify if (!file.copy("../../CMakeLists.txt", "../CMakeLists.txt", overwrite = TRUE)){ stop("Copying CMakeLists failed") }
4. Rscript> install.packages(file.path("C:", "LightGBM", "R-package", fsep = "\\"), repos = NULL, type = "source")
5. done!

the gcc compiler can be reproducible and tested in my Windows computer

[teru111]

I finally successed to install lightgbm package !
Your guidance is perfect. (I used vs compiler. )
Thank you so much !!

[falconm7]

I was facing problem to install R package error from last two days. I have searched a lot, even I took help from Windows 10 support number but nothing works. This is the really nice solution. It helps me a lot.

[guolinke]

@Laurae2 @jameslamb
Did we change the build in the windows ?

[BruceZhaoR]

@guolinke changed a little.
lgbdl works fine now when your Windows tempdir is in C:\ disk.

[Kallikrates]

I've tried to install this package every suggeted way and still no joy. I keep getting a CMake Error. I've set my PATH variable to the following. Any ideas?

C:\Program Files\R\R-3.5.1\bin\x64; C:\Rtools\mingw_64\bin; C:\Program Files\Git\bin; C:\Program Files\CMake\bin

Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\WINDOWS\system32>cd C:\

C:>git clone --recursive https://github.com/Microsoft/LightGBM
Cloning into 'LightGBM'...
remote: Enumerating objects: 85, done.
remote: Counting objects: 100% (85/85), done.
remote: Compressing objects: 100% (76/76), done.
remote: Total 11652 (delta 33), reused 17 (delta 8), pack-reused 11567
Receiving objects: 100% (11652/11652), 8.62 MiB | 764.00 KiB/s, done.
Resolving deltas: 100% (8206/8206), done.
Submodule 'include/boost/compute' (https://github.com/boostorg/compute) register
ed for path 'compute'
Cloning into 'C:/LightGBM/compute'...
remote: Enumerating objects: 21464, done.
remote: Total 21464 (delta 0), reused 0 (delta 0), pack-reused 21464
Receiving objects: 100% (21464/21464), 8.45 MiB | 617.00 KiB/s, done.
Resolving deltas: 100% (17406/17406), done.
Submodule path 'compute': checked out '509ebe4a9282eec8a92c65ce3bbc1925f1fdbe07'

C:>cd LightGBM

C:\LightGBM>git checkout master
Already on 'master'
Your branch is up to date with 'origin/master'.

C:\LightGBM>mkdir build && cd build

C:\LightGBM\build>cmake -G "MinGW Makefiles" -DUSE_R35=1 ..
CMake Error: CMake was unable to find a build program corresponding to "MinGW Ma
kefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a differe
nt build tool.
CMake Error: CMake was unable to find a build program corresponding to "MinGW Ma
kefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a differe
nt build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

C:\LightGBM\build>

[BruceZhaoR]

@Kallikrates
make sure you have mingw32-make.exe in your PATH.

$ mingw32-make.exe --version
GNU Make 4.1
Built for x86_64-w64-mingw32
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

1. add C:\Rtools\bin; to your system PATH, close terminal and reopen try again;
2. reinstall CMake to C:\ProgramFiles\CMake (without space in the path) and add it to the PATH variable, close and reopen , try again;

[Kallikrates]

Tried the above but still getting :"'cmake' is not recognized as an internal or external command,
operable program or batch file." when running: cmake -G "MinGW Makefiles" -DUSE_R35=1 ..

My PATH string is as follows with no spaces.
C:\PROGRA~1\R\R-3.5.1\bin\x64; C:\PROGRA~1\Git\bin; C:\PROGRA~1\CMake\bin; C:\Rtools\mingw_64\bin

mingw32-make.exe is sitting in C:\Rtools\mingw_64\bin.
BTW. I also tried reinstalling CMake to C:\ProgramFiles\CMake (without space in the path) and adding it to the PATH variable. Same result.

[BruceZhaoR]

'cmake' is not recognized as an internal or external command

make sure you can use cmake in cmd

C:\Users\Bruce>cmake --version
cmake version 3.10.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).

C:\Users\Bruce>where cmake
D:\ProgramFiles\CMake\bin\cmake.exe

C:\Users\Bruce>make --version
GNU Make 4.2.1
Built for i686-pc-msys
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

C:\Users\Bruce>where make
C:\Rtools\bin\make.exe

C:\Users\Bruce>R --version
R version 3.5.1 (2018-07-02) -- "Feather Spray"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
http://www.gnu.org/licenses/.

C:\Users\Bruce>gcc --version
gcc (x86_64-posix-seh, Built by MinGW-W64 project) 4.9.3
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

C:\Users\Bruce>g++ --version
g++ (x86_64-posix-seh, Built by MinGW-W64 project) 4.9.3
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

C:\Users\Bruce\Documents\Github\LightGBM\build>cmake -G "MinGW Makefiles" -DUSE_R35=ON ..
CMake Error at D:/ProgramFiles/CMake/share/cmake-3.10/Modules/CMakeMinGWFindMake.cmake:12 (message):
  sh.exe was found in your PATH, here:

  C:/Rtools/bin/sh.exe

  For MinGW make to work correctly sh.exe must NOT be in your path.

  Run cmake from a shell that does not have sh.exe in your PATH.

  If you want to use a UNIX shell, then use MSYS Makefiles.

Call Stack (most recent call first):
  CMakeLists.txt:7 (PROJECT)


CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

C:\Users\Bruce\Documents\Github\LightGBM\build>cmake -G "MinGW Makefiles" -DUSE_R35=ON ..
-- The C compiler identification is GNU 4.9.3
-- The CXX compiler identification is GNU 4.9.3
-- Check for working C compiler: C:/Rtools/mingw_64/bin/gcc.exe
-- Check for working C compiler: C:/Rtools/mingw_64/bin/gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Rtools/mingw_64/bin/g++.exe
-- Check for working CXX compiler: C:/Rtools/mingw_64/bin/g++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -fopenmp (found version "4.0")
-- Found OpenMP_CXX: -fopenmp (found version "4.0")
-- Found OpenMP: TRUE (found version "4.0")
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/Bruce/Documents/Github/LightGBM/build

@Kallikrates make sure all these can be accessed in you cmd. Maybe you need to restart your computer.

[Kallikrates]

Thanks but still no good ever after restart. Here's the transcript. Am I missing something?

C:>cmake --version
cmake version 3.12.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).

C:>where cmake
C:\ProgramFiles\CMake\bin\cmake.exe

C:>cd Rtools\bin

C:\Rtools\bin>where make
C:\Rtools\binmake.exe

C:\Rtools\bin>make --version
GNU Make 4.2.1
Built for i686-pc-msys
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

C:\Rtools\bin>cd C:\Program Files\R\R-3.5.1\bin

C:\Program Files\R\R-3.5.1\bin>R --version
R version 3.5.1 (2018-07-02) -- "Feather Spray"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
http://www.gnu.org/licenses/.

C:\Program Files\R\R-3.5.1\bin>cd C:\Program Files (x86)\Dev-Cpp\MinGW64\bin

C:\Program Files (x86)\Dev-Cpp\MinGW64\bin>gcc --version
gcc (tdm64-1) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

C:\Program Files (x86)\Dev-Cpp\MinGW64\bin>g++ --version
g++ (tdm64-1) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

C:\Program Files (x86)\Dev-Cpp\MinGW64\bin>cd C:\LightGBM\build

C:\LightGBM\build>

C:\LightGBM\build>cmake -G "MinGW Makefiles" -DUSE_R35=ON ..
CMake Error: CMake was unable to find a build program corresponding to "MinGW Ma
kefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a differe
nt build tool.
CMake Error: CMake was unable to find a build program corresponding to "MinGW Ma
kefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a differe
nt build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

PATH Variables:

C:\PROGRA~1\R\R-3.5.1\bin\x64; C:\PROGRA~1\Git\bin; C:\ProgramFiles\Cmake\bin; C:\Rtools\bin; C:\PROGRA~2\Dev-Cpp\MinGW64\bin; C:\Rtools\mingw_64\bin

[Laurae2]

Remove the other MinGW sitting on your PATH which is before the last one.

[jameslamb]

@Laurae2 need my help on this? I have limited usefulness in the Windows-specific stuff (probably need to get better at it)

also do we need to reopen this issue?

[Laurae2]

@jameslamb The user has 2 MinGW installed at the same time and they conflict each other as cmake can't know what to take. The tdm MinGW must be removed to let R's MinGW to be used.

[Kallikrates]

Left my path as:

C:\PROGRA~1\R\R-3.5.1\bin\x64; C:\PROGRA~1\Git\bin; C:\ProgramFiles\Cmake\bin; C:\Rtools\bin; C:\Rtools\mingw_64\bin

Ran: C:\LightGBM\build>cmake -G "MinGW Makefiles" -DUSE_R35=ON ..

Still getting an error:

CMake Error: CMake was unable to find a build program corresponding to "MinGW Ma
kefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a differe
nt build tool.
CMake Error: CMake was unable to find a build program corresponding to "MinGW Ma
kefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a differe
nt build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

[mustaphaweather]

hey guys i just installed lightgbm package these are the steps to reproduce :

1 - git clone --recursive https://github.com/Microsoft/LightGBM
2 - cd LightGBM
3 - open R-package/src/install.lib and change use_precompile from FALSE to TRUE
4 - go to root directory of lightgbm and copy cmakelists and paste it to R-package/inst/bin
5 - cd lightGBM
6 - Rscript build_r.R
7 - cd r_lightgbm
8 - R CMD INSTALL --build . --no-multiarch

[spedygiorgio]

It is a while I did not reinstall ligthgbm and I came again in trouble. I followed the procedure outlined by @mustaphaweather .
I got the following:

`[1] "C:/LightGBM/lightgbm_2.2.3.tar.gz"
In R CMD INSTALL

• installing to library 'D:/RPackages'
• installing source package 'lightgbm' ...
  ** libs
  c:/Rtools/mingw_64/bin/gcc -I"C:/PROGRA~1/R/R-35~1.1/include" -DNDEBUG -O2 -Wall -std=gnu99 -mtune=generic -c R_init.c -o R_init.o
  c:/Rtools/mingw_64/bin/gcc -shared -s -static-libgcc -o lightgbm.dll tmp.def R_init.o -LC:/PROGRA~1/R/R-35~1.1/bin/x64 -lR
  installing via 'install.libs.R' to D:/RPackages/lightgbm
  Error in eval(ei, envir) : Cannot find lib_lightgbm.dll
• removing 'D:/RPackages/lightgbm'
  PS C:\LightGBM> Rscript build_r.R
  v checking for file 'C:\LightGBM\lightgbm_r/DESCRIPTION' (370ms)
• preparing 'lightgbm': (1.5s)
  v checking DESCRIPTION meta-information ...
• cleaning src
• checking for LF line-endings in source and make files and shell scripts (357ms)
• checking for empty or unneeded directories (740ms)
• looking to see if a 'data/datalist' file should be added
• building 'lightgbm_2.2.3.tar.gz'

[1] "C:/LightGBM/lightgbm_2.2.3.tar.gz"
In R CMD INSTALL

• installing to library 'D:/RPackages'
• installing source package 'lightgbm' ...
  ** libs
  c:/Rtools/mingw_64/bin/gcc -I"C:/PROGRA~1/R/R-35~1.1/include" -DNDEBUG -O2 -Wall -std=gnu99 -mtune=generic -c R_init.c -o R_init.o
  c:/Rtools/mingw_64/bin/gcc -shared -s -static-libgcc -o lightgbm.dll tmp.def R_init.o -LC:/PROGRA~1/R/R-35~1.1/bin/x64 -lR
  installing via 'install.libs.R' to D:/RPackages/lightgbm
  Error in eval(ei, envir) : Cannot find lib_lightgbm.dll
• removing 'D:/RPackages/lightgbm'`

My two questions are:

1. @Laurae2, is it possible to amend the documentation to point out more strainghtforwardly how to manually compile R and Python package using precompiled dlls in windows?
2. Where have I to put the precompired dll that I find in the Release folder?

Thanks in advance

[Laurae2]

@spedygiorgio Once you compiled the dll/lib, put it at the root of LightGBM folder.

Then, change the following line to TRUE: https://github.com/Microsoft/LightGBM/blob/5457ef6b76fa9d80bb9a72d0e72080b255d7e8d0/R-package/src/install.libs.R#L2

And you should be able to compile LightGBM for R.

For Python, ping @StrikerRUS

[StrikerRUS]

@spedygiorgio

1. We already have instructions about how to use precompiled DLL in Python-package installation. It's in the last paragraph of Install from GitHub section: https://github.com/Microsoft/LightGBM/tree/master/python-package#install-from-github . I can cite it here:

   If you get any errors during installation or due to any other reasons, you may want to build dynamic library from sources by any method you prefer (see Installation Guide) and then just run python setup.py install --precompile.

2.

No any additional file manipulations are needed. Just compile the library and run setup with special flag. We use a separate Python module to search for library file in many places: https://github.com/Microsoft/LightGBM/blob/master/python-package/lightgbm/libpath.py

[robwec]

When I attempt to install the R package on Windows with the precompiled dll, even after following the install instructions and copying the .dll, I get

Error in eval(ei, envir) : Cannot find lib_lightgbm.dll

This is unusual. Followed all the install instructions and the instructions in this comment, and even tried putting the .dll all over the place. But it still couldn't be found.

It is the same as spedygiorgio's issue.