Lightgbm: R-Package Install

[masaru-tsuruta]

Hi, I have tried to install LightGBM R-package.

I have prepared below software and OS:
Windows10 / R3.6.3 / Rtools3.5 / VS Build Tools 2019 / CMake3.17.1

Additionally, I have set the below directory to PATH.

C:\Rtools\bin
C:\Program Files\CMake\bin
C:\Program Files\R\R-3.6.3\bin\x64
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin
C:\Program Files\Git\cmd

Then, I have tried to run the below commands in CMD using VS tool:
git clone --recursive https://github.com/microsoft/LightGBM
cd LightGBM
Rscript build_r.R

However, the below error has occurred. Could you please advise me?

.....\LightGBM> Rscript build_r.R
* checking for file 'lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_2.3.2.tar.gz'

* installing to library 'C:/Users/masaru/Documents/R/win-library/3.6'
ERROR: failed to lock directory 'C:/Users/masaru/Documents/R/win-library/3.6' for modifying
Try removing 'C:/Users/masaru/Documents/R/win-library/3.6/00LOCK-lightgbm'
 .run_shell_command(cmd) ﾅG[: Command failed with exit code: 3
 ﾀsｪ~ｳﾜｵｽ
PS C:\Users\masaru\LightGBM2\LightGBM> Rscript build_r.R
* checking for file 'lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_2.3.2.tar.gz'

* installing to library 'C:/Users/masaru/Documents/R/win-library/3.6'
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to C:/Users/masaru/Documents/R/win-library/3.6/00LOCK-lightgbm/00new/lightgbm
[1] "Trying to build with: 'Visual Studio 16 2019'"
-- The C compiler identification is MSVC 19.25.28612.0
-- The CXX compiler identification is MSVC 19.25.28612.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/Hostx64/x64/cl.exe - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/Hostx64/x64/cl.exe - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
R version passed into FindLibR.cmake: 3.6.3
 警告メッセージ:
 normalizePath(path.expand(path), winslash, mustWork) で:
  path[1]="C:/PROGRA~1/R/R-36~1.3/lib": 指定されたファイルが見つかりません。
Creating R.lib and R.def
CMake Error at cmake/modules/FindLibR.cmake:40 (message):
  LIBR_LIB_DIR, 'C:/PROGRA~1/R/R-36~1.3/lib', not found
Call Stack (most recent call first):
  cmake/modules/FindLibR.cmake:194 (create_rlib_for_msvc)
  CMakeLists.txt:76 (find_package)
.....

[jameslamb]

Hi @masaru-tsuruta , thanks for the report and all the information on your environment!

Have you tried going into Windows Explorer and manually removing C:/Users/masaru/Documents/R/win-library/3.6/00LOCK-lightgbm before re-installing?

I have tried to run the below commands in CMD using VS tool

Our Visual Studio setup is not tested on every build yet (we are working on it! https://github.com/microsoft/LightGBM/pull/2936#issuecomment-612707832), so there might be some problems.

If deleting that directory doesn't work, could you please try this:

1. Open R-package/src/install.libs.R and set use_mingw <- TRUE
2. Be sure that the path to Rtools3.5 mingw is in your PATH environment variable. (link to Rtools if you don't have it). It will be something like C:\Rtools\mingw_64\bin
3. Open CMD and run Rscript build_r.R

[masaru-tsuruta]

Hi @jameslamb , thanks for your reply.

I have already tried to remove C:/Users/masaru/Documents/R/win-library/3.6/00LOCK-lightgbm before installing.
The same error is occurred after removing that.

I have also tried to install LGBM using mingw with below PATH environment variables.
C:\Rtools\mingw_64\bin and C:\MinGW\bin.

However, the below error is occurred

..../LightGBM>Rscript build_r.R
* checking for file 'lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_2.3.2.tar.gz'

* installing to library 'C:/Users/masaru/Documents/R/win-library/3.6'
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to C:/Users/masaru/Documents/R/win-library/3.6/00LOCK-lightgbm/00new/lightgbm
-- The C compiler identification is GNU 5.1.0
-- The CXX compiler identification is GNU 5.1.0
-- Check for working C compiler: C:/TDM-GCC-64/bin/gcc.exe
-- Check for working C compiler: C:/TDM-GCC-64/bin/gcc.exe - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/TDM-GCC-64/bin/g++.exe
-- Check for working CXX compiler: C:/TDM-GCC-64/bin/g++.exe - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
R version passed into FindLibR.cmake: 3.6.3
 警告メッセージ:
 normalizePath(path.expand(path), winslash, mustWork) で:
  path[1]="C:/PROGRA~1/R/R-36~1.3/lib": 指定されたファイルが見つかりません。
CMake Error at C:/Program Files/CMake/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:164 (message):
  Could NOT find LibR (missing: LIBR_CORE_LIBRARY)
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:445 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindLibR.cmake:202 (find_package_handle_standard_args)
  CMakeLists.txt:76 (find_package)


-- Configuring incomplete, errors occurred!
See also "C:/Users/masaru/AppData/Local/Temp/RtmpaoCrRA/R.INSTALL583c63ec1272/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
R version passed into FindLibR.cmake: 3.6.3
 警告メッセージ:
 normalizePath(path.expand(path), winslash, mustWork) で:
  path[1]="C:/PROGRA~1/R/R-36~1.3/lib": 指定されたファイルが見つかりません。
CMake Error at C:/Program Files/CMake/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:164 (message):
  Could NOT find LibR (missing: LIBR_CORE_LIBRARY)
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:445 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindLibR.cmake:202 (find_package_handle_standard_args)
  CMakeLists.txt:76 (find_package)


-- Configuring incomplete, errors occurred!
See also "C:/Users/masaru/AppData/Local/Temp/RtmpaoCrRA/R.INSTALL583c63ec1272/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
MINGW3~1.EXE: *** No rule to make target '_lightgbm'.  Stop.
 eval(ei, envir) ﾅG[: Cannot find lib_lightgbm.dll
* removing 'C:/Users/masaru/Documents/R/win-library/3.6/lightgbm'
 .run_shell_command(cmd) ﾅG[: Command failed with exit code: 1
 ﾀsｪ~ｳﾜｵｽ

From instruction site, if we use mingw, then the performance of LGBM is not efficient. Is this fiexed?

Warning for Windows users: it is recommended to use Visual Studio for its better multi-threading efficiency in Windows for many core systems. For very simple systems (dual core computers or worse), MinGW64 is recommended for maximum performance. If you do not know what to choose, it is recommended to use Visual Studio, the default compiler. Do not try using MinGW in Windows on many core systems. It may result in 10x slower results than Visual Studio.

[masaru-tsuruta]

Hi @jameslamb ,
I have tried to change the version of cmake(3.17.1 ->3.12.0).
Then, the installation with mingw has been scceeded.
However, its performance is not efficient. I cannot use all threads in my pc.

[jameslamb]

Hi @jameslamb ,
I have tried to change the version of cmake(3.17.1 ->3.12.0).
Then, the installation with mingw has been scceeded.
However, its performance is not efficient. I cannot use all threads in my pc.

Woo! Ok that is progress, I'm glad it at least works with mingw.

Could you try the version from https://github.com/microsoft/LightGBM/pull/2963?

git clone --recursive https://github.com/microsoft/LightGBM
cd LightGBM
git fetch origin r/msvc-support
Rscript build_r.R

I _hope_ that that will fix your issue, and if it does then we'll get it reviewed and merged as fast as possible.

[jameslamb]

but it is concerning that you had to downgrade CMake 😬

[masaru-tsuruta]

Thanks, some hope is in sight.

I have tried to run the below command.
git fetch origin r/msvc-support
Then, the below error is occurred.
fatal: Couldn't find remote ref r/msvc-support
Could you please advise me?

[jameslamb]

ah! A mistake in my code snippet, sorry. You need to clone from my fork since the PR is still in progress.

git clone --recursive https://github.com/jameslamb/LightGBM
cd LightGBM
git fetch origin r/msvc-support
Rscript build_r.R

[masaru-tsuruta]

I appreciate your quick response.
Since I downgraded cmake and this cmake don't support VS2019, I have installed VS2017 community before trying to run the new command.
And, I have added the bellows to PATH environment variable.

C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin

Then, I run the new comman. However, the bellow error occurred again ... :cry:

...\LightGBM>Rscript build_r.R
* checking for file 'lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_2.3.2.tar.gz'

* installing to library 'C:/Users/masaru/Documents/R/win-library/3.6'
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to C:/Users/masaru/Documents/R/win-library/3.6/00LOCK-lightgbm/00new/lightgbm
[1] "Trying to build with: 'Visual Studio 16 2019'"
CMake Error: Could not create named generator Visual Studio 16 2019
..... an omission of a middle part .....

[1] "Trying to build with: 'Visual Studio 15 2017'"
-- The C compiler identification is MSVC 19.16.27038.0
-- The CXX compiler identification is MSVC 19.16.27038.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
R version passed into FindLibR.cmake: 3.6.3
 �x�����b�Z�[�W:
 normalizePath(path.expand(path), winslash, mustWork) ��:
  path[1]="C:/PROGRA~1/R/R-36~1.3/lib": �w�肳�ꂽ�t�@�C�������‚���܂���B
Creating R.lib and R.def
CMake Error at cmake/modules/FindLibR.cmake:40 (message):
  LIBR_LIB_DIR, 'C:/PROGRA~1/R/R-36~1.3/lib', not found
Call Stack (most recent call first):
  cmake/modules/FindLibR.cmake:194 (create_rlib_for_msvc)
  CMakeLists.txt:76 (find_package)


-- Configuring incomplete, errors occurred!
See also "C:/Users/masaru/AppData/Local/Temp/RtmpwzZXcR/R.INSTALL76685f3b9a/lightgbm/src/build/CMakeFiles/CMakeOutput.log".

[jameslamb]

haha GAH I missed yet another line in the code I wrote in this issue. I bet if you run git branch -v right now you will still be on master.

You need to git checkout r/msvc-support. I know I forgot this because the code is complaining about LIBR_LIB_DIR, a variable I completely removed in #2963.

Ok so, for those arriving at this issue in the future, it is:

git clone --recursive https://github.com/jameslamb/LightGBM
cd LightGBM
git fetch origin r/msvc-support
git checkout r/msvc-support
Rscript build_r.R

Thanks for your patience

[masaru-tsuruta]

Thanks, for your continued support!!

I have tried to run new command.
Another message occurred. The code is complaining LIBR_CORE_LIBRARY.

................
R version passed into FindLibR.cmake: 3.6.3
Creating R.lib and R.def
CMake Error at cmake/modules/FindLibR.cmake:39 (message):
  LIBR_CORE_LIBRARY, 'LIBR_CORE_LIBRARY-NOTFOUND', not found
Call Stack (most recent call first):
  cmake/modules/FindLibR.cmake:187 (create_rlib_for_msvc)
  CMakeLists.txt:76 (find_package)

[jameslamb]

Ok I got out my Windows laptop and looked into this. Think I found the issue! Basically the find_library() function from CMake is not behaving how I expected it to. You can see what I had to change in this commit: https://github.com/microsoft/LightGBM/pull/2963/commits/9a6d720eac23dc78616bb25d9774268cfabe81c9

If you are still on branch r/msvc-support, run

git pull r/msvc-support

and then retry installation

It just worked for me with these specs on my laptop:

• Windows 10
• Visual Studio 16 2019
• R 3.6.1

[masaru-tsuruta]

It works well. Installation has been completed!
Thanks for your great support!! 😄

[jameslamb]

Great, thanks for the help! We'll try to get it merged soon so you don't have to rely on my fork.

[mgzde]

Hi everyone,

I'm using Windows 10, RStudio 3.6.3 and I'm trying to install "lightgbm" package in R. I have tried everything I guess but I can not install it. Does it work on RStudio and how can I install it successfully?
p.s: I'm new at this so I'd be grateful if you could explain it step by step.

Thank you so much.

[jameslamb]

@mgzde thank you for using LightGBM!

The official documentation for how to install the R pckage is at https://github.com/microsoft/LightGBM/blob/master/R-package/README.md

If the steps there do not work for you, please open a new issue by clicking "New Issue" at https://github.com/microsoft/LightGBM/issues and tell us what you are experiencing. We will do our best to help you get it working.

In the future, please do not post the same question in multiple issues here (https://github.com/microsoft/LightGBM/issues/2997#issuecomment-626378424). That fragments the conversation and will make it harder for maintainers to help you.