Xgboost: GPU Acceleration Phase 2

[RAMitchell]

We recently added a new fast histogram algorithm to the gpu plugin. Future efforts will focus on extending this to the multi gpu and multi node cases as well is integrating gpu algorithms into the main code base. We will also focus on making the algorithm more available to users (our build process is not user friendly) and testing to production level.

We will have contribution from from Nvidia and h2o.ai on this project.

Updated 2017/9/28

Completed

Fast hist algorithm extended to multi gpu
Extended unit tests
Improved exact GPU algorithm
Improved numerical precision
GPU accelerated prediction
Integration of GPU plug-in into main XGBoost code base
Support for R
Nvidia blog post

In progress

GPU continuous integration with python wheels available
Binaries/images available for multiple platforms
Improved histogram algorithm with loss guided mode

Nice to have

Multi node distributed algorithm (possible integration of NCCL 2.0)
Extended benchmarks (real datasets on dgx1/aws p2 systems)
DMLC blog post
Docker images available

[zhukunism]

Can this GPU updater be used by R xgboost package? The installation guide and test script is pretty much for python only. It would be super helpful if this GPU updater plugin can be used by the R part. Thanks!

[RAMitchell]

@hetong007 what would we have to do to support the R package? I believe the R package build process would not support compiling cuda code.

[hetong007]

@thirdwing Could you please help with the GPU + R combination based on your experience? Thanks.

[thirdwing]

It should be quite easy to use cuda code in R on Linux. I don't have any experience on Windows.

[tqchen]

We can do windows like we did for MXNet, just build libxgboost dll with MSVC and build the R version separately. That release would be different from what we did for CRAN, which uses amalgamation so it build as a whole library.

[tqchen]

We give kind support from AWS to reuse some of the CI infra structures for DMLC projects(mxnet etc). As long as we setup the jenkins tests, we can add those

[zhukunism]

@thirdwing, could we start with supporting GPU with xgboost R in linux? I am using Ubuntu. Thanks!

[Laurae2]

@RAMitchell you may do like I did for LightGBM with OpenCL GPU full support: https://github.com/Microsoft/LightGBM/blob/master/docs/GPU-Windows.md#installation-method-1

It requires to add small changes to the Makevars (enabling flags, asking user to add NVIDIA CUB to either the PATH or user Makeconf) and to declare the corresponding functions.

[SteveBronder]

Just wanted to bump this. Is there a doc for GPU support with xgboost in R? I have a problem where this would be v nice

[khotilov]

@Stevo15025 there's no official support for GPU xgboost with R yet.

[RAMitchell]

Gpu xgboost can be used through h2o r for now.

[arnocandel]

https://github.com/h2oai/h2o-3/blob/master/h2o-r/h2o-package/R/xgboost.R

[RAMitchell]

Updated roadmap. Regarding R package support, I have no plans to currently work on this but instead will seek contribution from the community.

If there is anyone interested in working on this let me know.

[khotilov]

I've update the roadmap with completed "Integration of GPU plug-in into main XGBoost code base" and "Support for R".

[RAMitchell]

Many thanks for your hard work @khotilov

[huucp]

@RAMitchell Would you build a gpu version for JVM packages?

[RAMitchell]

@superbobry could this be added as an option to the typical JVM installation? AFAIK you are already using cmake so it can't be too difficult.

[rknimmakayala]

Is there a link that explains the steps for installing on R+ windows7?

[khotilov]

@rknimmakayala http://xgboost.readthedocs.io/en/latest/build.html#installing-r-package-with-gpu-support

[trivialfis]

For "Improved histogram algorithm with loss guided mode", what does "improved" mean?