Xgboost: Nightly releases

[RAMitchell]

I have had requests from some users for access to nightly releases from our CI. I believe it would simplify the workflow for many groups. This thread is to track the issue and discuss implementation.

As far as I know we do have packages being produced on master branch, I'm not sure if they are stored in any kind of persistent storage. I would imagine it would just be a case of archiving binaries on S3 and providing a link in documentation?

[hcho3]

@RAMitchell We store artifacts from 10 last commits: https://xgboost-ci.net/blue/organizations/jenkins/xgboost/detail/master/338/pipeline

[trivialfis]

@hcho3 Can we put out some short document in installation about this?

[randerzander]

I see that it's possible to manually find the download URL for this wheel and install it.

It was be really useful if there could be a pip package alias for the latest version of this nightly package so that CI or other script created environments can more easily find it.

For instance https://xgboost-ci.net/job/xgboost/job/master/344/artifact/python-package/dist/xgboost-1.0.0_SNAPSHOT-py2.py3-none-any.whl is the "latest" URL, but that "344" in the URL changes for every build, making it difficult to use in a conda environment yml file.

[JohnZed]

Replacing the build number with lastSuccessfulBuild seems to work. So the above would become https://xgboost-ci.net/job/xgboost/job/master/lastSuccessfulBuild/artifact/python-package/dist/xgboost-1.0.0_SNAPSHOT-py2.py3-none-any.whl

This will still be an issue when the artifact name changes from 1.0.0_SNAPSHOT to something else (1.1.0_SNAPSHOT maybe?).

[RAMitchell]

Should we put them in an S3 bucket with some consistent naming?

[hcho3]

@RAMitchell Yes, I think we should store artifacts in a S3 bucket and name it consistently (e.g. use commit IDs)

[randerzander]

Is it possible that CI could publish wheels for varying architectures?

I'm trying to use this nightly artifact with Google Colab, but on install I get:

ERROR: xgboost-1.0.0-SNAPSHOT-py2.py3-none-any.whl is not a supported wheel on this platform.

For reference, here's the Python and system info:

sys.version:
3.6.8 (default, Jan 14 2019, 11:02:34) 
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]]

print(platform.architecture())
('64bit', '')

print(platform.machine())
x86_64

[hcho3]

@randerzander We target x86-64 architecture exclusively. Can you check what processor architecture Google Colab uses? Currently we do not plan to build wheels for any other architecture.

[randerzander]

I edited my comment above. It looks like Colab _is_ x86_64

In case you want to try it directly, the command I'm using to try installing is:

!pip install https://xgboost-ci.net/job/xgboost/job/master/lastSuccessfulBuild/artifact/python-package/dist/xgboost-1.0.0-SNAPSHOT-py2.py3-none-any.whl

Result:

ERROR: xgboost-1.0.0-SNAPSHOT-py2.py3-none-any.whl is not a supported wheel on this platform.

Any suggestions on other information to get?

[hcho3]

@randerzander The wheel URL is wrong. Try

pip install https://xgboost-ci.net/job/xgboost/job/master/lastSuccessfulBuild/artifact/python-package/dist/xgboost-1.0.0_SNAPSHOT-py2.py3-none-any.whl

[hcho3]

I will submit a PR this weekend to upload all wheel artifacts to an S3 bucket, so that users can easily locate the wheels by the commit IDs. The expectation is that users can choose an exact commit from GitHub and install the corresponding XGBoost wheel with simple pip3 install command:

pip3 install https:/\/s3-us-west-2.amazonaws.com/xgboost-nightly-builds/xgboost-1.0.0+[commit hash]-py2.py3-none-manylinux1_x86_64.whl

[hcho3]

Nightly builds now available from https://s3-us-west-2.amazonaws.com/xgboost-nightly-builds/list.html.

[mirekphd]

Anyone care to share a current working link example / limitations (last available version etc)? The nightlies list page is empty, could not construct a working link using the docs.

[trivialfis]

In the doc you referred, there's a link https://s3-us-west-2.amazonaws.com/xgboost-nightly-builds/list.html

[mirekphd]

In the doc you reffed, there's a link https://s3-us-west-2.amazonaws.com/xgboost-nightly-builds/list.html

Thanks, I realized it's an issue with my apparently unsupported browser..., so got this listing page finally to work (by switching to incognito mode in Firefox).

However, the links format is not exactly as the docs specifies... there are several differences (two extra strings: "_SNAPSHOT" and "2010" and the "py2" token is gone - the docs need updating...):

[mir@CentOS-80-64-minimal ~]$ wget https://s3-us-west-2.amazonaws.com/xgboost-nightly-builds/xgboost-1.1.0_SNAPSHOT%2B057c762ecdabc97000f26223181d94f28ab28239-py3-none-manylinux2010_x86_64.whl
--2020-08-06 13:53:13--  https://s3-us-west-2.amazonaws.com/xgboost-nightly-builds/xgboost-1.1.0_SNAPSHOT%2B057c762ecdabc97000f26223181d94f28ab28239-py3-none-manylinux2010_x86_64.whl
Resolving s3-us-west-2.amazonaws.com (s3-us-west-2.amazonaws.com)... 52.218.184.8
Connecting to s3-us-west-2.amazonaws.com (s3-us-west-2.amazonaws.com)|52.218.184.8|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 128498134 (123M) [application/octet-stream]
Saving to: ‘xgboost-1.1.0_SNAPSHOT+057c762ecdabc97000f26223181d94f28ab28239-py3-none-manylinux2010_x86_64.whl’

                       xgboost-1.1.0_SNAPSHOT+057   0%[                                                                                                             ] 245.63K   100MB/s  

And what's really nice, according to the docs, these are not just CPU-only builds, but universal ones, that should enable training in NVIDIA GPUs as well if you have CUDA libraries installed (as in the case of catboost).