Lightgbm: Lightgbm not giving same results for same environments on different computers

[MuhammedBuyukkinaci]

How you are using LightGBM?

LightGBM component: Installed by conda

Environment info

Operating System: Ubuntu 16.04 LTS

CPU/GPU model: i7 7th generation

gcc version: 5.4.0

Java version:

Python version: 3.7.2

LightGBM version or commit hash:2.3.1

Our findings

For different computers, we built up the same environment. We ran the same piece of lightgbm code and saw different results many times.

We googled and investigated why we got differents results for different computers. The point that we arrive is as follows:

When number of threads in lightgbm set more than 1, the determinism disappears. The results are changing on different computers. When we set number of threads parameter to 1, all of us(3 people) are getting the same result, even the 10th decimal after comma is the same.

[guolinke]

Before v2.3.1 (included), when using bagging, you should use the same num_threads for determinism.
And we fix this in the master branch recently.

[anilcelikurbanstat]

Hi @guolinke,

Different computers with same environments returned different results for us even if the num_threads parameter was set to the same number. We all use 2.3.1 and our environments are identical in python, the only difference is our hardware.

Do yo believe that this will be fixed if we use the latest available version on master branch?

[guolinke]

@anilcelikurbanstat when num_threads is fixed (and the same parameters), the results should be determinism, even in different machines, in old versions like 2.3.1.
you said "When we set number of threads parameter to 1, all of us(3 people) are getting the same result, even the 10th decimal after comma is the same."
Did you try to disable bagging and test again?
Do you have other tools, which use openmp, used in the pipeline ?