Lightgbm: What's the difference between refit method and train with init_model?

[abhishek-choudhary-guavus]

Refit method is giving same results as base trained model.
For Experiment part I am using 200k rows as train data and 700k rows as test data.

## LightGBM Base Model
lightGBM_clf = lgbm.train(params,lgbm.Dataset(x_train,label=y_train),
                          num_boost_round=500)

y_pred_wu = lightGBM_clf.predict(x_test)
auc_value = print_result(y_test, y_pred_wu,0) ## function returning auc_values; AUC output is 0.8130

Using refit on test data and trying to predict on same test data

estimator_refit_final = deepcopy(lightGBM_clf)
estimator_refit_final.refit(data=x_test, label=y_test)
y_pred_refit_final = estimator_refit_final.predict(x_test)
auc_value = print_result(y_test, y_pred_refit_final,0) ## AUC output is 0.8130

But, If I retrain on test data using (init_model = lightGBM_clf); It is showing gain in AUC which is expected.

What's difference between refit and init_model method and Why refit method is not making any changes in results.

[guolinke]

refer to https://github.com/microsoft/LightGBM/issues/1529, https://github.com/microsoft/LightGBM/issues/1473, https://github.com/microsoft/LightGBM/issues/2433

did you compare the content of estimator_refit_final and lightGBM_clf.
you can also try smallerdecay_rate.

[abhishek-choudhary-guavus]

Feaature Importance for both estimator_refit_methodand lightGBM_clf is same. Tried different decay_rate (["0.99","0.9","0.5"]), even results are same for different decay rates.

[guolinke]

@abhishek-choudhary-guavus
refit will not update the split points, so the feature importance will be the same. you should compare the leaf values. you can do it by saving the model into text format, you check the leaf values in the content.

[abhishek-choudhary-guavus]

get_leaf_output: Both models are giving exactly same leaf outputs for all (tree_id, leaf_id).

I also tried for very small train data and relatively larger test data (Train size: 500 only and Test Size: 15K)
Feature Importance and AUC curve etc. are all same for this scenario also. It seems like refit method is not doing anything

[guolinke]

@abhishek-choudhary-guavus refit is not an inplace function.
you should use its returned value as new booster:
https://github.com/microsoft/LightGBM/blob/2c18a0f3ed732df77d4304ed1d2b46a345d661c8/python-package/lightgbm/basic.py#L2681-L2683