xgboost.core.XGBoostError: b"Invalid Parameter format for max_depth expect int but value

[weiboVictor]

Hello,

I am tuning xgboost regressor model with hyperopt, today a strange error just appeared "Invalid Parameter format for max_depth expect int ... ". The error still exist even though I removed "max_depth" from the param space.

Environment

I am using Anaconda 3 with python 3.4 on Windows 7

Below is a part of my testing code

def optimize(trials):
space = {
'n_estimators' : hp.quniform('n_estimators', 100, 1000, 1),
#'eta' : hp.quniform('eta', 0.025, 0.5, 0.025),
#'max_depth' : 6,
#'min_child_weight' : 1,
#'subsample' : hp.quniform('subsample', 0.5, 1, 0.05),
#'gamma' : hp.quniform('gamma', 0.5, 1, 0.05),
#'colsample_bytree' : hp.quniform('colsample_bytree', 0.5, 1, 0.05),
#'num_class' : 9,
#'eval_metric': 'mlogloss',
#'objective': 'multi:softprob',
#'nthread' : 6,
'silent' : 1
}

best = fmin(score, space, algo=tpe.suggest, trials=trials, max_evals=250)

print(best) 

def score(params):
print ("Training with params : ")
dtrain = xgb.DMatrix(X_train, label=y_train)
dvalid = xgb.DMatrix(X_test, label=y_test)
int_feat = ['max_depth','n_estimators']

for f in int_feat:
    if f in params:
        print('change to int')
        params[f] = int(params[f])
print (params)
# watchlist = [(dvalid, 'eval'), (dtrain, 'train')]
#model = xgb.train(params, dtrain, num_round)
model = xgb.XGBRegressor(params)

model.fit(X_train,y_train)
y_pred_cv = model.predict(X_test)
RMSE = np.sqrt(np.mean((y_pred_cv - y_test) ** 2))

print ("\tScore {0}\n\n".format(RMSE))
return {'loss': RMSE, 'status': STATUS_OK}

# Detailed error message

Training with params :
change to int
{'silent': 1, 'n_estimators': 745}
Traceback (most recent call last):
File "tune_models.py", line 136, in
optimize(trials)
File "tune_models.py", line 78, in optimize
best = fmin(score, space, algo=tpe.suggest, trials=trials, max_evals=250)
File "C:\Anaconda3\lib\site-packages\hyperopt\fmin.py", line 306, in fmin
return_argmin=return_argmin,
File "C:\Anaconda3\lib\site-packages\hyperopt\base.py", line 633, in fmin
return_argmin=return_argmin)
File "C:\Anaconda3\lib\site-packages\hyperopt\fmin.py", line 319, in fmin
rval.exhaust()
File "C:\Anaconda3\lib\site-packages\hyperopt\fmin.py", line 198, in exhaust
self.run(self.max_evals - n_done, block_until_done=self.async)
File "C:\Anaconda3\lib\site-packages\hyperopt\fmin.py", line 172, in run
self.serial_evaluate()
File "C:\Anaconda3\lib\site-packages\hyperopt\fmin.py", line 89, in serial_eva
luate
result = self.domain.evaluate(spec, ctrl)
File "C:\Anaconda3\lib\site-packages\hyperopt\base.py", line 838, in evaluate
rval = self.fn(pyll_rval)
File "tune_models.py", line 50, in score
model.fit(X_train,y_train)
File "C:\Anaconda3\lib\site-packages\xgboost-0.4-py3.5.egg\xgboost\sklearn.py"
, line 247, in fit
verbose_eval=verbose)
File "C:\Anaconda3\lib\site-packages\xgboost-0.4-py3.5.egg\xgboost\training.py
", line 177, in train
bst.update(dtrain, i, obj)
File "C:\Anaconda3\lib\site-packages\xgboost-0.4-py3.5.egg\xgboost\core.py", l
ine 749, in update
_check_call(_LIB.XGBoosterUpdateOneIter(self.handle, iteration, dtrain.handl
e))
File "C:\Anaconda3\lib\site-packages\xgboost-0.4-py3.5.egg\xgboost\core.py", l
ine 97, in _check_call
raise XGBoostError(_LIB.XGBGetLastError())
xgboost.core.XGBoostError: b"Invalid Parameter format for max_depth expect int b
ut value='{'silent': 1, 'n_estimators': 745}'"

[amdouni]

I've got the same bug

[tqchen]

If you read the error code, somehow the max_depth is set to the dict {'silent': 1, 'n_estimators': 745}. Please check if this indeed happen on the python side

[amdouni]

I'm using hyperopt and have assigned hp.quniform('') to max_depth which assign a float to max_depth instead of int as expected. I've solved this by using hp.choice.

[shahlaebrahimi]

I had the same problem so I used hp.choice('max_depth', np.arange(1, 14, dtype=int)). It solves the issue just in some iterations so again that error is reported.
I would greatly appreciate if you could let me know how to deal with it?
regards,

[bicepjai]

I have the same bug. is there any fix for this issue?
xgboost==0.6a2
also on
xgboost==0.6a1

[Interstella12]

when defining:
xgb_step = XGBClassifier(xgb_params)
I received the same error, and I solve it by doing this:
xgb_step = XGBClassifier(**xgb_params)

[iwangzhengchao]

I had the same problem, when do parameters tuning in XGBoost.
Below is a part of my testing code:

def model_cv(model, X, y, cv_fold=3, early_stopping_rounds=20, seed=3):
    dtrain = xgb.DMatrix(data=X, label=y)
    xgb_params = model.get_xgb_params()
    cv_result = xgb.cv(params=model.get_params(), dtrain=dtrain, num_boost_round=model.get_params()['n_estimators'], nfold=cv_fold, 
                       metrics='auc', seed=seed, callbacks=[
                           xgb.callback.print_evaluation(show_stdv=False),
                           xgb.callback.early_stop(early_stopping_rounds)])
    best_round = cv_result.shape[0]-1
    print('best_round:', best_round)
    return best_round


xgb_skt = XGBClassifier(max_depth=3, learning_rate=0.1, n_estimators=50, silent=0, 
              objective='binary:logistic', booster='gbtree', gamma=0, min_child_weight=1, max_delta_step=0, 
              subsample=1, colsample_bytree=1, colsample_bylevel=1, reg_alpha=0, 
              reg_lambda=1, scale_pos_weight=1,  random_state=0)
best_round = model_cv(xgb_skt, X_train, y_train)

Error message：
XGBoostError: b"Invalid Parameter format for seed expect int but value='None'"

[Ankit01Mishra]

I had the same problem so I used hp.choice('max_depth', np.arange(1, 14, dtype=int)). It solves the issue just in some iterations so again that error is reported.
I would greatly appreciate if you could let me know how to deal with it?
regards,

Thanks I resolved it,it;'s working.