Ax: Question: is the implementation of gen() for random forest planned?
hey,
i麓m writing my master thesis about bayesian optimization applied to a logistic simulation and want to benchmark GP-EI and RF-EI, because i麓m primary using discrete values. Currently RF in Ax has no generate-function. Is it planned in the near future or is there any workaround to add a gen-function to the RF-model?
thank you!
Philipp
Pzmijewski
All 3 comments
Hi @Pzmijewski! I'm not sure of the current plans around the RF model. Let me check in with the owner of that project, @bletham, and we should be able to have a response for you early next week.
sdsingh
on 1 Oct 2020
There aren't any current plans for implementing an RF-based BO method. The RF model in Ax is useful just for modeling and prediction, for instance as a baseline to which GPs model fit can be compared.
SMAC uses random forests in it's "HPO" mode (here is a usage example: https://github.com/automl/SMAC3/blob/master/examples/SMAC4HPO_acq_rosenbrock.py). It is the only RF-based BO method that I'm aware of.
The challenge with using an RF for BO is that they don't provide the extrapolative uncertainty that acquisition functions like EI use to ensure good exploration. Their predictions are also not differentiable so optimizing the acquisition function becomes harder than with a GP.
bletham
on 5 Oct 2020
Closing the issue, since this is unfortunately something that is not on our plan.
lena-kashtelyan
on 6 Oct 2020
Related issues
FelixNeutatz
路
4Comments
ksanjeevan
路
3Comments
Irynei
路
5Comments
winf-hsos
路
4Comments
natolambert
路
4Comments
Most helpful comment
There aren't any current plans for implementing an RF-based BO method. The RF model in Ax is useful just for modeling and prediction, for instance as a baseline to which GPs model fit can be compared.
SMAC uses random forests in it's "HPO" mode (here is a usage example: https://github.com/automl/SMAC3/blob/master/examples/SMAC4HPO_acq_rosenbrock.py). It is the only RF-based BO method that I'm aware of.
The challenge with using an RF for BO is that they don't provide the extrapolative uncertainty that acquisition functions like EI use to ensure good exploration. Their predictions are also not differentiable so optimizing the acquisition function becomes harder than with a GP.