Scikit-optimize: [Feature Request] Progressbar to estimate progress of `gp_optimize`

[jolespin]

Does anyone have any insight how one could add some type of progress bar or meter for gp_optimize? Many times I run it I have no idea if I should wait or not for the results by being too ambitious and making the values to high. This may be requesting too much but I think it will be extremely useful.

Is this possible? Is there some type of loop that we could tqdm to add progress?

[glouppe]

This should be possible through the callback API.

On Wed, 9 May 2018 at 20:58, Josh L. Espinoza [e-mail]
wrote:

Does anyone have any insight how one could add some type of progress bar
or meter for gp_optimize? Many times I run it I have no idea if I should
wait or not for the results by being too ambitious and making the values to
high.

Is this possible? Is there some type of loop that we could tqdm to add
progress?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/scikit-optimize/scikit-optimize/issues/674, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAdKS_qRSQdzoX9FWgt66iBFEotlpJuCks5twzxCgaJpZM4T40-N
.

[jolespin]

My first attempt after borrowing some code from here. It breaks on each iteration and doesn't update the current_minimum. Does anyone have a fix for this?

class ProgressBar(object):
    """
    Alternatively: Could call ProgBarLogger like in keras
    """
    def __init__(self, n_calls, file=sys.stderr):
        self.n_calls = n_calls
        self.iter_no = 0
        self.file = file
        self._start_time = time.time()

    def _to_precision(self, x, precision=5):
        return ("{0:.%ie} seconds" % (precision-1)).format(x)

    def progress(self, iter_no, curr_min):
        bar_len = 60
        filled_len = int(round(bar_len * iter_no / float(self.n_calls)))

        percents = round(100.0 * iter_no / float(self.n_calls), 1)
        bar = '=' * filled_len + '-' * (bar_len - filled_len)
        print('[%s] %s%% | Elapsed Time: %s | Current Minimum: %s' % (bar, percents, self._to_precision(time.time() - self._start_time), curr_min), file=self.file)
        self.file.flush()

    def __call__(self, res):
        curr_y = res.func_vals[-1]
        curr_min = res.fun

        while self.iter_no <= self.n_calls:
            self.progress(self.iter_no, curr_min)
            self.iter_no += 1

import numpy as np
from skopt import gp_minimize

def f(x):
    return (np.sin(5 * x[0]) * (1 - np.tanh(x[0] ** 2)) *
            np.random.randn() * 0.1)
n_calls = 25
res = gp_minimize(f, [(-2.0, 2.0)], n_calls=n_calls, callback=[ProgressBar(n_calls)])

# [------------------------------------------------------------] 0.0% | Elapsed Time: 3.7711e-03 seconds | Current Minimum: 0.000115824311481
# [==----------------------------------------------------------] 4.0% | Elapsed Time: 6.9220e-03 seconds | Current Minimum: 0.000115824311481
# [=====-------------------------------------------------------] 8.0% | Elapsed Time: 9.2552e-03 seconds | Current Minimum: 0.000115824311481
# [=======-----------------------------------------------------] 12.0% | Elapsed Time: 1.1648e-02 seconds | Current Minimum: 0.000115824311481
# [==========--------------------------------------------------] 16.0% | Elapsed Time: 1.4172e-02 seconds | Current Minimum: 0.000115824311481
# [============------------------------------------------------] 20.0% | Elapsed Time: 1.6934e-02 seconds | Current Minimum: 0.000115824311481
# [==============----------------------------------------------] 24.0% | Elapsed Time: 2.0049e-02 seconds | Current Minimum: 0.000115824311481
# [=================-------------------------------------------] 28.0% | Elapsed Time: 2.2993e-02 seconds | Current Minimum: 0.000115824311481
# [===================-----------------------------------------] 32.0% | Elapsed Time: 2.6198e-02 seconds | Current Minimum: 0.000115824311481
# [======================--------------------------------------] 36.0% | Elapsed Time: 2.9524e-02 seconds | Current Minimum: 0.000115824311481
# [========================------------------------------------] 40.0% | Elapsed Time: 3.2479e-02 seconds | Current Minimum: 0.000115824311481
# [==========================----------------------------------] 44.0% | Elapsed Time: 3.5749e-02 seconds | Current Minimum: 0.000115824311481
# [=============================-------------------------------] 48.0% | Elapsed Time: 3.8750e-02 seconds | Current Minimum: 0.000115824311481
# [===============================-----------------------------] 52.0% | Elapsed Time: 4.1820e-02 seconds | Current Minimum: 0.000115824311481
# [==================================--------------------------] 56.0% | Elapsed Time: 4.5218e-02 seconds | Current Minimum: 0.000115824311481
# [====================================------------------------] 60.0% | Elapsed Time: 4.8471e-02 seconds | Current Minimum: 0.000115824311481
# [======================================----------------------] 64.0% | Elapsed Time: 5.1492e-02 seconds | Current Minimum: 0.000115824311481
# [=========================================-------------------] 68.0% | Elapsed Time: 5.4785e-02 seconds | Current Minimum: 0.000115824311481
# [===========================================-----------------] 72.0% | Elapsed Time: 5.7333e-02 seconds | Current Minimum: 0.000115824311481
# [==============================================--------------] 76.0% | Elapsed Time: 5.9683e-02 seconds | Current Minimum: 0.000115824311481
# [================================================------------] 80.0% | Elapsed Time: 6.2042e-02 seconds | Current Minimum: 0.000115824311481
# [==================================================----------] 84.0% | Elapsed Time: 6.4593e-02 seconds | Current Minimum: 0.000115824311481
# [=====================================================-------] 88.0% | Elapsed Time: 6.7551e-02 seconds | Current Minimum: 0.000115824311481
# [=======================================================-----] 92.0% | Elapsed Time: 7.0077e-02 seconds | Current Minimum: 0.000115824311481
# [==========================================================--] 96.0% | Elapsed Time: 7.2763e-02 seconds | Current Minimum: 0.000115824311481
# [============================================================] 100.0% | Elapsed Time: 7.5439e-02 seconds | Current Minimum: 0.000115824311481

[betatim]

Maybe take a look at tqdm for the actual output of the progress bar. We'd get jupyter notebook support "for free" this way.

For debugging purposes could you look at all the evaluation results so far? This is what [this callback}(https://github.com/scikit-optimize/scikit-optimize/blob/1c4c0f12ad8c1fe0a33542108fc0e55164138f9d/skopt/callbacks.py#L202-L223) does. Either we get lucky and the best minimum is found on iteration one or there is a bug in setting that property of the result object :-/

[LucaCappelletti94]

A simple solution with tqdm, for the jupyter notebook in this case, could be:

from tqdm import tqdm_notebook as tqdm

class tqdm_skopt(object):
    def __init__(self, **kwargs):
        self._bar = tqdm(**kwargs)

    def __call__(self, res):
        self._bar.update()

def f(x):
    return (np.sin(5 * x[0]) * (1 - np.tanh(x[0] ** 2)) *
            np.random.randn() * 0.1)

n_calls = 25
res = gp_minimize(f, [(-2.0, 2.0)], n_calls=n_calls, callback=[tqdm_skopt(total=n_calls, desc="Gaussian Process")])

Enjoy :)

[yinanzhangepfl]

Hi,
Thank you very much for sharing your solution. I have tried your method. It is useful but when I run my code in the command line, the progress bar is mixed up with other outputs.

Does anyone have any suggestions for this?

[Zulex]

Did anyone get to make this work in any local IDE like pycharm? Never worked with tqdm but it's not showing anything.