Dlib: Saving/resuming global search progress in Python

[tsoernes]

As far as I can tell, intermediate progress for find_min_global is stored in dlib::upper_bound_function object in class global_function_search, but the latter two does not seem exposed to the Python API. Is there any way to save/resume search, and is there a way to do parallel searches, in Python?

Thanks

[davisking]

Right, the Python API is simpler. The C++ API lets you do all those things
though.

[tsoernes]

Can I make a feature request for the above-mentioned functionality. I've
had great success using the LIPO search from your library on a
reinforcement learning problem compared to using Tree of Parzen Estimators.

2018-02-23 3:21 GMT+01:00 Davis E. King [e-mail]:

Right, the Python API is simpler. The C++ API lets you do all those things
though.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/davisking/dlib/issues/1147#issuecomment-367889585,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGd9xO_HaE1CICVNRWE9UBnV-hvCAPfeks5tXiCpgaJpZM4SQNDL
.

[davisking]

Save and resume is relatively straightforward, however, I don't really see how parallel search is going to work in Python because of the GIL.

[tsoernes]

I don't think that's a problem, you can just spawn a process for each parallel optimization run.

[davisking]

Ok, well, I'm not very familiar with how python deals with parallelism. If you want to set that up and submit a PR that would be cool :)

[tsoernes]

Sure, no problem. Once the Python bindings for global_function_search are there I can PR a couple of examples.

[aldanor]

Definitely interested in this as well, the Python side of global optimizer could use some love :)

[davisking]

I just pushed an update that adds the more detailed global_function_search interface to python. So now you can run this program, for instance:

import dlib
from math import pow

def F(a,b):
    return -pow(a-2,2.0) - pow(b-4,2.0);
def G(x):
    return 2-pow(x-5,2.0); 

spec_F = dlib.function_spec([-10,-10], [10,10])
spec_G = dlib.function_spec([-2], [6])

opt = dlib.global_function_search([spec_F, spec_G])

for i in range(15):
    next = opt.get_next_x()
    print("next x is for function {} and has coordinates {}".format(next.function_idx, next.x))

    if (next.function_idx == 0):
        a = next.x[0]
        b = next.x[1]
        next.set(F(a,b))
    else:
        x = next.x[0]
        next.set(G(x))

[x,y,function_idx] = opt.get_best_function_eval()

print("\nbest function was {}, with y of {}, and x of {}".format(function_idx,y,x))

assert(abs(y-2) < 1e-7)
assert(abs(x[0]-5) < 1e-7)
assert(function_idx==1)

@tsoernes show me something cool :)

[tsoernes]

Hmm. I pulled from master and did python setup.py install, but the new functions are not available. Am I missing something?

>>> import dlib
>>> dlib.function_spec
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'dlib' has no attribute 'function_spec'
>>> dlib.__version__
'19.9.99'

[davisking]

What is dlib.__time_compiled__? I wager you didn't really install it.

[tsoernes]

Ah yes, that's right. Because version number was unchanged since previous install it did not overwrite even though setuptools said Installed /home/torstein/anaconda3/lib/python3.6/site-packages/dlib-19.9.99-py3.6-linux-x86_64.egg
Did pip3 uninstall dlib then installed again, works now.

[dlib-issue-bot]

Warning: this issue has been inactive for 170 days and will be automatically closed on 2018-09-07 if there is no further activity.

If you are waiting for a response but haven't received one it's likely your question is somehow inappropriate. E.g. you didn't follow the issue submission instructions, or your question is easily answerable by reading the FAQ, dlib's documentation, or a Google search.

[dlib-issue-bot]

Notice: this issue has been closed because it has been inactive for 174 days. You may reopen this issue if it has been closed in error.

[mvphilip]

Hello, I see mention of saving/resuming state but I don't see how that's actually done. Is there any chance you could give an example? Is there a set_next_x() api available for example?

Thank you

[tsoernes]

@mvphilip You use optimizer.get_function_evaluations() to get the results, where optimizer is the
global-function search object. The results can be saved to disk using pickle or numpy, in the latter case with some preprocessing.
For a concrete example see https://github.com/tsoernes/gfsopt/blob/master/gfsopt/gfsopt.py

[mvphilip]

Thank you very much. I really appreciate it.