Optuna: njobs

[uvinetz]

Does the parameter njobs in the optimize function actually perform parallel trials? There is no reference to it in the Distributed optimization section in the docs. When I tried it (using sqlite for storing the study), it didn't print any output although when I continued the optimization, from the number of trials it did seem to run the parallel trials.

[toshihikoyanase]

Thank you for your question.

Does the parameter njobs in the optimize function actually perform parallel trials?

Yes, Optuna supports the parallel execution of trials, but it has some limitations.
More concretely, only the multi-threading has been supported. By specifying n_jobs=n, Optuna spawns n threads to execute the optimization. Due to the Python's GIL, n_jobs will be effective for I/O heavy workload, but it will not for CPU-intensive workload.

In addition, we can change the parallelization backend including loky and dask-ml because we employed joblib in #692. But, some components (i.e., most of the samplers and pruners) are not ready for such backends. I prepared an example notebook to explain the limitations, so please take a look at it if you're interested in the details: https://colab.research.google.com/drive/18SlBT9p_FYFxiZlVjSo21Izb05JnwEad

There is no reference to it in the Distributed optimization section in the docs.

Sorry for the inconvenience. We'll add the document after fixing the samplers and pruners since the current situation about n_jobs is complicated.

[uvinetz]

Thanks! that helped me a lot

[toshihikoyanase]

I'm glad I could help you.
Let me close this issue. Please feel free to reopen it if you have further questions.