Tpot: Parallelize computation via DEAP?

[fferroni]

Hello,

As I understand it, TPOT uses DEAP library in the backend. This library lists two options of parallelizing its toolbox computations, either using SCOOP or multiprocessing libraries.
https://github.com/DEAP/deap/blob/232ed17142da1c8bb6a39179018f8278b122aada/doc/tutorials/basic/part4.rst

I am wondering whether this has been considered as an avenue for parallelising TPOT, and if yes, if it's been successful?

Thanks!

[weixuanfu]

I tried to use both SCOOP and multiprocessing to parallelize TPOT but both did not work due to some functions (like lambda func) are not pickleable. So I used pathos to parallelize TPOT and it use dill instead of pickle for function serialization (check this for more details) . Check this PR #338 for a demo for parallelizing TPOT.

[fferroni]

Hi,
Thanks! Looks good. The Travis build fails though?
Best,
Francesco

[weixuanfu]

I think it is because the Travis build does not have the pathos module.

On Jan 11, 2017, at 5:54 AM, fferroni [e-mail] wrote:

Hi,
Thanks! Looks good. The Travis build fails though?
Best,
Francesco

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[rhiever]

We're closing all questions that haven't been updated in a while. Please feel free to re-open this issue if your issue persists.