Distributed: Error starting up `Client`

[stefanseefeld]

The following chunk of code fails:

from distributed.client import *
import time

client = Client()

time.sleep(5)
print('five seconds later')

with:

distributed.utils - ERROR - 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

(plus a long traceback)

[mrocklin]

Interestingly this only occurs if you add the import *

If you just do

from distributed import Client

Then this runs fine

[mrocklin]

The minimal failing example appears to be as follows:

from distributed.client import *
import time

client = Client()

[mrocklin]

Nope, I take it back. I can reproduce with the clean import

[mrocklin]

This may be resolved by #515

[mrocklin]

This is, I believe, resolved. Thanks for reporting!

[stsievert]

I believe I ran into this same issue, resolved by updating from 1.13.0 to 1.14.3.

from distributed import Client

client = Client()
ncores = sum([c for _, c in client.ncores().items()])
print(ncores)
# prints 0

client = Client('127.0.0.1:8786')
ncores = sum([c for _, c in client.ncores().items()])
print(ncores)
# prints 32

[wholmgren]

I see this issue (the original post) on mac and linux machines with distributed 1.16.3.

edited to add: This is an issue when trying to run the code in a module. It works fine in the python interpreter or in jupyter notebooks.

[mrocklin]

By default Client() forks processes. If you use it within a module you should hide it within the __main__ block.

# client = Client()

if __name__ == '__main__':
    client = Client()
    ...

Alternatively you can choose to use threads safely.

client = Client(processes=False)

[wholmgren]

Ah, of course. Thanks. Sorry for the noise.