Joblib: Windows permission error

[albertcthomas]

I have a permission error while using Parallel(n_jobs=2) on windows:
````python-traceback
~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packagesjoblib\disk.py:122: UserWarning: Unable to delete folder xxxx\AppData\Local\Tempjoblib_memmapping_folder_15364_9553521538 after 5 tentatives.

.format(folder_path, RM_SUBDIRS_N_RETRY))

PermissionError Traceback (most recent call last)
~\Desktop\all_repos\gitlab_projects\wireless_optim\code\complete_experiment.py in ()
422 simulator_data = np.concatenate([simulator_data, new_simulator_data],
423 axis=0)
--> 424 n_admissible_actions.append(n_admissible_actions_d)
425
426 if 0:

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packagesjoblibparallel.py in __exit__(self, exc_type, exc_value, traceback)
664
665 def __exit__(self, exc_type, exc_value, traceback):
--> 666 self._terminate_backend()
667 self._managed_backend = False
668

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packagesjoblibparallel.py in _terminate_backend(self)
694 def _terminate_backend(self):
695 if self._backend is not None:
--> 696 self._backend.terminate()
697
698 def _dispatch(self, batch):

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packagesjoblib_parallel_backends.py in terminate(self)
528 # in latter calls but we free as much memory as we can by deleting
529 # the shared memory
--> 530 delete_folder(self._workers._temp_folder)
531 self._workers = None
532

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packagesjoblib\disk.py in delete_folder(folder_path, onerror)
113 while True:
114 try:
--> 115 shutil.rmtree(folder_path, False, None)
116 break
117 except (OSError, WindowsError):

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\shutil.py in rmtree(path, ignore_errors, onerror)
492 os.close(fd)
493 else:
--> 494 return _rmtree_unsafe(path, onerror)
495
496 # Allow introspection of whether or not the hardening against symlink

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\shutil.py in _rmtree_unsafe(path, onerror)
387 os.unlink(fullname)
388 except OSError:
--> 389 onerror(os.unlink, fullname, sys.exc_info())
390 try:
391 os.rmdir(path)

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\shutil.py in _rmtree_unsafe(path, onerror)
385 else:
386 try:
--> 387 os.unlink(fullname)
388 except OSError:
389 onerror(os.unlink, fullname, sys.exc_info())

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'xxxx\AppData\Local\Temp\joblib_memmapping_folder_15364_9553521538\15364-3031207306352-94b02d62d9b44d709a9a405235589ead.pkl'
````
Is this related to the fact that removing a memmap can fail on Windows as said in the doc?
I can try to have a small reproducible example.

I'm using joblib 0.13.
(FWIW this code is working on Ubuntu 16.04.5 LTS)

[tomMoral]

Yes this is specific to windows and its related to removing the memmaps folder.
We should probably try to make it more robust. What I don't understand is why we do not ctach the permission error line 117:

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packages\joblib\disk.py in delete_folder(folder_path, onerror)
    113             while True:
    114                 try:
--> 115                     shutil.rmtree(folder_path, False, None)
    116                     break
    117                 except (OSError, WindowsError):

[albertcthomas]

This is a random error (which you might already know).

What I don't understand is why we do not ctach the permission error line 117

I think this is because deleting the foder fails 5 times as explained by the UserWarning

UserWarning: Unable to delete folder xxxx\AppData\Local\Temp\joblib_memmapping_folder_15364_9553521538 after 5 tentatives.
  .format(folder_path, RM_SUBDIRS_N_RETRY))

and as can be understood when looking at the whole function code in joblib/disk.py

def delete_folder(folder_path, onerror=None):
    """Utility function to cleanup a temporary folder if it still exists."""
    if os.path.isdir(folder_path):
        if onerror is not None:
            shutil.rmtree(folder_path, False, onerror)
        else:
            # allow the rmtree to fail once, wait and re-try.
            # if the error is raised again, fail
            err_count = 0
            while True:
                try:
                    shutil.rmtree(folder_path, False, None)
                    break
                except (OSError, WindowsError):
                    err_count += 1
                    if err_count > RM_SUBDIRS_N_RETRY:
                        warnings.warn(
                            "Unable to delete folder {} after {} tentatives."
                            .format(folder_path, RM_SUBDIRS_N_RETRY))
                        raise
                    time.sleep(RM_SUBDIRS_RETRY_TIME)

[albertcthomas]

If I enter the debugger and try to run os.unlink(fullname) I still get the PermissionError but after a few minutes no error is raised:

ipdb> print(time.strftime("%I %M %S %p",time.localtime(time.time()))), os.unlink(fullname)
11 37 31 AM
*** PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'xxxx\\AppData\\Local\\Temp\\joblib_memmapping_folder_7028_801144278\\7028-2594394109152-8f6b4e3f1d074e2dbc6346190273d9df.pkl'
ipdb> print(time.strftime("%I %M %S %p",time.localtime(time.time()))), os.unlink(fullname)
11 37 34 AM
*** PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'xxxx\\AppData\\Local\\Temp\\joblib_memmapping_folder_7028_801144278\\7028-2594394109152-8f6b4e3f1d074e2dbc6346190273d9df.pkl'
ipdb> print(time.strftime("%I %M %S %p",time.localtime(time.time()))), os.unlink(fullname)
11 37 52 AM
*** PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'xxxx\\AppData\\Local\\Temp\\joblib_memmapping_folder_7028_801144278\\7028-2594394109152-8f6b4e3f1d074e2dbc6346190273d9df.pkl'
ipdb> print(time.strftime("%I %M %S %p",time.localtime(time.time()))), os.unlink(fullname)
11 38 57 AM
*** PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'xxxx\\AppData\\Local\\Temp\\joblib_memmapping_folder_7028_801144278\\7028-2594394109152-8f6b4e3f1d074e2dbc6346190273d9df.pkl'
ipdb> print(time.strftime("%I %M %S %p",time.localtime(time.time()))), os.unlink(fullname)
11 41 46 AM
*** PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'xxxx\\AppData\\Local\\Temp\\joblib_memmapping_folder_7028_801144278\\7028-2594394109152-8f6b4e3f1d074e2dbc6346190273d9df.pkl'
ipdb> print(time.strftime("%I %M %S %p",time.localtime(time.time()))), os.unlink(fullname)
11 47 18 AM
(None, None)

[albertcthomas]

Issue scikit-learn/scikit-learn#12546 is giving something closer to a MVCE, but depends on unknown data.

[ogrisel]

The memory mapped files gets closed at some point (when garbage collected) but we have no control when and under windows deleting an open file raises a PermissionError.

At the very least we should catch the permission error (I don't understand why the existing code does not catch it either) to turn it into a warning. We could also try to call gc.collect() and try to call os.unlink(fullname) one more time.

[albertcthomas]

Thanks for the details. For now I'm using Parallel with max_nbytes=None.

[ogrisel]

If you have a windows at hand I would be curious to know if the following can reproduce the issue:

import numpy as np
from joblib import Parallel, delayed

data = np.ones(int(2e6))
for i in range(3):
    results = Parallel(n_jobs=2)(delayed(np.sum)(data) for i in range(4))

[albertcthomas]

At the very least we should catch the permission error (I don't understand why the existing code does not catch it either) to turn it into a warning.

I am not sure to understand: the delete_folder function will raise the error even when the warning is raised because of the raise statement:

                try:
                    shutil.rmtree(folder_path, False, None)
                    break
                except (OSError, WindowsError):
                    err_count += 1
                    if err_count > RM_SUBDIRS_N_RETRY:
                        warnings.warn(
                            "Unable to delete folder {} after {} tentatives."
                            .format(folder_path, RM_SUBDIRS_N_RETRY))
                        raise

[albertcthomas]

I cannot reproduce the issue on Windows with the code of the above comment. I will try to play a bit more with it to see if I can succeed to reproduce it.

[ogrisel]

Ah ok, maybe we should just remove the raise statement then.

We also need to setup a garbage collector for the joblib memmap folder to delete old folders of python processes that died or where killed without cleaning such files properly.

[albertcthomas]

I finally have a snippet reproducing the error.

1. This fails

import numpy as np
from joblib import Parallel, delayed


def test_data(data):
    data_view = data[0:20]
    return data_view


data = np.ones(int(2e6))
results = Parallel(n_jobs=2, verbose=5)(
    delayed(test_data)(data) for _ in range(10))

Results:

[Parallel(n_jobs=2)]: Using backend LokyBackend with 2 concurrent workers.
[Parallel(n_jobs=2)]: Done  10 out of  10 | elapsed:    0.3s remaining:    0.0s
[Parallel(n_jobs=2)]: Done  10 out of  10 | elapsed:    0.3s finished
C:\xxxx\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packages\joblib\disk.py:122: UserWarning: Unable to delete folder C:\xxxx\AppData\Local\Temp\joblib_memmapping_folder_21368_6181193125 after 5 tentatives.
  .format(folder_path, RM_SUBDIRS_N_RETRY))
---------------------------------------------------------------------------
PermissionError                           Traceback (most recent call last)
<ipython-input-1-802489859579> in <module>()
     10 data = np.ones(int(2e6))
     11 results = Parallel(n_jobs=2, verbose=5)(
---> 12     delayed(test_data)(data) for _ in range(10))

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packages\joblib\parallel.py in __call__(self, iterable)
    938                 self._backend.stop_call()
    939             if not self._managed_backend:
--> 940                 self._terminate_backend()
    941             self._jobs = list()
    942             self._pickle_cache = None

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packages\joblib\parallel.py in _terminate_backend(self)
    694     def _terminate_backend(self):
    695         if self._backend is not None:
--> 696             self._backend.terminate()
    697
    698     def _dispatch(self, batch):

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packages\joblib\_parallel_backends.py in terminate(self)
    528             # in latter calls but we free as much memory as we can by deleting
    529             # the shared memory
--> 530             delete_folder(self._workers._temp_folder)
    531             self._workers = None
    532

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\site-packages\joblib\disk.py in delete_folder(folder_path, onerror)
    113             while True:
    114                 try:
--> 115                     shutil.rmtree(folder_path, False, None)
    116                     break
    117                 except (OSError, WindowsError):

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\shutil.py in rmtree(path, ignore_errors, onerror)
    492             os.close(fd)
    493     else:
--> 494         return _rmtree_unsafe(path, onerror)
    495
    496 # Allow introspection of whether or not the hardening against symlink

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\shutil.py in _rmtree_unsafe(path, onerror)
    387                 os.unlink(fullname)
    388             except OSError:
--> 389                 onerror(os.unlink, fullname, sys.exc_info())
    390     try:
    391         os.rmdir(path)

~\AppData\Local\Continuum\anaconda3\envs\wireless_optim\lib\shutil.py in _rmtree_unsafe(path, onerror)
    385         else:
    386             try:
--> 387                 os.unlink(fullname)
    388             except OSError:
    389                 onerror(os.unlink, fullname, sys.exc_info())

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'xxxx\\AppData\\Local\\Temp\\joblib_memmapping_folder_21368_6181193125\\21368-2477599267192-6d81b296704d47bfbd81102522ddd9a5.pkl'

1. This does not fail:

import numpy as np
from joblib import Parallel, delayed


def test_data(data):
    data_view = data[0:20]
    return np.sum(data_view)


data = np.ones(int(2e6))
results = Parallel(n_jobs=2, verbose=5)(
    delayed(test_data)(data) for _ in range(10))

Results

[Parallel(n_jobs=2)]: Using backend LokyBackend with 2 concurrent workers.
[Parallel(n_jobs=2)]: Done  10 out of  10 | elapsed:    0.3s remaining:    0.0s
[Parallel(n_jobs=2)]: Done  10 out of  10 | elapsed:    0.3s finished

[albertcthomas]

For the failing snippet, if I close the ipython session I can immediately delete the folder which seems to confirm that the program using the file is indeed python and not another program.

[ogrisel]

Thank you very much for the minimal reproduction case, this is going to be very helpful. I am currently recreating a Windows VM to debug this kind of issues on my laptop.

[albertcthomas]

I am currently recreating a Windows VM to debug this kind of issues on my laptop.

Let me know if you want me to test some code.

However this error seems to be an expected behavior: as far as I understand data_view increments the reference count and the memmap cannot be completely closed. I have not read the whole discussion but issue astropy/astropy#7404 seems similar.

[albertcthomas]

In case removing the raise statement would be a solution (see comment) maybe using the tempfile module would be a solution for the following

We also need to setup a garbage collector for the joblib memmap folder to delete old folders of python processes that died or where killed without cleaning such files properly.

as such temporary files are deleted as soon as they are closed.

[ogrisel]

The problem of the tempfile module is that it's a python level construct: it cannot track system level usage of a memory mapped buffer whose open file descriptors are used across several Python processes (main process and worker processes).

[HansolSL]

I had the same problem.
When I installed "cloudpickle" module and imported, the problem solved.

[robintwhite]

I am seeing this same error for joblib 0.13 in python 2.7.15 on Windows. I am using code for an image processing algorithm developed a few years ago which might have originally been on Ubuntu. Is there any suggested fix?
The line giving the same error is:
res= Parallel(n_jobs=num_parallel,verbose=100)(delayed(minimizer_downsampled)(partnum,step) for partnum in producer_parallel())
As you can see delayed is used here. Any help would be greatly appreciated

[albertcthomas]

@robintwhite for now, if this is possible in your case, you can disable memmaping by setting max_nbytes=None in Parallel

[robintwhite]

@albertcthomas Thank you, I think that has solved that issue - I have run into another one but I don't think it is related

[cliffrunner]

having the same issue here but the solution from @albertcthomas works for me

[albertcthomas]

This solution is just a workaround at it disables memmaping

[albertcthomas]

I retried the code which lead me to open this issue and in which I was passing a pandas dataframe to the function to parallelize. If I use a numpy array instead of the pandas dataframe (using pandas.DataFrame.values) I no longer have the PermissionError. Note that I also retried the original code using a pandas dataframe and I still get a PermissionError.

I tried this in light of the discussion of the related scikit-learn issue scikit-learn/scikit-learn#12546 and where I discovered that the PermissionError was raised with pandas dataframes. See this comment for a reproducible example using the scikit-learn function RandomizedSearchCV.

This won't fix the reproducible example that I gave above (see this comment) but this might be useful to fix the related scikit-learn issue as we can control the inputs and outputs of the function passed to the delayed function called in the scikit-learn source code.

[tyokota]

It is possible to somehow suppress the need for the temp folder to be deleted? I can set the tmp folder myself using the temp_folder parameter in Parallel() so I would know where/what to delete afterwards...

[zhanghongyong123456]

我也遇到同样问题

[Krzysiaczek99]

I have the same error. Initially warning and error later. Any idea ?? Setting max_nbytes=None is not applicable for me as I don't have enough memory so must use memmaping

Pickling array (shape=(1,), dtype=object).
Pickling array (shape=(1,), dtype=object).
Pickling array (shape=(1,), dtype=object).
[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:  2.7min remaining:    0.0s
[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:  2.7min finished
C:\ProgramData\Anaconda3\lib\site-packages\joblib\disk.py:122: UserWarning: Unable to delete folder C:\Users\KRZYSZ~1.FAJ\AppData\Local\Temp\1\joblib_memmapping_folder_36852_3554356199 after 5 tentatives.
  .format(folder_path, RM_SUBDIRS_N_RETRY))
Traceback (most recent call last):

  File "<ipython-input-27-996e5fda73ca>", line 8, in <module>
    delayed(create_stat_parallelML)(df_full_split, smoother, sensML, gainML, nstdML, window, k, num_cores) for k in range(0, len(ranges_list))))

  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\parallel.py", line 944, in __call__
    self._terminate_backend()

  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\parallel.py", line 696, in _terminate_backend
    self._backend.terminate()

  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\_parallel_backends.py", line 530, in terminate
    delete_folder(self._workers._temp_folder)

  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\disk.py", line 115, in delete_folder
    shutil.rmtree(folder_path, False, None)

  File "C:\ProgramData\Anaconda3\lib\shutil.py", line 507, in rmtree
    return _rmtree_unsafe(path, onerror)

  File "C:\ProgramData\Anaconda3\lib\shutil.py", line 391, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())

  File "C:\ProgramData\Anaconda3\lib\shutil.py", line 389, in _rmtree_unsafe
    os.unlink(fullname)

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\KRZYSZ~1.FAJ\\AppData\\Local\\Temp\\1\\joblib_memmapping_folder_36852_3554356199\\36852-2668685291928-000f27bc27894873a5937850b6ca0ce6.pkl'

[Krzysiaczek99]

'0.13.1'

[albertcthomas]

Could you please try with 0.13.2? And it seems that you are using pandas dataframes? If that's the case have you tried what is described in the comment above?

[albertcthomas]

Based on what I tried, I also think that it's important to restart the ipython session before trying a new code, see this scikit-learn comment

[Krzysiaczek99]

I did it already. I deleted folder manually and started from scratch. Its quite possible that this error occurs due to full RAM memory during processing, however I beleive in tis case data should be send to swap. I also found that under windows its not possible to use custom python module i.e. even import statement is in worker code it does not find module functions. For Linux is OK. Binary of module is of course in working directory in both cases and for 1 job works perfect (and multiple jobs for Linux)

[Krzysiaczek99]

I will try with 0.13.2 and yes pandas dataframes

[Krzysiaczek99]

the same

Pickling array (shape=(1,), dtype=object).
Pickling array (shape=(1,), dtype=object).
[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:  3.0min remaining:    0.0s
[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:  3.0min finished
C:\ProgramData\Anaconda3\lib\site-packages\joblib\disk.py:122: UserWarning: Unable to delete folder C:\Users\KRZYSZ~1.FAJ\AppData\Local\Temp\1\joblib_memmapping_folder_10792_2433262334 after 5 tentatives.
  .format(folder_path, RM_SUBDIRS_N_RETRY))
Traceback (most recent call last):

  File "<ipython-input-16-996e5fda73ca>", line 8, in <module>
    delayed(create_stat_parallelML)(df_full_split, smoother, sensML, gainML, nstdML, window, k, num_cores) for k in range(0, len(ranges_list))))

  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\parallel.py", line 944, in __call__
    self._terminate_backend()

  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\parallel.py", line 696, in _terminate_backend
    self._backend.terminate()

  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\_parallel_backends.py", line 530, in terminate
    delete_folder(self._workers._temp_folder)

  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\disk.py", line 115, in delete_folder
    shutil.rmtree(folder_path, False, None)

  File "C:\ProgramData\Anaconda3\lib\shutil.py", line 507, in rmtree
    return _rmtree_unsafe(path, onerror)

  File "C:\ProgramData\Anaconda3\lib\shutil.py", line 391, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())

  File "C:\ProgramData\Anaconda3\lib\shutil.py", line 389, in _rmtree_unsafe
    os.unlink(fullname)

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\KRZYSZ~1.FAJ\\AppData\\Local\\Temp\\1\\joblib_memmapping_folder_10792_2433262334\\10792-1630601519456-0038b82e984f484488f2e7beaed69b42.pkl'




joblib.__version__
Traceback (most recent call last):

  File "<ipython-input-17-51223e7b14d0>", line 1, in <module>
    joblib.__version__

NameError: name 'joblib' is not defined




import joblib

joblib.__version__
Out[19]: '0.13.2'

[albertcthomas]

Ok, if it does not work with the latest version of joblib and when you convert your pandas dataframes to numpy arrays you need to wait for a better solution. Do you have a reproducible example of your error?

[Krzysiaczek99]

it is always reproducible but not in short form but I can make some tracing for you if you want. What about this import of custom module which don't work for worker code under windows ?? Shall I open a new ticket ??

[draimundo]

Hi guys!
I also have this problem, using
numpy 1.16.3,scipy 1.2.1,scikit-learn 0.20.3,python 3.7.3, and joblib 0.13.0 (included in scikit-learn). I made sure to have everything in numpy arrays, and also tried to use a parallel instance with max_nbytes=None as input parameter, but no success.

My workaround was to comment out delete_folder(self._workers._temp_folder):

    def terminate(self):
        if self._workers is not None:
            # Terminate does not shutdown the workers as we want to reuse them
            # in latter calls but we free as much memory as we can by deleting
            # the shared memory
            #delete_folder(self._workers._temp_folder)
            self._workers = None

        self.reset_batch_stats()

in \sklearn\externals\joblib\_parallel_backends.py. This is a quick and dirty fix, but at least I get my results this way (the crash happening at the end, when you trained your model for 12h is very frustrating), and I delete the folder in \AppData\Local\Temp\ myself, which didn't pose any problem until now. Just writing this here if anyone is in the same situation, this is clearly not a good fix.

[pford221]

+1 for a permanent solution. I cannot convert pandas.DataFrame to numpy arrays because I'm using pandas.Series.map() in parallel for fast lookups.

I'm using:
pandas 0.24.0
jobllib 0.13.2
numpy 1.15.4
OS: Windows 10

[albertcthomas]

@pford221 @draimundo, @Krzysiaczek99 @tyokota @cliffrunner @robintwhite @zhanghongyong123456 @HansolSL please upvote (+1) the original comment of this issue so as to better track the number of people having this error and that would benefit from a solution

[pacorofe]

Hi, I am getting the same error. Using joblib version 0.13.2 in Windows 10 Pro.

[eternoaprendiz]

Hi, everyone,
Same error for me. It works with max_nbytes=None, but for some reason in for cycle it work in about 3 times faster.

python 3.7.4
pandas 0.25.1
jobllib 0.13.2
numpy 1.16.4
OS: Windows 10 Enterprise

P.S. Upvoted first comment

[johndahl01]

I did also meet this problem on a Windows 10 machine while running training sessions using Tensorflow in parallel processes. Based on the findings by albertcthomas in the previous post about the time dependency on allocation I made the following changes to the constants in disk.py:

RM_SUBDIRS_RETRY_TIME = 1 (was 0.1 before)
RM_SUBDIRS_N_RETRY = 240 (was 5 before)

I don't know what part solved the issue but using this made the scripts run until it was done in 4 days without any problems compared to crashes in 5-24 hour as before. I have no clue if it generalizes for other machines but it did work for me.

Anyhow, the five tries in within 0.5 s seems to be rather short in comparison to the results of albertcthomas example from above.

Edit: I use joblib v. 0.14.0

[izabella197]

Also experienced this error on Windows.

After experimenting with versions of scikit-learn, pandas, numpy & joblib , found that:

scikit-learn: 0.21.2
pandas: 0.25.1
numpy: 1.18.1
joblib: 0.14.1

do not cause the above error.
(using GridSearch with n_jobs=-1)

[GeorgeG92]

I had the exact same issue when running GridSearchCV for a sklearn-wrapped keras model in parallel.
For me the issue was resolved when I used 'threading' as joblib's backend, instead of the default which seems to be 'loky':

with joblib.parallel_backend('threading'): 
    grid_result = grid.fit(train_X, train_Y)

Windows 10 Home, sklearn=0.21.2, tensorflow=1.14.0, joblib=0.14.1

[pierreglaser]

@GeorgeG92 do not hesitate to test #966 - this PR should fix this problem without you having to change the backend of joblib (note that threading and loky backends have different performance characteristics).

[jorgelmadrid]

I have the same issue when running GridSearchCV for a sklearn model.

Windows 10 Pro, scikit-learn=0.22.2, joblib=0.14.1