Pymc3: Sampling in parallel bugs under Windows OS

[HH-1]

Hi,
I'm trying to use the sample() method with the parameter njobs ( >1) but I have the following error :
TypeError: No context on context stack

As a model, I used the Linear regression given in the documentation. It works fine without the njobs argument.
I inserted all my code in a if __name__=='__main__' statement as usual on windows.

I'm running this code with the following configuration:

• windows 10,
• python 3.5 (anaconda),
• pymc3 3.1,
• theano 0.9.0

Does anyone face the same problem or is able to reproduce it?

Thanks a lot

EDIT :

This is my code
`

import numpy as np

# Initialize random number generator
np.random.seed(123)

# True parameter values
alpha, sigma = 1, 1
beta = [1, 2.5]

# Size of dataset
size = 100

# Predictor variable
X1 = np.random.randn(size)
X2 = np.random.randn(size) * 0.2

# Simulate outcome variable
Y = alpha + beta[0] * X1 + beta[1] * X2 + np.random.randn(size) * sigma

import pymc3 as pm

basic_model = pm.Model()

with basic_model:
    # Priors for unknown model parameters
    alpha = pm.Normal('alpha', mu=0, sd=10)
    beta = pm.Normal('beta', mu=0, sd=10, shape=2)
    sigma = pm.HalfNormal('sigma', sd=1)

    # Expected value of outcome
    mu = alpha + beta[0] * X1 + beta[1] * X2

    # Likelihood (sampling distribution) of observations
    Y_obs = pm.Normal('Y_obs', mu=mu, sd=sigma, observed=Y)

# draw 500 posterior samples
if __name__ == '__main__':
    trace = pm.sample(njobs=2, model=basic_model)

`

[ColCarroll]

A snippet helps! That error offer happens when you call sample outside
of the context manager (with pm.Model() as my_model:). You can also pass
the model object tosample, as in sample(..., model=my_model).

If this is the problem, it also should not work with any number of jobs, or
on any operating system!

On Wed, Jul 5, 2017, 11:11 AM HH-1 [e-mail] wrote:

Hi,
I'm trying to use the sample() method with the parameter njobs ( >1) but
I have the following error :
TypeError: No context on context stack

As a model, I used the Linear regression given in the documentation. It
works fine without the njobs argument.
I inserted all my code in a if __name__=='__main__' statement as usual on
windows.

I'm running this code with the following configuration:

• windows 10,
• python 3.5 (anaconda),
• pymc3 3.1,
• theano 0.9.0

Does anyone face the same problem or is able to reproduce it?

Thanks a lot

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pymc-devs/pymc3/issues/2383, or mute the thread
https://github.com/notifications/unsubscribe-auth/ACMHEP4HkyvtrWmV9zgvnOnwy96hXhcLks5sK1NDgaJpZM4OOD5G
.

[junpenglao]

@HH-1 You can change the last line:

if __name__ == '__main__':
    with basic_model:
            trace = pm.sample(njobs=2)

to make it work.

However, I see no reason why sample(..., model=my_model) doesnt work with njobs > 1. This is likely a bug.

[HH-1]

Thanks for your answers 👍

@junpenglao : Unfortunately, it still doesn't work with your changes.

[junpenglao]

That's odd. I will wait for somebody with a Windows system to test it.

[paddyhoran]

Yes, on windows with the same issue.

I am working through "Doing Bayesian Analysis", the data used below can be obtained from the website for the book.

Joblib will always fail on windows if you do not protect main, but the following fails.

`

import os
import pymc3 as pm

import pandas as pd

data_folder = os.sep.join(os.path.realpath(__file__).split(os.sep)[:-2] + ['data'])

data_location = os.path.join(data_folder, 'z15N50.csv')

y = pd.read_csv(data_location)['y']


if __name__ == '__main__':

    with pm.Model() as model:

        theta = pm.Beta(name='theta', alpha=1, beta=1)
        y_obs = pm.Bernoulli(name='y_obs', p=theta, observed=y)

        trace = pm.sample(njobs=2, model=model)

    print(pm.gelman_rubin(trace))

`

[ctm22396]

This seems relevant:
http://rhodesmill.org/brandon/2010/python-multiprocessing-linux-windows/

[ctm22396]

I think I've discovered how the code breaks down when spawning new processes.

The error is occurring when unpickling and reconstructing the step_method. There is already a "hack" in place in the object constructor to store the arguments for unpickling. However, the hack does not update the kwargs entry for 'model'.

in step_methods/arraystep.py:

class BlockedStep(object):

    generates_stats = False

    def __new__(cls, *args, **kwargs):
        blocked = kwargs.get('blocked')
        if blocked is None:
            # Try to look up default value from class
            blocked = getattr(cls, 'default_blocked', True)
            kwargs['blocked'] = blocked

        model = modelcontext(kwargs.get('model')) <------- gets the model from the arguments or context
        #### Note: 'model' is not a key in kwargs here, even upon initial execution in the main script
        #### Even if explicitly set as a kwarg in pm.sample

        # vars can either be first arg or a kwarg
        if 'vars' not in kwargs and len(args) >= 1:
            vars = args[0]
            args = args[1:]
        elif 'vars' in kwargs:
            vars = kwargs.pop('vars')
        else:  # Assume all model variables
            vars = model.vars

        # get the actual inputs from the vars
        vars = inputvars(vars)

        if len(vars) == 0:
            raise ValueError('No free random variables to sample.')

        if not blocked and len(vars) > 1:
            # In this case we create a separate sampler for each var
            # and append them to a CompoundStep
            steps = []
            for var in vars:
                step = super(BlockedStep, cls).__new__(cls)
                # If we don't return the instance we have to manually
                # call __init__
                step.__init__([var], *args, **kwargs)
                # Hack for creating the class correctly when unpickling.
                step.__newargs = ([var], ) + args, kwargs
                steps.append(step)

            return CompoundStep(steps)
        else:
            step = super(BlockedStep, cls).__new__(cls)
            # Hack for creating the class correctly when unpickling.
            step.__newargs = (vars, ) + args, kwargs <------- saves the args for unpickling
            ##### As kwargs does not have any model key in it, the model will not be saved as an
            ##### argument for unpickling. Thus, when constructing the new step method in
            ##### the spawned process, it will search for a model in the context, but find none
            ##### because the context is only set locally in the main execution.
            return step

I propose the quick and simple fix as follows:

        model = modelcontext(kwargs.get('model'))
        kwargs['model'] = model

or more in line with the function-based dictionary manipulation in the rest of the codebase:

        model = modelcontext(kwargs.get('model'))
        kwargs.update({'model': model})

Testing this on my own machine remedies this problem for me.

[junpenglao]

Thanks for the effort @ctm22396!! Could you please make a PR?

[ctm22396]

No problem! I created the pull request here: https://github.com/pymc-devs/pymc3/pull/2425

[junpenglao]

@HH-1 This should be fix now - could you please reinstall pymc3 from master and see if this resolve your issue?

[HH-1]

Hi,
I just tried reinstalling from master and it works !
Thanks everyone for your help!

[KirbyFears]

Hi,

I'm currently experiencing the same error in Windows for Python 2.7 and latest pip install of pymc3.

I ran the code pasted by HH-1 above to reproduce the error. Was the fix at #2425 only for Python 3, or is pip really far behind head/master?

Errors below. Each worker throws TypeError: No context on context stack.

Multiprocess sampling (2 chains in 2 jobs)
NUTS: [sigma_log__, beta, alpha]
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
C:\ProgramData\Anaconda2\lib\site-packages\h5py\__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
C:\ProgramData\Anaconda2\lib\site-packages\h5py\__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Process PoolWorker-1:
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda2\lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\ProgramData\Anaconda2\lib\multiprocessing\process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "C:\ProgramData\Anaconda2\lib\multiprocessing\pool.py", line 102, in worker
    task = get()
  File "C:\ProgramData\Anaconda2\lib\site-packages\joblib\pool.py", line 362, in get
    return recv()
  File "C:\ProgramData\Anaconda2\lib\site-packages\pymc3\step_methods\arraystep.py", line 39, in __new__
    model = modelcontext(kwargs.get('model'))
  File "C:\ProgramData\Anaconda2\lib\site-packages\pymc3\model.py", line 147, in modelcontext
    return Model.get_context()
  File "C:\ProgramData\Anaconda2\lib\site-packages\pymc3\model.py", line 139, in get_context
    raise TypeError("No context on context stack")
TypeError: No context on context stack

[moongraber]

Hi ,

I am running windows 10, python 3.6 and pymc3 v3.4.1

if I do:
with basic_model: # draw 500 posterior samples trace = pm.sample(500)

I get an error:

D:\Anaconda64bit\envs\python3\lib\site-packages\tqdm_tqdm.py in fp_write(s='\r 0%| | 0/1000 [00:00 186 """
187 fp = file
188 fp_flush = getattr(fp, 'flush', lambda: None) # pragma: no cover
189
190 def fp_write(s):
--> 191 fp.write(_unicode(s))
s = '\r 0%| | 0/1000 [00:00 192 fp_flush()
193
194 last_len = [0]
195

AttributeError: 'NoneType' object has no attribute 'write'

if I do

with basic_model: # draw 500 posterior samples trace = pm.sample(500,njobs=1)

works fine, seems pymc3 is not able to cope with multiple jobs using windows?

[junpenglao]

@moongraber could you please try updating tqdm?

[moongraber]

hi junepenglao

thanks for helping out!

I did on the cmd window
conda update tqdm

then python
import tqdm; tqdm.__version__

shows version '4.23.0'

but I get the same problem:
AttributeError: 'NoneType' object has no attribute 'write'

[BraulioSI]

Hi people. Can someone explain me how to "reinstall pymc3 from master"? I'm having the same issue in windows.
Thanks and greetings!!!

[fonnesbeck]

@BraulioSI pip install -U git+https://github.com/pymc-devs/pymc3.git

[BraulioSI]

@fonnesbeck Thank you very much!!!!!!

[BraulioSI]

Hi again. I've just executed that command line in the anaconda prompt, and pymc3 reinstalled well.
However, when I try to sample with njobs>1, I'm still getting the same issue: JoblibAttributeError: JoblibAttributeError

The most recent traceback is:

File "", line 2, in 
trace = pm.sample(500)

File "C:\Anaconda3\lib\site-packages\pymc3\sampling.py", line 444, in sample
trace = _mp_sample(**sample_args)

File "C:\Anaconda3\lib\site-packages\pymc3\sampling.py", line 984, in _mp_sample
traces = Parallel(n_jobs=cores, mmap_mode=None)(jobs)

File "C:\Anaconda3\lib\site-packages\joblib\parallel.py", line 789, in call
self.retrieve()

File "C:\Anaconda3\lib\site-packages\joblib\parallel.py", line 740, in retrieve
raise exception

JoblibAttributeError: JoblibAttributeError`

I'll appreciate any help. Thanks!!

[NikTheGeek1]

Same problem here... I am using spyder from anaconda. I've copy-pasted the code from the "getting_started" tutorial of pymc3 (http://docs.pymc.io/notebooks/getting_started) and when I reach the "with_basic_model:
trace = pm.sample(500)
I'm getting the same error (AttributeError: 'NoneType' object has no attribute 'write'). When I specify njobs = 1 it works. Can anyone help? Thank you.

EDIT:
When I am trying to run the code from cmd, rather than spyder, then it works fine. Python on cmd runs from "...\Python\Python36\py.exe" while I suppose that spyder's python is different? So I think the problems resides on anaconda's installation of python..
EDIT 2:
It also works fine on jupyter notebook.
In my case, only spyder has the problem. Can anyone replicate or is it just me?