Arviz: NoneType error with standard pymc3 workflow

[yannikschaelte]

Describe the bug

In a standard pymc3 sampling workflow (see below, simplified from here), an error occurs. This happens only with arviz 0.8.0, works with 0.7.0.

I was not sure whether this is pymc3 or arviz related.

To Reproduce

import numpy as np
import pymc3 as pm
import pandas as pd

drug = (101,100,102,104,102,97,105,105,98,101,100,123,105,103,100,95,102,106,
        109,102,82,102,100,102,102,101,102,102,103,103,97,97,103,101,97,104,
        96,103,124,101,101,100,101,101,104,100,101)
placebo = (99,101,100,101,102,100,97,101,104,101,102,102,100,105,88,101,100,
           104,100,100,100,101,102,103,97,101,101,100,101,99,101,100,100,
           101,100,99,101,100,102,99,100,99)

y1 = np.array(drug)
y2 = np.array(placebo)
y = pd.DataFrame(dict(value=np.r_[y1, y2], group=np.r_[['drug']*len(drug), ['placebo']*len(placebo)]))

μ_m = y.value.mean()
μ_s = y.value.std() * 2

with pm.Model() as model:
    group1_mean = pm.Normal('group1_mean', mu=μ_m, sd=μ_s)

with model:
    trace = pm.sample(200, chains=2)

gives:

Only 200 samples in chain.
Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (2 chains in 2 jobs)
NUTS: [group1_mean]
Sampling 2 chains, 0 divergences: 100%|██████████| 1400/1400 [00:00<00:00, 2235.17draws/s]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-f80168e6f648> in <module>
      1 with model:
----> 2     trace = pm.sample(200, chains=2)

~/anaconda3/lib/python3.7/site-packages/pymc3/sampling.py in sample(draws, step, init, n_init, start, trace, chain_idx, chains, cores, tune, progressbar, model, random_seed, discard_tuned_samples, compute_convergence_checks, **kwargs)
    496             warnings.warn("The number of samples is too small to check convergence reliably.")
    497         else:
--> 498             trace.report._run_convergence_checks(trace, model)
    499 
    500     trace.report._log_summary()

~/anaconda3/lib/python3.7/site-packages/pymc3/backends/report.py in _run_convergence_checks(self, trace, model)
     82                 varnames.append(rv_name)
     83 
---> 84         self._ess = ess = ess(trace, var_names=varnames)
     85         self._rhat = rhat = rhat(trace, var_names=varnames)
     86 

~/anaconda3/lib/python3.7/site-packages/pymc3/stats/__init__.py in wrapped(*args, **kwargs)
     22                 )
     23                 kwargs[new] = kwargs.pop(old)
---> 24             return func(*args, **kwargs)
     25 
     26     return wrapped

~/anaconda3/lib/python3.7/site-packages/arviz/stats/diagnostics.py in ess(data, var_names, method, relative, prob)
    189             raise TypeError(msg)
    190 
--> 191     dataset = convert_to_dataset(data, group="posterior")
    192     var_names = _var_names(var_names, dataset)
    193 

~/anaconda3/lib/python3.7/site-packages/arviz/data/converters.py in convert_to_dataset(obj, group, coords, dims)
    175     xarray.Dataset
    176     """
--> 177     inference_data = convert_to_inference_data(obj, group=group, coords=coords, dims=dims)
    178     dataset = getattr(inference_data, group, None)
    179     if dataset is None:

~/anaconda3/lib/python3.7/site-packages/arviz/data/converters.py in convert_to_inference_data(obj, group, coords, dims, **kwargs)
     89             return from_pystan(**kwargs)
     90     elif obj.__class__.__name__ == "MultiTrace":  # ugly, but doesn't make PyMC3 a requirement
---> 91         return from_pymc3(trace=kwargs.pop(group), **kwargs)
     92     elif obj.__class__.__name__ == "EnsembleSampler":  # ugly, but doesn't make emcee a requirement
     93         return from_emcee(sampler=kwargs.pop(group), **kwargs)

~/anaconda3/lib/python3.7/site-packages/arviz/data/io_pymc3.py in from_pymc3(trace, prior, posterior_predictive, log_likelihood, coords, dims, model, save_warmup)
    483         dims=dims,
    484         model=model,
--> 485         save_warmup=save_warmup,
    486     ).to_inference_data()
    487 

~/anaconda3/lib/python3.7/site-packages/arviz/data/io_pymc3.py in to_inference_data(self)
    413         """
    414         id_dict = {
--> 415             "posterior": self.posterior_to_xarray(),
    416             "sample_stats": self.sample_stats_to_xarray(),
    417             "log_likelihood": self.log_likelihood_to_xarray(),

~/anaconda3/lib/python3.7/site-packages/arviz/data/base.py in wrapped(cls, *args, **kwargs)
     35                 if all([getattr(cls, prop_i) is None for prop_i in prop]):
     36                     return None
---> 37             return func(cls, *args, **kwargs)
     38 
     39         return wrapped

~/anaconda3/lib/python3.7/site-packages/arviz/data/io_pymc3.py in posterior_to_xarray(self)
    207                 )
    208             data[var_name] = np.array(
--> 209                 self.trace[-self.ndraws :].get_values(var_name, combine=False, squeeze=False)
    210             )
    211         return (

TypeError: bad operand type for unary -: 'NoneType'

The error does not occur with chains=1.

Expected behavior

No error :)

Additional context

Arviz 0.8.0 (0.7.0 worked), pymc3 3.8, all installed via pypi on anaconda3 python 3.7, 3.8.

[OriolAbril]

Yeah, sorry about that, we have been working to better integrate ArviZ with PyMC3, the end of sampling report has been delegated to ArviZ for a while but there were some issues when storing warmup and with memory usage.

There have been some changes in from_pymc which now require the development version of PyMC3 or version 3.9 which will be coming very soon (and will at the same time require ArviZ>=0.8.0). Until pymc3 3.9 is release, to use ArviZ 0.8.0 with pymc3 you'll need to install pymc3 with:

pip install git+https://github.com/pymc-devs/pymc3

Side note: It works for one chain because with one chain no convergence report can be generated and therefore ArviZ is not called, working with 1 chain is not really a useful workaround

[yannikschaelte]

Thanks for the fast response! That clarifies it. No, one chain is not useful, that was just to pinpoint the error.

[OriolAbril]

My memory failed me above, we initially planned on requiring 3.9 but eventually decided on supporting old but recent versions too and there was a bug. I have had a flash and double checked the code :sweat_smile:

I have also added a warning triggered only if save_warmup=True.