Arviz: Plot KDEs horizontally?

[ljwolf]

Hey, not sure if there's an interest in this or if this package is actively maintained? I really like this idea of centralizing MCMC plotting tools.

When doing a traceplot, it can help to link the kernel density/histogram to the trace if the histogram/kdeplot closes over the trace. If seaborn is used for the kernel density, it's only a keyword argument to flip. I think, if this option would be useful in this package, then it should be possible to use ax.fill_betweenx.

There are a lot of other views I've built that might be more useful generally for MCMC that could take arbitrary sample chains.

[aloctavodia]

It seems that many persons like the idea of having a general MCMC plotting tool, but we never really push the idea, at least that's my case (_mea culpa_) I have made several changes to plots in PyMC3 but then I do not add them here :(
I think we have and egg-chicken situation, nobody (with a MCMC package) wants to relegate they plotting utilities to a third party (like this package) and then nobody wants to spend time working on this package...

Regarding seaborn, I used to think seaborn was a nice option, but now I think it is probably better to have custom tailored plots/functions, for example PyMC3 includes a function to compute KDEs. This function is way faster than the one in scipy/seaborn (and also includes corrections for boundaries). Is faster because the algorithm is different, but also is faster because is not a general solution, works only for 1D arrays and is not a good approach when you have few samples (less than ~200), but that is OK when using a MCMC trace. I think the same could apply to other situations, so now I think having seaborn as a dependency is not the best option.

I will check your other _views_ it seems to be interesting stuff there. Thanks for sharing!

[ariddell]

I really appreciate the comment. I wonder if the problem isn't simpler: matplotlib (and visualization generally) is a language unto itself and nobody currently maintaining a HMC/MCMC package speaks the language very well.

[ljwolf]

On the pooling of effort, I'd be interested in committing what I've done and helping where possible, whether it's here or in PyMC3. I don't use PyMC3 as much as I do pystan & my own Gibbs samplers, though. Regardless, I agree it's useful to pool these common views of mcmc results, and it's cool if that's in pymc3.plots, too, especially now that there's a release candidate. From my end, I know targeting a common datastructure, (like, pandas dataframe? flat numpy arrays?) is needed before defining a consistent plotting API, and I'm not yet clear on how to stuff arbitrary results into a PyMC3 trace.

On the diff to make this happen, seaborn isn't required. It's just about dispatch to the right ax.fill_between or ax.plot behavior.

[twiecki]

I encourage this effort. But from my perspective, PyMC3 should wait until PySTAN adopted this package, as it makes no sense for pymc3 to factor out its plotting code into an external one that then only we use. Of course, happy to have our recent plotting code to help kick-start this.

Perhaps this is also relevant for Edward CC @dustinvtran.

[aloctavodia]

I agree with @twiecki.

At this point mcmcplotlib is an outdated version of the PyMC3 plotting routines. I can move the new stuff here and also I could try to keep mcmcplotlib and PyMC3 plot in sync in the future. This could help to kick-start mcmcplotlib and help PyMC3 with a future (possible) transition to mcmcplotlib. What do you think?

[dustinvtran]

I think mcmcplotlib would have to be fairly feature-heavy to warrant adoption. It is bad that we're all reinventing the wheel, although at the same time, depending on an external library has costs. But I'm interested to see where it goes.

Also, plots for MCMC seems like a rather targeted subset of visualizations we might like to do in probabilistic modeling. Any particular reason for this?

[twiecki]

The other thing we keep reinventing are convergence diagnostics and model comparison metrics.

Also, plots for MCMC seems like a rather targeted subset of visualizations we might like to do in probabilistic modeling. Any particular reason for this?

How do you mean?

[ariddell]

@twiecki PyStan definitely should start using this package since it supports the one kind of plot that PyStan currently generates (traceplot). I made an issue stan-dev/pystan#315. Help would be appreciated.

[dustinvtran]

How do you mean?

I'm curious why the focus is on "MCMC plots".

[ariddell]

I thought it was a friendly and distinctive package name. If there's a
better name, I'd be happy to switch to that.

On 03/17/2017 07:53 PM, Dustin Tran wrote:

How do you mean?

I'm curious why the focus is on "MCMC plots".

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mcmcplotlib/mcmcplotlib/issues/8#issuecomment-287498570,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABN7vnTuy3Tf5yKwM4jrIEy3NmWcSz-ks5rmx0KgaJpZM4Me2RU.

[ahartikainen]

We need wikipage (?) to decide what needs to be implemented and behaviour with 1D to nD data and how data are shared.

Also common interface to matplotlib: mpl API, pyplot or pylab.

In my opinion dependencies should be minimal.

[twiecki]

@dustinvtran Since this wasn't adopted the name can easily be changed. The focus on MCMC is because Stan and PyMC3 are focused on MCMC and there are a certain set of plots people do over and over.

Maybe we should decide on what code we would like to share. We seem to agree on plotting, but some other things come to mind:

• Convergence diagnostics for MCMC
• Model comparison metrics (DIC, WAIC etc)
• Sampling backends to store traces (this could be quite useful for users as it makes interoperability between our packages trivial).

@dustinvtran Not sure if there are variational analyses that would make sense to share?

In any case, before doing any code changes here, we should update to pymc3's most recent plotting code.

[aloctavodia]

So we could be dealing with a general package for analysis of probabilistic programming results, right?

I have just updates to PyMC3's recent functions (including stats and diagnostics).

The functionality for computing highest posterior interval is partially duplicated (hpd.py and the hpd function inside stats.py), hpd.py is able to compute hpds for multimodal 1d distributions (I originally write it when porting DBDA), not sure if still useful.

Most of the functions are expecting a PyMC3 trace object. So to make it general we must ensure to convert other packages outputs to traces or we should make the functions work with some common data structure like a DataFrame. There is already a function to convert a PyMC3 trace to a DataFrame.

[aloctavodia]

The idea of using a Pandas DataFrame has been already discussed see #4, omit the part were I say we should also use seaborn :-P

[ahartikainen]

what is the convention to transform matrices to dataframe?

Are there any specific plotting methods that uses higher dimensions?

[dustinvtran]

@dustinvtran Not sure if there are variational analyses that would make sense to share?

@twiecki. More to the point, I guess my question is what the specific scope is for the package.

An ideal visualization toolkit for probabilistic programming would by definition span all of it. This might be impossible to achieve while being language-agnostic, or at least without making key design tradeoffs. MCMC plots are an excellent start, although it would nice to make explicit the package's scope and limitations. This helps determine design choices both at the infrastructure level and the API. For example:

• Should it include diagnostics beyond MCMC, such as point estimation, likelihood-free inferences, or variational inference?
• Should it include plots for individual distributions such as tables, density plots, and heatmaps?
• What about components of models such as marginals, conditionals, and multi-dimensional scaling?
• Should it include criticism techniques such as PPCs, feature maps, and decision boundaries?

If it's "only" MCMC plots, that's okay too.

[twiecki]

I agree on that being the key question. What do you think?

On Mar 19, 2017 11:46 AM, "Dustin Tran" [e-mail] wrote:

@dustinvtran https://github.com/dustinvtran Not sure if there are
variational analyses that would make sense to share?

@twiecki https://github.com/twiecki. More to the point, I guess my
question is what the specific scope is for the package.

An ideal visualization toolkit for probabilistic programming would by
definition span all of it. This might be impossible to achieve while being
language-agnostic, or at least without making key design tradeoffs. MCMC
plots are an excellent start. As the package progresses, it would nice to
make explicit the package's scope and limitations. This helps determine
design choices both at the infrastructure level and the API. For example:

• Should it include diagnostics beyond MCMC, such as point estimation,
  likelihood-free inferences, or variational inference?
• Should it include plots for individual distributions such as tables,
  density plots, and heatmaps?
• What about components of models such as marginals, conditionals, and
  multi-dimensional scaling?
• Should it include criticism techniques such as PPCs, feature maps,
  and decision boundaries?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mcmcplotlib/mcmcplotlib/issues/8#issuecomment-287608004,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AApJmAaHLWnD7koJCgZ8JRNo5_ajpX9oks5rnQdsgaJpZM4Me2RU
.

[cs224]

I am not sure where to add my comment, either here in this issue or in #4, but as the current activity seems to happen here I also put it here.

I am new to Python, PyMC3, pystan and pyjags. Actually my only reason to deal with Python is that I am interested in bayesian data analysis. I currently use all the three mentioned packages in order to better understand where the individual packages have their strength and weaknesses. One major annoyance in that regard is that all three of them use different plotting libraries or like pyjags none at all, e.g. I have to come up with my own solution.

This is how I ended up looking into mcmcplotlib. But as far as I can tell mcmcplotlib is bound to PyMC data structures, e.g. does not help with the other MCMC packages.

So far about the background. Now to my contribution to this conversation. I strongly would recommend to not bind this library to ANY implementation. Does not matter if it is pandas or whatever. I personally start using xarray more and more. I would suggest to define a hierarchy of minimal and composable interfaces based on which this library generates the visualizations. Then the only thing one needs to do to make this functionality available for a given MCMC package is to write an adapter between the native data structure used in the MCMC package and the visualization library. This keeps your path free and open in all directions.

Then even I, not related to any of the MCMC packages, could contribute an adapter to the visualization/plotting library.

[ahartikainen]

Comment on the package name: If you google mcmcplotlib, google will change it to matplotlib.

[aloctavodia]

Closing as this have been addressed. If you think sometimes still needs attention, I would suggest to open a a new issue.