Pymc3: Remove tracetab.py / trace_to_dataframe in release >=3.10.0

[michaelosthege]

With #3902 a DeprectationWarning is added to the trace_to_dataframe.

As this goes into release 3.9.0, the removal should be done with 3.10.0.

[joannadiong]

Hi @michaelosthege, I am updating some Notebooks for Rethinking_2 and am not familiar with Arviz and PyMC development.

In testing the Chp_04 code, I receive a warning that the trace_to_dataframe() function will soon be removed, and to use ArviZ to save traces. This will eventually break code e.g.

# Code 4.32
trace_df = pm.trace_to_dataframe(trace_4_1)

If I understand correctly, pymc3 3.10.0 now available on PyPI should have trace_to_dataframe() removed, but the current version of arviz 0.10.0 does not have a trace_to_dataframe() function. I'm not sure what the best approach is to stop the code above from breaking in future. For now, do you think we should leave the Notebook code as is and keep this Issue open? Or is there a better approach?

Many thanks.

[twiecki]

CC @AlexAndorra

[AlexAndorra]

Hi @joannadiong, and thanks for contributing to the Rethinking notebooks!
trace_to_dataframe is only available in PyMC3 (not in ArviZ) and will be removed in the next version, so the good solution is indeed to move away from it.

The good news is that we have a better solution ready for you to use: ArviZ's InferenceData format 🥳 It's much better because it can handle multidimensional data natively, with their associated names instead of their axis number. To get started, you can take a look at the introductory guide and the InferenceData cookbook.
I also strived to show case InferenceData capabilities in the _Rethinking 2_ notebooks, starting from chapter 11. The multilevel modeling tutorial on PyMC3's website is also full of such examples (thanks @OriolAbril !).
Finally, InferenceData is based on the xarray package, which is a great resource to get familiar with its capabilities.

Hope this helps, and feel free to open a WIP PR on the resources repo, where we'll be able to provide guidance.
Happy holidays and PyMCheers 🖖

[joannadiong]

Thanks @AlexAndorra for the helpful info! It will take me a while to digest, but should be a good learning opportunity.

Richard McElreath started teaching his course again and some of us in the class are working through the ported Python code, so it seemed a good opportunity to update the notebooks. We are all new to Bayes, so some aspects of the updates will be more manageable than others. In discussing with @aloctavodia, we thought a good approach to updating the notebooks is to open an Issue for each chapter with TODO list of things to fix, then an experienced person could do all changes at once. The learners could open smaller PRs (or work-in-progress PRs) if manageable. Of course, happy to take a different approach if it works better.

Hope this is cool. Many thanks for the support, and happy holidays too!

[twiecki]

@joannadiong This is great. I would like to encourage all students to attempt to fix those issues themselves, best way to learn!