Probability: Any plans to update the examples to TensorFlow 2?

[nbro]

The examples in https://github.com/tensorflow/probability/tree/master/tensorflow_probability/examples do not run with TensorFlow 2, but it's been a while the stable TF2 is available. So, any plans to update the examples to TensorFlow 2 (eager execution)? I am particularly interested in https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/bayesian_neural_network.py.

[shashankg7]

Even I am interested in TF2.0 examples.

[Pyrsos]

I am interested in giving this a go. Can I be assigned to work on it?

[nbro]

@Pyrsos Well, if you have a go at it, share it with us, please! However, note that there are a few bugs, which may prevent you from updating certain examples to TF2. See https://github.com/tensorflow/probability/issues/620.

[patelmiteshn]

Is there any timeline on this or has anyone played with any examples ?

[brianwa84]

We have taken a pass over the examples to assess what work might need to be done (which might be a bit out of date, as @davmre has already made some updates). If anyone wants to take some of these on, please feel free to send PRs, esp for the easy-ish ones!

File | Purpose | Showcases | What needs work | Proposal
-- | -- | -- | -- | --
tensorflow_probability/examples/bayesian_neural_network.py | Trains a Bayesian neural network to classify MNIST digits. | tfp.layers.Convolution2DFlipouttfp.layers.DenseFlipoutinline in a keras modelCategorical output (but not tfpl.DistributionLambda) | TF2 (placeholder, session)tf.compat.v1.metrics.accuracytf.compat.v1.data.make_one_shot_iterator |  
tensorflow_probability/examples/cifar10_bnn.py | Trains a Bayesian neural network to classify CIFAR-10 images. | tfd.Categorical(+bayesian_{vgg,resnet}) | TF2 (placeholder, session)tf.compat.v1.metrics.accuracymigrate to colab?uses tensorflow_probability.examples.models.* | Easy to update
tensorflow_probability/examples/deep_exponential_family.py | Trains a sparse Gamma deep exponential family on NIPS 2011 conference papers. | tfp.edward2 (could be JDCoroutine) | tf.compat.v1 variables, summary, global stepuses edward2Looks like great opp to showcase JDCoroutineUses NIPS2011 papers dataset (migrate to TFDS?) | Delete or rewrite
tensorflow_probability/examples/disentangled_vae.py | Disentangled Sequential Variational Autoencoder. | Uses a handful of distributions- Independent(Normal, 3) images- MVNDiag | Written in eager, so easier TF2 migrationuses sprites_datasettf.compat.v1.train.cosine_decayuses tf.contrib.checkpointLearnableMultivariateNormalDiag can probably use TF2 var safety work | Delete or rewrite
tensorflow_probability/examples/disentangled_vae_test.py | test for dvae |   | tf.compat.v1.global_variables_initializer() |  
tensorflow_probability/examples/generative_adversarial_network.py | Trains a GAN to generate synthetic images of MNIST digits. | n/a (Bernoulli!) | Connection to TFP is tenuous (tfd.Bernoulli), an extremely basic GAN | Delete
tensorflow_probability/examples/grammar_vae.py | Trains a grammar variational auto-encoder on synthetic data. | Works on molecular representations (SMILES strings)Ed as universal PPL (no batching), uses tracer for branchy model | Uses edward2, ed.tapetf.compat.v1.nn.rnn_cell.LSTMCell (would need to -> keras RNN) | Delete (or figure out how one would write with TFP!)
tensorflow_probability/examples/latent_dirichlet_allocation_distributions.py | Trains a Latent Dirichlet Allocation (LDA) model on 20 Newsgroups. | Dirichlet, OneHotCategorical, kl_divergence | uses tf.compat.v1:.summary.get_or_create_global_step.metrics.mean.py_func.data.make_one_shot_iteratorEstimatorDoesn't use JD (merge w/ lda_ed2) | Delete or rewrite
tensorflow_probability/examples/latent_dirichlet_allocation_edward2.py | Trains a Latent Dirichlet Allocation (LDA) model on 20 Newsgroups. | Easily rewritten w/ JDCoro (merge w/ lda_dists) | Uses edward2 (12 lines shorter than the above) | Delete
tensorflow_probability/examples/logistic_regression.py | Trains a Bayesian logistic regression model on synthetic data. | tfp.layers.DenseFlipouttfd.Bernoulli | - Uses toy data (could we use covertype?)- tfp.layers.DenseFlipout- make_one_shot_iterator- plotting in shell script, why not colab?- small, easy upgrade | Update
tensorflow_probability/examples/models/bayesian_resnet.py | Constructs a ResNet18 model. | tfp.layers.Convolution2DFlipouttfp.layers.DenseFlipout(inline in deep keras model) | Uses tfp.layers.DenseFlipout (needs audit for TF2) | Easy to keep
tensorflow_probability/examples/models/bayesian_vgg.py | Constructs a VGG16 model. | tfp.layers.Convolution2DFlipouttfp.layers.DenseFlipout(inline in deep keras model) | Uses tfp.layers.DenseFlipout (needs audit for TF2) | Easy to keep
tensorflow_probability/examples/README.md | This directory contains standalone usage examples of TensorFlow Probability API. |   | Update to reflect decisions | Update readme
tensorflow_probability/examples/sprites_dataset.py | Liberated Pixel Cup [(LPC)][1] Sprites Dataset. | n/a | Uses tf.contrib.lookup.index_table_from_tensor | Move to tfds, or delete
tensorflow_probability/examples/vae.py | Trains a variational auto-encoder (VAE) on binarized MNIST. | MultivariateNormalDiagIndep(Bern)MixSameFam(MVNDiag)kl_divergence | - upgrade to tfds binarized_mnist- tf.compat.v1: - py_func - make_one_shot_iterator - summary - metrics - train.cosine_decay- Estimator- MoG prior, MVNDiag latents | Delete or rewrite, I think we would do this differently today
tensorflow_probability/examples/vq_vae.py | Trains a vector quantized-variational autoencoder (VQ-VAE) on MNIST. | Indep(Bern)Multinomial | Uses moving_averages direct TF import.Three variants of MNIST - are they all in TFDS?tf.compat.v1.data.Iterator.from_string_handlepy_func (for dataset parsing)tf.compat.v1.summaryA lot of plotting --> colab?TF2 (Graph/Session) | Rewrite, VQVAE is (imo) an important example to have
tensorflow_probability/examples/jupyter_notebooks/A_Tour_of_TensorFlow_Probability.ipynb | On the label | Tons of stuff (dists, bijectors, psd kernels, gp regression, hmc, step size adapt, etc) | tfp.positive_semidefinite_kernelsmay need to update for mcmc changes when those landmaybe drop the tfp-nightly picker and the tf2/tf1 picker? | Update
tensorflow_probability/examples/jupyter_notebooks/Bayesian_Gaussian_Mixture_Model.ipynb | On the label | dists, bijectors, hmc, ttk | MVNCholPrecisionTriL should move to tfp proper?Needs TF2 updatesuggest update to JointDist as wellupdate unconstraining_bijectors naming? | Update
tensorflow_probability/examples/jupyter_notebooks/Bayesian_Switchpoint_Analysis.ipynb | On the label (mining disasters dataset) | ed2 | Update to tfp101 changepoint model?ed2 -> JD?Needs TF2 updateuses legacy step size adaptuse log_rate for poisson? | Delete or update (update not too hard)
tensorflow_probability/examples/jupyter_notebooks/Credit_Card_Interest.ipynb | Some guessing about whether interest rates go up, simulating possible futures | tfd.Binomial | Too trivial | Delete
tensorflow_probability/examples/jupyter_notebooks/Eight_Schools.ipynb | Classic hierarchical | ed2 | Needs TF2 update, ed2->JD update | updated in cl/278789359
tensorflow_probability/examples/jupyter_notebooks/Factorial_Mixture.ipynb | Nifty little demo/trick | batching, MixSameFam | Good demo case for tape safety, could use quick TF2 audit, already eager. | Update
tensorflow_probability/examples/jupyter_notebooks/Fitting_DPMM_Using_pSGLD.ipynb | Showcases some synergies of TF+TFP | SGLD optimizer, transformations of tf.Variable | Needs the colab link badgeNeeds TF2 updateNice showcase for tfp.util.TransformedVariable, tape-safety | Update
tensorflow_probability/examples/jupyter_notebooks/Gaussian_Copula.ipynb | Nifty demo/trick | bijectors, transformed dist | Concat bijector can become tfb.Blockwise?Needs TF2 updateShowcase other CDF bijectors? Off-hand I know we have Weibull. | Update
tensorflow_probability/examples/jupyter_notebooks/Gaussian_Process_Latent_Variable_Model.ipynb | Demo the GP library w/ MNIST | GP, GPRM | tfp.positive_semidefinite_kernelsNeeds TF2 update, doesn't look very hard | Update
tensorflow_probability/examples/jupyter_notebooks/Gaussian_Process_Regression_In_TFP.ipynb | Kind of classic GP demo | GP, GPRM | tfp.positive_semidefinite_kernelsNeeds TF2 update, doesn't look very hardUses AdamOptimizer, so loss needs to become a function, probably tf.function | updated in cl/279842677
tensorflow_probability/examples/jupyter_notebooks/Generalized_Linear_Models.ipynb | Explain GLMs, demo the tfp.glm library | GLM | Needs TF2 update, ed2->JD updateI don't expect a user to be able to execute the R section (suggest rpy2, https://colab.research.google.com/drive/1h5-lqnPz3WRXLXIt41N5B_aDngIIlfyL prototype, but note the r install took FOREVER for some reason)Uses tf.gradients at the bottom, consider removing that section unless we really value the didactic nature. | Update
tensorflow_probability/examples/jupyter_notebooks/HLM_TFP_R_Stan.ipynb | Comparing TFP (SA-EM), R, Stan for the radon dataset | Hierarchical model | Similar to GLMs.ipynb re: rpy2Needs TF2 migrationShould probably switch to JDAppears to autodiff through HMC? Is this still supported? | Update or delete
tensorflow_probability/examples/jupyter_notebooks/Linear_Mixed_Effects_Model_Variational_Inference.ipynb | VI on radon dataset | dists, autodiff, trainable surrogate posterior, tfp.vi.monte_carlo_csiszar_f_divergence, tfp.vi.kl_reverse | tf.make_templateuse JDTF2 migration | Update or delete
tensorflow_probability/examples/jupyter_notebooks/Linear_Mixed_Effects_Models.ipynb | InstEval dataset from lme4 | dists, autodiff, expectation maximization | Needs TF2 migration | Updated in cl/282628627
tensorflow_probability/examples/jupyter_notebooks/Modeling_with_JointDistribution.ipynb | JDSeq | JD, HMC, batching, L-BFGS, VI, multiple datasets, MF VI, TriL VI | Includes GLMM on radon, can we delete the LME notebooks?tfd.fill_triangular TODO(junpenglao): update to use NUTS, also clean up VI part. | updated in cl/279789837
tensorflow_probability/examples/jupyter_notebooks/Multilevel_Modeling_Primer.ipynb | Similar to HLM, LMEMVI, LMEM, GLM - nice graphics and renderings of the graphical models | JD, deeper dive, and newer than prev GLMM notebooks | Already TF2! | Keep
tensorflow_probability/examples/jupyter_notebooks/Multiple_changepoint_detection_and_Bayesian_model_selection.ipynb | Multiple changepoint inhomogenous poisson process, using HMM | tfd.HMM, MAP estimation, autodiff | TF2 migrationTODO(davmre): once tfp.HiddenMarkovModel implements the Viterbi algorithm to find highest-probability trajectories, update this section to use it. | Update
tensorflow_probability/examples/jupyter_notebooks/Optimizers_in_TensorFlow_Probability.ipynb |   |   |   |  
tensorflow_probability/examples/jupyter_notebooks/Probabilistic_Layers_Regression.ipynb |   |   |   |  
tensorflow_probability/examples/jupyter_notebooks/Probabilistic_Layers_VAE.ipynb |   |   |   |  
tensorflow_probability/examples/jupyter_notebooks/Probabilistic_PCA.ipynb |   |   |   | updated in cl/278789359
tensorflow_probability/examples/jupyter_notebooks/Structural_Time_Series_Modeling_Case_Studies_Atmospheric_CO2_and_Electricity_Demand.ipynb | bsts, vi interface | tfp.sts, tfp.vi | Already TF2! | Keep
tensorflow_probability/examples/jupyter_notebooks/TensorFlow_Distributions_Tutorial.ipynb |   |   |   |  
tensorflow_probability/examples/jupyter_notebooks/TensorFlow_Probability_Case_Study_Covariance_Estimation.ipynb |   |   |   |  
tensorflow_probability/examples/jupyter_notebooks/Understanding_TensorFlow_Distributions_Shapes.ipynb |   |   |   |  

[mjul]

In addition to the example notebooks _etc._ mentioned above, the doc comments also have v1 code. For instance in tfp.distributions.Mixture where the main doc comment has an example using .eval() which is not needed in TF2 where eager execution is default.