Pyro: Add tutorial on enumeration

[fritzo]

As TraceEnum_ELBO gains functionality, we should describe how to use it.

Note that the gmm tutorial mentions enumeration, but I'd like a tutorial that focuses on the language aspects rather than a single model. We can point the the gmm tutorial and hmm example from the enumeration tutorial.

Previous discussions on forum.pyro.ai

• https://forum.pyro.ai/t/question-on-local-global-random-variable-in-pyro-hmm-example/444/9
• https://forum.pyro.ai/t/traceenum-elbo-parallel-enumeration-fails-with-error-about-cartesian-products/452/6

Topics

• [ ] restrictions on iarange dependency structure of TraceEnum_ELBO
• [ ] guide-side vs model-side enumeration
• [ ] restrictions on iarange nesting structure in model-side enumeration
• [ ] replacing a problematic iarange with an irange (see this comment)
• [ ] restrictions on model-guide dependencies
• [ ] how to estimate computational complexity
• [ ] using with cached_paths() for large graphs
• others?

[mbabadi]

Chiming in my experience -- I have learned most of what I know about Pyro by working through the realistic examples (e.g. VAE, DMM, AIR, etc.); those are _great_, and that's perhaps the best way for newcomers to dive in. That being said, I had to go through http://pyro.ai/examples/tensor_shapes.html multiple times in order to understand the logic of enumeration, nested iarange, etc.

It would be great to have a formal guide/tutorial on the language aspects, perhaps using abstract graphical models that showcase typical examples of dependency structures (rather than concrete models) to facilitate the discussion.

Anyway, I love where Pyro is headed! it is so liberating to be able to just think about models and worry less about logsumexps!