Pymc3: Unconverged chain in dp mixture example

[twiecki]

https://docs.pymc.io/notebooks/dp_mix.html

Probably just rerunning with more tuning steps will fix that.

[junpenglao]

I think there are some problem with the model (mixture model is hard), see attempt here https://github.com/pymc-devs/pymc3/pull/2956

[Drwangchao]

I just met a similar problem. I have post the details at https://discourse.pymc.io/t/questions-about-the-convergence-of-sampling-the-mixture-of-hierarchical-model/2169.

I wonder how to solve this problem. I expect reply.
@twiecki
@junpenglao

[sameshl]

I am new here and would like to start contributing by taking up this issue.Can somebody link me up with relevant info so that I can get started.

[twiecki]

@sameshl The relevant info should be found in the links above. Try running the NB and doubling the tuning steps and setting target_accept=0.9.

[Harivallabha]

Hi, could I get started on this issue? I'm new to pyMC3 and would like to work on the GSoC, 2019 project on Dirichlet processes :)

[junpenglao]

@Harivallabha sure!

[Harivallabha]

Hi, sorry for the delay. Had mid-semester examinations for the past two weeks :)

I tried increasing the # of tuning steps, and setting target_accept to 0.9. It still diverges. I've included a couple of snapshots of the results. Here's a link to my notebook where I tried out a few other configurations as well: https://github.com/Harivallabha/DP_pyMC3-Docs_Issue/blob/master/Unconverged_DP_Chain.ipynb

I'll now try reparameterizing it, and use advi+adapt_diag for initializing the sampler. Hopefully a proper reparameterization should do the trick.

[canyon289]

@Harivallabha I mean this in the kindest way, posting screenshots of a notebook is not a great way to share code. I would suggest a minimal example, or even better a github gist or repo where was can inspect the results and download the code

[Harivallabha]

@canyon289 That makes sense, thank you. Edited it :)

[Harivallabha]

Hi, I tried a few other things but seem to be unable to fix the divergences. Here's a link to my notebook outlining my progress: https://github.com/Harivallabha/harivallabha.github.io
I've also included in it, a compilation of resources and notes that I found useful while trying to understand and fix this issue (for the benefit of any newcomer looking to gain some insight about the topic).
Any suggestions would be deeply appreciated :) Thanks!

[twiecki]

Thanks for your work on this! Where exactly is the funnel occurring, do you think it can be reparameterized? I'm not sure ADVI is more than just masking the problem.