Pymc3: Implement a CorrCholesky transformation for LKJCorr

[junpenglao]

Came across this issue in TFP: https://github.com/tensorflow/probability/issues/400 and found out that our transformation of LKJ is just a interval transformation - that's incorrect as it will produce invalid correlation matrix right?
https://github.com/pymc-devs/pymc3/blob/master/pymc3/distributions/multivariate.py#L1158-L1159

[aseyboldt]

Our LKJCorr will return a logp of -inf in some cases, yes. The proper transformation was hard to implement in theano, that's why I wrote LKJCholescyCov.
Unless we implement it however, we should probably deprecate pm.LKJCorr.

[junpenglao]

That makes sense, should we revisit the transformation for correlation matrix now since there is a version in TFP? It would need some theano.scan magic but it should be doable.

[junpenglao]

See also https://github.com/pyro-ppl/numpyro/blob/master/numpyro/distributions/transforms.py#L172

[junpenglao]

Proposal:

• implement a LKJCorrCholesky (or add a Cholesky parameterization to LKJCorr and make that the default)
• implement a CorrCholesky transformation
• clean up the deprecation/warning related to LKJCorr

Regarding the transformation, we can opt to use the simplier version in TFP. Some related discussion see:
https://github.com/tensorflow/probability/issues/694#issuecomment-599192616