Probability: Laplace approximation

[dionman]

Is there any function computing Laplace approximation (similar to ed.Laplace) supported at tensorflow probability at the moment?

[jeffpollock9]

I'm not aware of any Laplace approximation functionality but would definitely be keen to help with contributing it.

[brianwa84]

Building a cookie-cutter laplace_approx(target_density, location) shouldn't
be too hard.
Could probably do something like
tfd.MVNLinearOperator(LinearOperatorInvert(LinearOperstorFull(tf.hessians(target_density(xs),
xs))))
A couple thoughts:
tf.hessians d/n work in tf2, here's a workaround:
https://github.com/tensorflow/tensorflow/issues/29781#issuecomment-504980912
The hessian is not necessarily PD unless the location is in a local minimum.

Bigger challenge may be how are you going to find the location.

Brian Patton | Software Engineer | [email protected]

On Thu, Sep 26, 2019 at 1:51 PM Jeff [e-mail] wrote:

I'm not aware of any Laplace approximation functionality but would
definitely be keen to help with contributing it.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/probability/issues/570?email_source=notifications&email_token=AFJFSI7N622KMV63T42PSUDQLTZDJA5CNFSM4I2X5PMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7WNR5I#issuecomment-535615733,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFJFSI6DPVKUDU2G3E2EL7TQLTZDJANCNFSM4I2X5PMA
.

[junpenglao]

I would try using lbgfs to find the location - it should be even possible to find multi mode using tfp.optimizer.converged_all when the log_prob is batch-friendly.

[Padarn]

Would it be valuable to have laplace_approx(target_density, location) exposed, or should this look more like the Edward version that does the location finding.

From looking at the Edward implementation, there is a Base class Inference for which tfp doesn't seem to have an equivalent of - or am I missing something looking through the code?

[jeffpollock9]

If at all useful to anyone coming across this issue, I coded up some Laplace approximation stuff recently. Usage is in this comment, which also contains a gist of the implementation: https://github.com/tensorflow/probability/pull/1178#issuecomment-737183260

Would be happy to spend some time on this making a more formal proposal for adding something like this to TFP if that was of interest.