Pyro: Implement higher dimensional newton solvers

[jpchen]

currently there is a 2d solver, but we require a 3d solver and soon will require higher-dimensions:

• [x] #1210 newton_step() method that dispatches among {1d, 2d, 3d, nd}.
• [x] #1210 newton_step_1d() solver that computes min eig of Hessian
• [x] #1210 newton_step_3d() solver that computes min eig of Hessian
• [ ] newton_step_nd() solver using Powell's dog-leg method (see See section 3.5 of Madsen's Method for nonlinear least squares problems)

[kamathhrishi]

Is Powell's dog leg method the same as this https://en.m.wikipedia.org/wiki/Powell%27s_method ?

[kamathhrishi]

Could I have a relevant paper or source about Powell dogleg method ?

[fritzo]

@kamathhrishi I've added a reference above. Actually since #1670 we have a general batch inverse, and I think it may be possible to implement also an exact newton_step_nd().

[kamathhrishi]

@kamathhrishi I've added a reference above. Actually since #1670 we have a general batch inverse, and I think it may be possible to implement also an exact newton_step_nd().

Thank you @fritzo , incase I have queries do I post on forum and enquire or make a base PR and then incrementally work on it?

[fritzo]

Hi @kamathhrishi feel free to open up a draft PR for discussion.