Gpytorch: [Feature Request] Sparse Spectrum GP Implementation

[acxz]

🚀 Feature Request

Hey guys, I'm back, hope everyone has been taking care and staying safe.

I've doing some research in Sparse GPs and thought it would be great if GPytorch can provide a Sparse Spectrum Gaussian Process (SSGP) implementation.

Associated Paper here

Motivation

Is your feature request related to a problem? Please describe.

Just would be nice GPytorch can support more sparsity related GPs.

Pitch

Describe the solution you'd like

An implementation of SSGP similar to the support level of the VFE SGPR provided implementation.

Describe alternatives you've considered

N/A

Are you willing to open a pull request? (We LOVE contributions!!!)

I would but won't be able to get around to it for a while.

Additional context

N/A

[gpleiss]

This idea seems similar to RFFs (which would also be a good algorithm to have in the library for benchmarking), and would play nicely with our existing CG framework.

@jacobrgardner and I probably don't have time to get around to it, but we would appreciate a PR! I imagine the implementation would look similar to our spectral mixture implementation, but with some sampling added, and using some LazyTensors for computational efficiency.

[acxz]

@gpleiss thx for responding! I don’t have really have any experience with GPytorch’s internals but thanks for pointing a direction forward with this.

Would it be alright if I pester you guys with any questions when I try to implement this feature?

[gpleiss]

Absolutely! We might be a bit slow to respond for the next week and a half, but ask questions in this issue or in a PR.

[KeAWang]

There's a rff-kernel branch that I implemented for a course project last year that can be used. I could turn that into a PR when I get a chance