My current understanding is that SVD/TSVD may be available in cuSparse. @jeaton32 is exploring adding more sparse functionality. @davidsebfischer is interested in sparse GPU functionality as a user -- most of the data he works with is sparse and PCA is critical for operational is the single cell analysis space.
@davidsebfischer can speak to the flavor of SVD you required ? Would TSVD work ?
quasiben
All 4 comments
Thanks for the issue, @quasiben! TSVD is great, it is the current default for sparse matrices in our workflows.
davidsebfischer
on 2 Aug 2019
@danielhanchen has been working on a Sparse RSVD primitive in cuML using cuSparse and cuSolver [1]
If we find that it brings a great benefit to performance, we'd love to support getting this functionality into cupy.
cjnolet
on 19 Nov 2019
@venkywonka has built an LOBPCG solver in cupy for cuML. Specifically, we would like to use this to implement sparse TSVD. The LinearOperator API has also been implemented in his PR, which will allow us to implement PCA w/ the implicit mean centering trick (see related Scikit-learn issue).
If @venkywonka's Cupy-based LOBPCG implementation were to live in Cupy instead of cuML, we would be able to support the option solver='lobpcg' in cupyx.scipy.sparse.linalg.svds and just precondition with a LinearOperator in PCA to support sparse inputs by mean centering right in the solver (see implementation by Facebook).
cjnolet
on 21 Oct 2020
@venkywonka has built an LOBPCG solver in cupy for cuML. Specifically, we would like to use this to implement sparse TSVD. The
LinearOperatorAPI has also been implemented in his PR, which will allow us to implement PCA w/ the implicit mean centering trick (see related Scikit-learn issue).If @venkywonka's Cupy-based LOBPCG implementation were to live in Cupy instead of cuML, we would be able to support the option
solver='lobpcg'incupyx.scipy.sparse.linalg.svdsand just precondition with aLinearOperatorin PCA to support sparse inputs by mean centering right in the solver (see implementation by Facebook).
hey @cjnolet , although I have implemented LOBPCG solver in cupy in cuML, I have not implemented the LinearOperator feature that scipy contains. The corresponding places, instead, use simple cupy matrix multiplication. Just wanted to clarify. That said, I'm all in for redirecting the contributions to cupy repo 馃憤
venkywonka
on 22 Oct 2020
Related issues
asi1024
路
3Comments
Bartzi
路
4Comments
kmaehashi
路
3Comments
ma-sadeghi
路
3Comments
rafael-fuente
路
3Comments
Most helpful comment
@venkywonka has built an LOBPCG solver in cupy for cuML. Specifically, we would like to use this to implement sparse TSVD. The
LinearOperatorAPI has also been implemented in his PR, which will allow us to implement PCA w/ the implicit mean centering trick (see related Scikit-learn issue).If @venkywonka's Cupy-based LOBPCG implementation were to live in Cupy instead of cuML, we would be able to support the option
solver='lobpcg'incupyx.scipy.sparse.linalg.svdsand just precondition with aLinearOperatorin PCA to support sparse inputs by mean centering right in the solver (see implementation by Facebook).