Scanpy: integrate with CCA and pyscenic

[wangjiawen2013]

Dear,

Is it possible to integrate scanpy with CCA and pyscenic?
CCA (canonical correlation analysis to alignment different datasets and batch effect correction):
https://satijalab.org/seurat/immune_alignment.html

pyscenic (single-cell regulatory network inference and clustering):
https://github.com/aertslab/pySCENIC

[fidelram]

CCA does not have code in python, which will make it difficult to integrate, pySCENIC is probably easier but I would rather ask the developers.

@falexwolf We should consider a way to facilitate scanpy 'plugins'. A quick search shows me that this could be possible: https://packaging.python.org/guides/creating-and-discovering-plugins/ but honestly I don't know how it works. Nevertheless, given the number of tools that continue to appear we should consider a scheme that facilitate how developers can take advantage of scanpy preprocessing, storing, analysis and visualization tools.

[falexwolf]

I like Seurat's CCA. A pull request using rpy2 similar to the R wrapper of Haghverdi et al.'s version of MNN would be welcome.

Regarding "plugins": I guess a lot of Scanpy's functionality already consists in "plugins":

• https://scanpy.readthedocs.io/en/latest/api/scanpy.api.pp.mnn_correct.html
• https://scanpy.readthedocs.io/en/latest/api/scanpy.api.pp.dca.html
• https://scanpy.readthedocs.io/en/latest/api/scanpy.api.pp.magic.html
• https://scanpy.readthedocs.io/en/latest/api/scanpy.api.tl.phate.html
• https://scanpy.readthedocs.io/en/latest/api/scanpy.api.tl.louvain.html

and a lot more are on the way, as far as I know. I guess the strategy of having an optional dependency of the respective and a small wrapper in Scanpy is a scalable strategy. Do you think we need to do more?

[dawe]

Sklearn has its implementation of CCA but that would allow the alignment of two samples only. Recently a multi sample approach was implemented in pyrcca library for which there is a biorXiv paper.

[fidelram]

@falexwolf I think it is worth to have a scanpy.plugin or scanpy.extension or something shorter like scanpy.pg or scanpy.ext that aggregates all plugins. First, this clarifies for the user that the tool he/she is using is not directly developed by scanpy. Second, this allows plugins to be installed separately without having to update scanpy's code. The idea is that scanpy will be able to discover any plugins installed. On the side of developers, this could facilitate integration with scanpy. We can get inspired by flask extensions: http://flask.pocoo.org/docs/1.0/extensiondev/

[wangjiawen2013]

Sklearn has its implementation of CCA but that would allow the alignment of two samples only. Recently a multi sample approach was implemented in pyrcca library for which there is a biorXiv paper.

Is it suitable for single cell data ?

[falexwolf]

@fidelram Yes, makes sense. Let's see whether we manage to organize it this way. There will be a few plugins coming soon and I'll talk with the one doing it about this.

@wangjiawen2013 The Seurat developers did a bit more than simply fitting a standard CCA. So I'd assume that it'd be some work to wrap sklearn's CCA or pyrcca so that it performs similar to Seurat's CCA on single cell data...

[dawe]

Given that UMAP can be used for manifold learning, shouldn’t be possibile to align experiments using UMAP? Who wants to join me in this evaluation?

[falexwolf]

UMAP won't do any correction of batch effects for you, like CCA (it looks at the basis that leads to the greatest overlap between the batches, assuming that this captures the common biological variation and projects out everything else, assuming it's nuisance/technical batch effects). Similar for all other "alignment tools": you throw away some information in order to align. When you map a new dataset into an existing dataset using UMAP, this will do an _exact_ mapping. If you have pronounced batch effects, the second dataset will cluster as a whole far away from the first.

So, I don't think that there will be much to gain.

Why not give BBKNN (https://github.com/Teichlab/bbknn) a try? It integrates nicely with Scanpy.

[dawe]

@falexwolf you are right, although I have to say that in my (surely limited) experience, batch effect doesn't seem to play a major role when using UMAP. I don't know BBKNN, I will give it a look for sure. I guess this evaluation may be a side project (for volunteers) which may eventually merge into scanpy.

[falexwolf]

Cool!

You're right, if you don't have strong batch effects across your samples, you don't need any batch correction like CCA. A a simple UMAP of all the samples gives you a reasonable picture of what happens.

[flying-sheep]

Let’s continue the discussion about a general plugin mechanism in #271, and this thread for CCA specifically.

[MichaelPeibo]

Just curious about the case of 'batch effect', it looks like to me library construction protocol/chemicals is main source of batch effects.

However, if I use same protocols, sequencing platform and slight difference of sequencing depth for some sample, but in different time course, would you call it batch effect?

A more specific case is, if I have time-course data1 which has not geneX, however, I time-course data2 will have geneX till days later. In mnn correction, a prerequisite is same genes, will it filter out some genes meaningful?

[wangjiawen2013]

In different time course, the batch effect and true biological variation will be entangled.

Batch effects, which occur because measurements are affected by laboratory conditions,reagent lots and personnel differences. This becomes a major problem when batch effects are correlated with an outcome of interest and lead to incorrect conclusions. However, in single cell RNAseq, different datasets should be integrated with suitable algorithm (such as mnn, CCA, bbknn, harmony, scvi et al.), even no batch effect exists.

[wangjiawen2013]

CCA does not have code in python, which will make it difficult to integrate, pySCENIC is probably easier but I would rather ask the developers.

@falexwolf We should consider a way to facilitate scanpy 'plugins'. A quick search shows me that this could be possible: https://packaging.python.org/guides/creating-and-discovering-plugins/ but honestly I don't know how it works. Nevertheless, given the number of tools that continue to appear we should consider a scheme that facilitate how developers can take advantage of scanpy preprocessing, storing, analysis and visualization tools.

Pyscenic has been integrated into scanpy now! Here is the hyper link:
https://github.com/aertslab/pySCENIC/blob/master/notebooks/pySCENIC%20-%20Integration%20with%20scanpy.ipynb

[flying-sheep]

Great! Should we announce this somehow?

[ivirshup]

Maybe we should have a section in the docs for linking to packages that work with scanpy?