Scanpy: Regress Out using all genes

[Khalid-Usman]

Why can't I use regress_out function for scRNA-seq data without applying highly_variable_genes. Also I think regress_out function should be before highly_variable_genes, because in this way we can first remove batch effect and then select important genes.

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)
sc.pl.highly_variable_genes(adata)

adata = adata[:, adata.var['highly_variable']]

print(adata)

Regressing

sc.pp.regress_out(adata, ['n_counts', 'percent_mito'])

[LuckyMD]

Hi @Khalid-Usman,

Regressing out should indeed be performed before highly variable gene selection. This was not in the original scRNA-seq tutorials from Seurat and Scanpy though. If you're interested in a current best-practices tutorial (based on scanpy, but also including R tools), you can find it here.

The reason it might not have been done on all genes initially is for speed.

[GuidoBarzaghi]

Hi everyone!

I was wondering about the same issue. Would you then suggest to regress after subsetting HVGs (for speed reasons) and then re-searching and re-subsetting HVGs after the unwanted source of variation is corrected for? In such a way, one would inevitably loose some interesting genes (due to the first HVGs subsetting), but a cleaner signal would be obtained at the end.

[LuckyMD]

I don't think I would do HVG selection twice. I reckon it's often not such a time-taking process to regress out across all genes. Just do it once, and then do HVG selection afterwards. However, with the current setup this is difficult as sc.pp.regress_out also removes the offset, and thus has 0 mean per gene. This prevents sc.pp.highly_variable from binning the genes by mean expression (see #722).