Scanpy: counts_per_cell has no effect on pp.normalize_per_cell(adata)

[VolkerBergen]

counts_per_cell has no effect on pp.normalize_per_cell(adata) as it is overwritten here:

https://github.com/theislab/scanpy/blob/e0d2ea60fa2394f1158618bc69dff4fc639b51c4/scanpy/preprocessing/_simple.py#L599

Would be great if one could use, e.g. the initial cell size (from before filtering). Note, that this issue only holds when passing AnnData. With AnnData.X it works fine.

[falexwolf]

@Koncopd: will this still be an issue in the new implementations of normalize_total and normalize_quantile? Would be great if you checked!

[Koncopd]

@falexwolf
Hm, new versions don't have counts_per_cell at all.

[falexwolf]

@VolkerBergen, is this really important? I had the intend of allowing passing a precomputed counts_per_cell vector, but I think it wasn't really ever used... So, for a simpler function and cleaner code, it would be nice to get rid of it; as @Koncopd did for the new version. Any objections?

[VolkerBergen]

Totally fine. Was just wondering whether we can make it work as expected for the deprecated normalize_per_cell module, which is just modifying a single line (PR?).

My use case: I have found that, in the context of RNA velocity analysis, normalizing by initial sizes (cell sizes before filtering) can be quite important. That is where I used to pass the counts_per_cell attribute.

Here, I made it the default:
https://github.com/theislab/scvelo/blob/c86e4530485e6e62a055c5a9285177b9554613ce/scvelo/preprocessing/utils.py#L256

[falexwolf]

Yes, of course, happy to see a PR! Thank you!

Also, thank you for the details!

[Koncopd]

Shoild i add counts_per_cell to the new normalization functions?

[VolkerBergen]

I find normalising by initial cell sizes quite important. That's the only use case for me where I use counts_per_cell. And I believe that applies to many others as well. Thus, one might think of having a bool option such as by_initial_size instead. What do you think, @falexwolf ?

In that case, one would have to store the initial cell size before filtering (either when doing the gene filtering or even when reading the data).

[falexwolf]

I like @VolkerBergen's suggestion. On the other Hand, @LuckyMD uses the scran estimate of size factors for normalization. Processing something like that would need a counts_per_cell argument (which I'd call normalization_factor today, I guess).

If one needs to manually compute the counts_per_cell before calling the function, then the whole convenience and purpose of the function is gone, though. So, I'd say the convenience of an argument by_initial absolutely outweighs the flexibility of an argument normalization_factor (size_factor).

In case we have another size factor estimator in Scanpy, it will definitely not occur in normalize_total or normalize_quantile (the names already suggest that this is simple normalization) but in a new function normalize_......

[LuckyMD]

I was wondering if using the initial total_counts versus the post-filtering total_counts really matter that much. In the end we typically only filter out genes that have very few counts, so that the difference between the initial and post-filtering total_counts should be minimal. Principally using pre-filtering values is probably more logical, although I'm not sure it really changes anything.

I wonder how hard it would be to put scran's size factor calculation into python... that might be a good HiWi project.

[VolkerBergen]

It does not matter much in most cases. Just in the mentioned use case of RNA velocity analysis it does as many cells usually get filtered out because of low unspliced count nums. Whatsoever, it is no effort to compute initial sizes beforehand and you are ensured to do the "right thing", no matter how much you've filtered out.

[Koncopd]

Also new functions don't have filtering.

[VolkerBergen]

That's good to know! That means, filter_genes would simply annotate the genes kept just like highly_variable_genes?

[LuckyMD]

@VolkerBergen If you're filtering out cells, you wouldn't have different total counts per cell, right? Only if you filter out genes. Or is that what you meant?

[VolkerBergen]

Sorry, meant genes, of course.

[Koncopd]

@VolkerBergen
Hm, i also meant filtering of cells. No filtering of cells in normalize_{total, quantile}.
What do you mean by filtering of genes in relation to normalization?

[Koncopd]

What should the argument by_initial do?

[VolkerBergen]

Filtering out genes would obviously change the cell sizes. Hence, with the argument by_initial_size it was meant that initial sizes (before filtering) are used. But when filter_genes only annotates instead of actually filtering, it does not matter indeed.

[falexwolf]

@LuckyMD: Yes, scran's size factor calculation would be very nice-to-have and should be a simple task.

[LuckyMD]

I will put it on my long list of potential HiWi topics...

[saberyzy]

Hi, I am new to scanpy and trying to use scanpy.pp.normalize_total to normalize my dataset. I saw in the documentation saying 'key_added' parameter will store the normalization factor (size factor). However, the result in the new column stores the same value as total counts before normalization.

Are this function using similar strategy as scran? Do we have normalization factors stored somewhere else?

Thanks!

[LuckyMD]

Hi @saberyzy,

The size factor for counts per million normalization (the default for sc.pp.normalize_total) is the total counts. That value is used to compute the normalized values (X/total_counts * factor; where the factor can be 10,000 or the median total counts across cells), so it is the correct size factor. The function doesn't do anything fancier at the moment.

[saberyzy]

Hi, @LuckyMD,

Thanks for the answer!
In your tutorial paper you recommended to use scran for normalization. It seems like scanpy do not have a similar normalization function. So the current best way is still use a docker to use scran to normalize the data?

[LuckyMD]

Indeed, scran often works better than CPM normalization. I would recommend to use it via anndata2ri or rpy2 as I demonstrate in the case study with the paper. You can find that [here] (https://github.com/theislab/single-cell-tutorial/blob/master/latest_notebook/Case-study_Mouse-intestinal-epithelium_1906.ipynb)