Scanpy: mnn_correct() ValueError: not enough values to unpack (expected 3, got 1)

[Gpasquini]

Hello,

I am having hard times using the batch correction function running matching mutual nearest neighbors.

I have an anndata with 3 batches. I want to point out that the batch correction using the sc.pp.combat() function works.
On the other hand, if I run (on the uncorrected adata):

sce.pp.mnn_correct(adata, 
                               var_index=None, 
                               var_subset=None, 
                               batch_key='batch', 
                               index_unique='-', 
                               batch_categories=None, 
                               k=20, 
                               sigma=1.0, 
                               cos_norm_in=True, 
                               cos_norm_out=True, 
                               svd_dim=None, 
                               var_adj=True, 
                               compute_angle=False, 
                               mnn_order=None, 
                               svd_mode='rsvd', 
                               do_concatenate=True, 
                               save_raw=False, 
                               n_jobs=None)

I get:

in ..../site-packages/scanpy/preprocessing/_mnn_correct.py the man_correct function is:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-47-b453bc0c2cd4> in <module>
     16                                do_concatenate=True,
     17                                save_raw=False,
---> 18                                n_jobs=None)

~/Library/Python/3.7/lib/python/site-packages/scanpy/preprocessing/_mnn_correct.py in mnn_correct(var_index, var_subset, batch_key, index_unique, batch_categories, k, sigma, cos_norm_in, cos_norm_out, svd_dim, var_adj, compute_angle, mnn_order, svd_mode, do_concatenate, save_raw, n_jobs, *datas, **kwargs)
     97             batch_categories=batch_categories, k=k, sigma=sigma, cos_norm_in=cos_norm_in, cos_norm_out=cos_norm_out,
     98             svd_dim=svd_dim, var_adj=var_adj, compute_angle=compute_angle, mnn_order=mnn_order, svd_mode=svd_mode,
---> 99             do_concatenate=do_concatenate, save_raw=save_raw, n_jobs=n_jobs, **kwargs)
    100         return datas, mnn_list, angle_list
    101     except ImportError:

ValueError: not enough values to unpack (expected 3, got 1)

I checked _mnn_correct.py. and it basically defines a function mnn_cor on the mnn_correct from mnnpy package:

def mnn_correct(*datas, var_index=None, var_subset=None, batch_key='batch', index_unique='-',
                batch_categories=None, k=20, sigma=1., cos_norm_in=True, cos_norm_out=True,
                svd_dim=None, var_adj=True, compute_angle=False, mnn_order=None, svd_mode='rsvd',
                do_concatenate=True, save_raw=False, n_jobs=None, **kwargs):
    try:
        from mnnpy import mnn_correct as mnn_cor
        n_jobs = settings.n_jobs if n_jobs is None else n_jobs
        datas, mnn_list, angle_list = mnn_cor(
            *datas, var_index=var_index, var_subset=var_subset, batch_key=batch_key, index_unique=index_unique,
            batch_categories=batch_categories, k=k, sigma=sigma, cos_norm_in=cos_norm_in, cos_norm_out=cos_norm_out,
            svd_dim=svd_dim, var_adj=var_adj, compute_angle=compute_angle, mnn_order=mnn_order, svd_mode=svd_mode,
            do_concatenate=do_concatenate, save_raw=save_raw, n_jobs=n_jobs, **kwargs)
        return datas, mnn_list, angle_list
    except ImportError:

I think the point is that mnn_cor is not giving 3 values in this line:

datas, mnn_list, angle_list = mnn_cor(

Can you pleas help me with that?

[ivirshup]

Reproducible example:

import scanpy as sc
import scanpy.external as ice
from itertools import cycle

pbmc = sc.datasets.pbmc68k_reduced()
sce.pp.mnn_correct(pbmc, batch_key="phase")

It looks like mnn_correct is only returning one variable, through its documentation looks like it should return three. @chriscainx, could you offer some guidance here?

As a workaround for now, you could just call mnnpy.mnn_correct with the same signature you've been using. It'll return a one-tuple with a modified anndata object.

[Gpasquini]

Thanks, mnnpy.mnn_correct is actually working

[Gpasquini]

@ivirshup Sorry I need to correct my previous answer. mnnpy.mnn_correct is not giving errors, but is returning a tuple

Check my issue here https://github.com/chriscainx/mnnpy/issues/27

[grimwoo]

I also got the similar error.

[LuckyMD]

See above and use mnnpy.mnn_correct() the scanpy external version seems to no longer be maintained and is out of date.

[flying-sheep]

Why is it not maintained? If it doesn’t work, that’s a bug.

[LuckyMD]

This issue has existed for quite a while now. So I've been using mnnpy directly. I would have expected the scanpy external version to work in the same way.

[flying-sheep]

Ha, well, it was introduced in #1. Should be easy to fix.

[LuckyMD]

so "for a while" was accurate 😆

[flying-sheep]

Actually this is almost not a bug. The first line of mnn_correct is if len(datas) < 2: return datas

So we’re just holding it wrong, and it’d work if we passed it a list of anndatas as intended.

Why isn’t the API mnn_correct(adata: AnnData, batch_key: str, *, ...)?

[flying-sheep]

fixed in 1611d6301453972bc5872a696aed064fb07923f2