Scanpy: Getting the error: "numba.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)" when running sc.tl.umap with init_pos='paga'

[ndrubins]

Hi,

I'm trying to follow the Dahlin18 PAGA tutorial

And in the part where it calls the UMAP function providing it with the PAGA initial points (line 28 in the notebook: sc.tl.umap(adata, init_pos='paga')), I'm getting this error message:

computing UMAP
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nr/miniconda3/lib/python3.7/site-packages/scanpy/tools/_umap.py", line 145, in umap
    verbose=settings.verbosity > 3,
  File "/home/nr/miniconda3/lib/python3.7/site-packages/umap/umap_.py", line 1005, in simplicial_set_embedding
    verbose=verbose,
  File "/home/nr/miniconda3/lib/python3.7/site-packages/numba/dispatcher.py", line 401, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/home/nr/miniconda3/lib/python3.7/site-packages/numba/dispatcher.py", line 344, in error_rewrite
    reraise(type(e), e, None)
  File "/home/nr/miniconda3/lib/python3.7/site-packages/numba/six.py", line 668, in reraise
    raise value.with_traceback(tb)
numba.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Invalid use of type(CPUDispatcher(<function rdist at 0x7f8ffd913050>)) with parameters (array(float64, 1d, C), array(float32, 1d, C))
Known signatures:
 * (array(float32, 1d, A), array(float32, 1d, A)) -> float32
 * parameterized
[1] During: resolving callee type: type(CPUDispatcher(<function rdist at 0x7f8ffd913050>))
[2] During: typing of call at /home/nr/miniconda3/lib/python3.7/site-packages/umap/umap_.py (795)


File "miniconda3/lib/python3.7/site-packages/umap/umap_.py", line 795:
def optimize_layout(
    <source elided>

                dist_squared = rdist(current, other)
                ^

This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.

To see Python/NumPy features supported by the latest release of Numba visit:
http://numba.pydata.org/numba-doc/latest/reference/pysupported.html
and
http://numba.pydata.org/numba-doc/latest/reference/numpysupported.html

For more information about typing errors and how to debug them visit:
http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compile

If you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:
https://github.com/numba/numba/issues/new

I saw a relevant issue on the umap package and
even changed line 1138 in umap_.py from embedding to embedding..astype(np.float32, copy=True), but no success.

Any idea?

Thanks!

[paupuigdevall]

Hi,

I encounter exactly the same issue running the same line in Jupyter notebook.
Could you find any solution?

Best,

Pau.

[ndrubins]

Yes.
Within the implementation of the simplicial_set_embedding function in the umap_.py file (in my system it is located at: ~/miniconda3/lib/python3.7/site-packages/umap/umap_.py), I type-casted the embedding objects to astype(np.float32, copy=False) from within the call to the optimize_layout function: lines 1137 and 1138 in umap_.py are thus changed from:
embedding,
embedding

to:
embedding.astype(np.float32, copy=False),
embedding.astype(np.float32, copy=False)

[paupuigdevall]

Thank you very much. Your suggestion solved the problem.

[liboxun]

Solved same problem for me as well.

For the record, the output of sc.logging.print_versions() in my conda environment is:

scanpy==1.4.4.post1 anndata==0.6.22.post1 umap==0.3.10 numpy==1.17.2 scipy==1.3.1 pandas==0.25.2 scikit-learn==0.21.3 statsmodels==0.10.1 python-igraph==0.7.1 louvain==0.6.1

[flying-sheep]

Well, as that issue says, it’s fixed in lmcinnes/umap#261, which means it’s in umap 0.3.10

[mfansler]

Well, as that issue says, it’s fixed in lmcinnes/umap#261, which means it’s in umap 0.3.10

@flying-sheep unfortunately umap==3.10 does not fix this relative to the latest scanpy version on Bioconda (1.4.4.post1). The issue is that the UMAP fix does not address the branch of code that scanpy depends on (specifically the call follows this branch in the UMAP code), which still just passes the init_coords in as is.

Of course, there has been a workaround in scanpy since 1.4.5.post1. However, I would caution against the advice in that commit's message, which assumed that once umap==3.10 was released the workaround could be removed.

[flying-sheep]

Thanks for the info! So lmcinnes/umap#179 needs to be reopened?