Umap: AttributeError: module 'scipy.sparse' has no attribute 'csgraph'

[amirsani]

Hello,

Thank you for the great contribution.

I can't seem to get it running. Any help is appreciated.

Here are my versions:

Requirement already satisfied: umap-learn in ./anaconda3/lib/python3.6/site-packages
Requirement already satisfied: numba>=0.34 in ./anaconda3/lib/python3.6/site-packages (from umap-learn)
Requirement already satisfied: scipy>=0.19 in ./anaconda3/lib/python3.6/site-packages (from umap-learn)
Requirement already satisfied: scikit-learn>=0.16 in ./anaconda3/lib/python3.6/site-packages (from umap-learn)
Requirement already satisfied: llvmlite in ./anaconda3/lib/python3.6/site-packages (from numba>=0.34->umap-learn)
Requirement already satisfied: numpy in ./anaconda3/lib/python3.6/site-packages (from numba>=0.34->umap-learn)

Running the example,

import umap
from sklearn.datasets import load_digits

digits = load_digits()

embedding = umap.UMAP().fit_transform(digits.data)

outputs:

AttributeError                            Traceback (most recent call last)
<ipython-input-32-e5c7a5ee7150> in <module>()
      4 digits = load_digits()
      5 
----> 6 embedding = umap.UMAP().fit_transform(digits.data)

~/anaconda3/lib/python3.6/site-packages/umap/umap_.py in fit_transform(self, X, y)

~/anaconda3/lib/python3.6/site-packages/umap/umap_.py in fit(self, X, y)

~/anaconda3/lib/python3.6/site-packages/umap/umap_.py in simplicial_set_embedding(graph, n_components, initial_alpha, a, b, gamma, negative_sample_rate, n_epochs, init, random_state, verbose)

~/anaconda3/lib/python3.6/site-packages/umap/umap_.py in spectral_layout(graph, dim, random_state)

AttributeError: module 'scipy.sparse' has no attribute 'csgraph'

But I can import csgraph witout problems from scipy,
from scipy.sparse import csgraph

[lmcinnes]

I am a little puzzled by this -- scipy >= 0.19 does have that module according to the docs:

• https://docs.scipy.org/doc/scipy-1.0.0/reference/sparse.csgraph.html
• https://docs.scipy.org/doc/scipy-0.19.0/reference/sparse.csgraph.html
  I'm not sure why it wouldn't be found.

[amirsani]

I agree, the following works fine:
from scipy.sparse.csgraph import connected_components

If I add the following,
from scipy.sparse.csgraph import connected_components
before,
def spectral_layout(graph, dim, random_state):
It works fine.

Probably my install is messed up.

[lmcinnes]

It could be an issue in how some commits got merged in. I'll take a look soon just in case.

[amirsani]

Thanks Leland

[arnaudmiribel]

I have the same issue. Sorry, could you elaborate a little more on the solution that worked for you guys ?

[lmcinnes]

My understanding is that you should edit the file umap_.py to have

from scipy.sparse.csgraph import connected_components

Along with the other imports. In practice it may be worth grabbing the latest version on master here and installing from that (after uninstalling the pip version). That should fix this issue, but as I can't actually reproduce it myself I can't know for sure.

[arnaudmiribel]

Thanks @lmcinnes, that trick indeed worked for me :)

[sauln]

can't actually reproduce it myself

@arnaudmiribel and @amirsani, What system are you are using? Operating system w/ version and Python version.

We might be able add to add them to the Travis CI test runs. Currently, it's a little sparse running only on conda with 3.6 and 2.7.

[jputila]

Is this specific to Jupyter notebooks? I am on Ubuntu 16.04, python 3.5.2.

import umap
from sklearn.datasets import load_digits
#from scipy.sparse.csgraph import connected_components

digits = load_digits()

embedding = umap.UMAP(n_neighbors=5,
                      min_dist=0.3,
                      metric='correlation').fit_transform(digits.data)

The code above works if I start python from the command line, but doesn't work in a Jupyter notebook. If i uncomment the import it works in the notebook.

[amirsani]

Hello,

Yes. I was running on a Jupyter Notebook with Python 3.6.4 and Ubuntu 17.10.

from scipy.sparse.csgraph import connected_components
is all I had to add to umap_.py to get it working.

I installed umap-learn on my other system without any problems. I accidentally installed umap, as I did the first time, and wondered if this was causing any problems? I removed it and reinstalled umap-learn to get everything working on the other system.

[sauln]

Thanks for the info. Sounds like it could be Jupyter or Ubuntu related.

One more question for both of you, are you using conda or pip to install everything?

[amirsani]

pip install umap-learn

[wflynny]

This looks like it's related to SciPy version 1.0.0. I assume it's working for SciPy version =< 0.19.

I currently have SciPy version 1.0.0 (released October 25, 2017) installed via conda/pip and I also have this problem. This commit to SciPy a month ago adds back the import of csgraph to scipy.sparse but there hasn't been a new release of SciPy since.

It might be best to directly import from scipy.sparse.csgraph import connected_components as suggested.

[gforsyth]

I can confirm that I just ran into this using SciPy 1.0.0 and @wflynny has identified the fix for the compatibility regression.
If the preference is for the smallest change, I think adding in a
import scipy.sparse.csgraph
will do the trick (as a stopgap until SciPy 1.0.1 is released)

[arnaudmiribel]

@sauln I am using OSX High Sierra 10.13.2 and Python 3.6.4 (Anaconda, Inc)

[lmcinnes]

Thanks @wflynny and @gforsyth That looks like it is the source of the issue. I believe I added the relevant import in master, but hadn't rolled out a pip release yet. I'll try to get that done tonight and hopefully people will be seeing fewer issues.

[lmcinnes]

New version now on PyPI should fix this. 0.2.1 or greater is what you should need if you are currently seeing this issue.

[gokceneraslan]

You can maybe exclude scipy==1.0 in setup.py.

[lmcinnes]

@gokceneraslan Sounds like a reasonable plan ... I'm just not sure how to do that as I'm not much of an expert on packaging and the required semantics for doing that within setup.py. I believe the current version of PyPI should fix it as is, so hopefully that will suffice.

[wflynny]

@lmcinnes I think you could do something like (according to PEP440):

'install_requires' : ['scikit-learn >= 0.16',
                      'scipy >= 0.19, !=1.0.0',
                      'numba >= 0.34'],

I've opened a PR.

[gforsyth]

I mentioned this in #47 but I don't think excluding SciPy 1.0.0 is necessary and could make installation more of a headache. I think the import fix that @lmcinnes already made is sufficient.

[wflynny]

@gforsyth Fine by me, just giving @lmcinnes the option. The PR can sit open until newer versions of scipy are released. Or it can be closed.

[wangchuan2008888]

Scipy 1.1.0 has release, so in ubuntu python 3.6.4 just
pip install -U scipy
scipy 1.1.0 installed, problem sloved!

[sleighsoft]

Did this get resolved?

[anshumankmr]

I was having the exact same issue with scipy.

module 'scipy.linalg' has no attribute 'decomp'

[lmcinnes]

It should be resolved with the latest versions of scipy. Not sure why there are any lingering issues.