Hdbscan: clang: error: no such file or directory: '_hdbscan_tree.c'

[valid92]

Hello,
i am trying to import hdbscan but it was give me the error
clang: error: no such file or directory: '_hdbscan_tree.c'

the complete error text:

Collecting hdbscan
Using cached hdbscan-0.8.10.tar.gz
Requirement already satisfied: scikit-learn>=0.16 in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (from hdbscan)
Building wheels for collected packages: hdbscan
Running setup.py bdist_wheel for hdbscan: started
Running setup.py bdist_wheel for hdbscan: finished with status 'error'
Complete output from command /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/private/var/folders/8h/zr087l41463g28nrsz8hh5sr0000gn/T/pycharm-packaging/hdbscan/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /var/folders/8h/zr087l41463g28nrsz8hh5sr0000gn/T/tmpjqt6rmyapip-wheel- --python-tag cp36:
/private/var/folders/8h/zr087l41463g28nrsz8hh5sr0000gn/T/pycharm-packaging/hdbscan/setup.py:8: UserWarning: No module named 'Cython'
warnings.warn(e.args[0])
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.6-intel-3.6
creating build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/__init__.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/hdbscan_.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/plots.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/prediction.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/robust_single_linkage_.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/validity.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
warning: build_py: byte-compiling is disabled, skipping.

running build_ext
building 'hdbscan._hdbscan_tree' extension
creating build/temp.macosx-10.6-intel-3.6
/usr/bin/clang -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c _hdbscan_tree.c -o build/temp.macosx-10.6-intel-3.6/_hdbscan_tree.o
clang: error: no such file or directory: '_hdbscan_tree.c'
clang: error: no input files
error: command '/usr/bin/clang' failed with exit status 1

---

Running setup.py clean for hdbscan
Failed to build hdbscan
Installing collected packages: hdbscan
Running setup.py install for hdbscan: started
Running setup.py install for hdbscan: finished with status 'error'
Complete output from command /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/private/var/folders/8h/zr087l41463g28nrsz8hh5sr0000gn/T/pycharm-packaging/hdbscan/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/8h/zr087l41463g28nrsz8hh5sr0000gn/T/pip-_srb5f0d-record/install-record.txt --single-version-externally-managed --compile:
/private/var/folders/8h/zr087l41463g28nrsz8hh5sr0000gn/T/pycharm-packaging/hdbscan/setup.py:8: UserWarning: No module named 'Cython'
warnings.warn(e.args[0])
running install
running build
running build_py
creating build
creating build/lib.macosx-10.6-intel-3.6
creating build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/__init__.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/hdbscan_.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/plots.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/prediction.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/robust_single_linkage_.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
copying hdbscan/validity.py -> build/lib.macosx-10.6-intel-3.6/hdbscan
warning: build_py: byte-compiling is disabled, skipping.

running build_ext
building 'hdbscan._hdbscan_tree' extension
creating build/temp.macosx-10.6-intel-3.6
/usr/bin/clang -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c _hdbscan_tree.c -o build/temp.macosx-10.6-intel-3.6/_hdbscan_tree.o
clang: error: no such file or directory: '_hdbscan_tree.c'
clang: error: no input files
error: command '/usr/bin/clang' failed with exit status 1

----------------------------------------

Failed building wheel for hdbscan
Command "/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/private/var/folders/8h/zr087l41463g28nrsz8hh5sr0000gn/T/pycharm-packaging/hdbscan/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/8h/zr087l41463g28nrsz8hh5sr0000gn/T/pip-_srb5f0d-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/8h/zr087l41463g28nrsz8hh5sr0000gn/T/pycharm-packaging/hdbscan/

[lmcinnes]

I highly recommend using the conda packages if you can:

conda install -c conda-forge hdbscan

will generally just work; pip installing involves compilation and can be ... troublesome on some machines. As a first workaround if you need to compile the package I would suggest you install cython. Ideally that shouldn't be a requirement, and I'll check that all the appropriate files are committed for the current version, but certainly installing cython will make things go a little easier.

Thanks for reporting this -- this should work, and the fact that it doesn't is a problem. Hopefully one of the workarounds I proposed above will work for you in the meantime.

[valid92]

i did it with conda install -c conda-forge hdbscan
and i already have installed cython
but i still get : clang: error: no such file or directory: '_hdbscan_tree.c'

[lmcinnes]

Hmm, okay, that it definitely bad :-( Let me dig into this a little.

[lmcinnes]

Perhaps commit 181fcbf6df31dd98c9028bf4e24907fe6935ca1c fixes this for you? See if you can clone the current master and see if that builds better?

[lmcinnes]

I admit that I don't entirely understand quite what could be going wrong here -- the error output you provide seems pretty clearly to be trying to build hdbscan (as far as I can via pip), and the conda install, if done correctly should be pre-built and thus if it errors have a completely different error messgae. You may want to purge any hdbscan or related directories in your site-packages and try re-installing? I admit that I am slightly at a loss.

[valid92]

i am working with pycharm. Could this be an problem

[lmcinnes]

It may be trying to do things automatically -- I'm honestly not sure; I
don't have too much experience with that.

On Tue, Jun 20, 2017 at 2:42 PM, valid92 [e-mail] wrote:

i am working with pycharm. Could be this an problem

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/scikit-learn-contrib/hdbscan/issues/117#issuecomment-309850851,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALaKBUqaVBe8k9wnAFE9kwvtwjYe2vm5ks5sGBJ_gaJpZM4N_8_P
.

[valid92]

solution: if you have multiply version of python and want to use python 3 do:
pip3 install cython
pip3 install hdbscan
this solved the problem above :)

[lmcinnes]

Excellent. I'm glad a resolution was found -- sorry that it was not necessarily obvious. Is there anything we could add in documentation that might make this easier to find?

[valid92]

yes please! maybe a hint in documentation :
if nothing works and you work with multiply python version
then one may try :
pip3 install cython
pip3 install hdbscan

[lmcinnes]

Thanks -- I have a bunch of documentation overhaul to do when I get some time, so I'll add this to the list. Thanks for reporting the issue.

[dte128]

Hi Imcinness,
Firstly thanks a lot for sharing and develop this very useful module. I recently passed to Python3 and I noticed I've got the same issue reported above even following the suggested steps. At first, I thought was due to some sort of scipy/numpy/sklearn versioning issues, but the versions were matching with the requirements.txt. At the same time I noticed that removing and reinstalling the Python2 version everything was fine.

I believe that the error above is due to the missing python3 dev which it looks like not installed by default in some Ubuntu distro. Here how I fixed this error (hope it helps others with the same issue ):

For python3:
sudo -H pip3 install --upgrade pip
sudo apt-get install python3-dev
pip3 install cython
pip3 install hdbscan

For python2:
sudo -H pip2 install --upgrade pip
sudo apt-get install python-dev
pip2 install cython
pip2 install hdbscan

[lmcinnes]

Thanks for posting the details. I'm going to leave this issue open so others can find these instructions until I get the time for a documentation overhaul.

[patvdleer]

Can confirm, I was missing cython, after installing it (via pip) I could finish the hdbscan install successfully.

[proinsias]

@toofanix and I encountered this issue and were able to resolve it by installing from master.
@lmcinnes – could you create a new release so we can install this from pypi?

[lmcinnes]

I can try to get that done. Sorry for the issues; I've been very busy on other projects at the moment. When I finally get some time there are some major updates to hdbscan coming...

[lmcinnes]

There should be a new version on the new PyPI now (presuming I have successfully transitioned to the new way to upload stuff correctly). The conda-forge version will be shortly behind.