Pomegranate: pip install error

[rhydomako]

Hi,
When I try to install via pip (clean virtualenvs, trying both python 2.7 and 3.5) I get the follow exception upon import:

(env_pomegranate-p3.5)tacos:projects richard$ python
Python 3.5.1 (default, Dec  7 2015, 21:59:10)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pomegranate import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/richard/projects/env_pomegranate-p3.5/lib/python3.5/site-packages/pomegranate/__init__.py", line 38, in <module>
    from .hmm import *
  File "pomegranate/hmm.pyx", line 1, in init pomegranate.hmm (pomegranate/hmm.c:53522)
  File "pomegranate/distributions.pxd", line 9, in init pomegranate.base (pomegranate/base.c:22884)
  File "pomegranate/distributions.pyx", line 1, in init pomegranate.distributions (pomegranate/distributions.c:67644)
ImportError: No module named 'pomegranate.utils'

Indeed, when I look in site-packages/pomegranate, there isn't a utils.so shared library included. In contrast, when I checkout and build from source, utils.so is compiled and the importing works fine.

[jmschrei]

I can't reproduce this problem on my machine. I just successfully installed it. Make sure you're using the right version, because this is reminiscent of an error I had a while ago. Try pip install pomegranate==0.4.0

[rhydomako]

For whatever reason, it seems like the wheel file on my machine ended up being built without the utils.so file. When I removed the cached wheel file, the next build compiled the file properly and proceeded without issue.

So I'll close this issue because it was hopefully just some weirdness on my end.

[dat-boris]

FYI - Seen same transient error.
pip uninstall pomegranate && pip install pomegranate==0.4.0
fixed it but have not been able to narrow down.

[danstowell]

I've just hit a rather similar error.

% pip install pomegranate
[...]
Successfully installed pomegranate cython joblib networkx
% ipython
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
Type "copyright", "credits" or "license" for more information.
IPython 1.2.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
In [1]: import pomegranate
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
 in ()
----> 1 import pomegranate
/usr/local/lib/python2.7/dist-packages/pomegranate/__init__.py in ()
     36     pyximport.install()
     37 
---> 38 from .hmm import *
     39 from .BayesianNetwork import *
     40 from .FactorGraph import *
/home/dans/pomegranate/hmm.pyx in init pomegranate.hmm (pomegranate/hmm.c:53522)()
/home/dans/pomegranate/distributions.pxd in init pomegranate.base (pomegranate/base.c:22884)()
/home/dans/pomegranate/distributions.pyx in init pomegranate.distributions (pomegranate/distributions.c:67644)()
ImportError: No module named utils

If I do pip uninstall pomegranate && pip install pomegranate then this problem goes away. So there seems to be something about dependencies or order-of-operations in your package install?

[dflatow]

@rhydomako Hey I'm still having this exact issue and pip uninstall pomegranate && pip install pomegranate==0.4.0 doesn't fix it for me. Anything else I should try?

[dflatow]

ok, I just cloned the repo and installed, that works.

[rhydomako]

My suspicion is that occasionally, for whatever reason, utils.so (or one of the other library files) will fail to compile during installation. However, it must be some odd edge case, because I just tried about a dozen times and it succeeded for all of them.

Moreover, I think the reason that uninstalling and reinstalling sometimes doesn't work is that pip will bundle the compiled libraries into a wheel and store that wheel file in a local cache. The first time I ran into this problem, when I examined the wheel file it didn't have the utils.so library. But since the malformed wheel was cached, uninstalling and reinstalling just restored that same bad set of libraries. I even tried creating new virtualenv environments, but since the wheel cache was located outside of the virtualenv path, the problem continued.

So in case someone else runs into this problem, and uninstalling/reinstalling doesn't help, I suggest also trying to track down and delete and wheel files that might be cached. On my Mac, I was able to find where pip hides that file:

$ find ~/Library/Caches/pip/wheels -name pomegranate*
/Users/richard/Library/Caches/pip/wheels/e8/f1/da/1517d04d32c950d46d874e482735248bb156aeca81f37258b0/pomegranate-0.4.0-cp27-cp27m-macosx_10_11_x86_64.whl

Unfortunately, this is one of those edge cases that are really annoying. I've tried to reproduce the error state, but it has compiled successfully ever time since. I agree with @danstowell that it might be a subtle dependency or order-of-compilation corner-case, but I don't really know what to do to nail it down when it is so transient.

[dflatow]

Hey @rhydomako @jmschrei I've loved using pomegranate, however, I've encountered this problem, not only on my own computer but on a co-worker's computer and also when doing a fresh install to a brand new machine. It doesn't seem like a "edge case". Also, seems like this is a new issue of importing numpy?

For example I'm trying to make a heroku deployment and this is failing. Could you please take a look? I may have to ditch pomegranate otherwise :(

Thank you!

[jmschrei]

Is numpy otherwise installed correctly at its most recent version? That seems like a weird bug.

[jmschrei]

I don't know much about heroku (I've just started reading about it) but can you provide more information about your OS, and versions of properly installed dependencies? That would be most helpful in me trying to reproduce the error.

[dflatow]

hmmm not sure (it's remote dyno on heroku) architecture and here. It was the very first pip install on the machine. does numpy have to already be installed or can it be a dependency that's handled?

[jmschrei]

Try installing all of the dependencies individually first if you can. If it's possible to use the Anaconda distribution, I know that seems to work perfectly. Technically it should install all of the dependencies first, but this type of thing is not my strength so I likely messed it up.

[dflatow]

ok so it worked on the second try. you might want to check how pandas, for example, handles the dependency. I won't use anaconda and I can't always do pip twice.

[jmschrei]

Thanks for the suggestion, I'll take a look at it and try to resolve the problem. Sorry the inconvenience--I hope you find pomegranate worth the hassle!

[dflatow]

no worries! pomogranate is great! Always struggle with dependencies and installation -- it's tough

[jcornford]

FYI: I had the same issue with

38 from .hmm import *
...
ImportError: No module named utils

When installing on a conda environment. As suggested it was fixed with

pip uninstall pomegranate
pip install pomegranate --no-cache-dir

[andrew8088]

Just wanted to say that @jcornford's solution worked for me (even though I'm not using conda). Thanks!

[nikisix]

Thanks @rhydomako,
Other cache-busters may need to add quotes to the pattern:

find ~/Library/Caches/pip/wheels -name "pomegranate*"