Spacy: Issue in loading 'en_core_web_sm', getting **AttributeError: 'module' object has no attribute 'unpack'**

[Devadath]

System details
OS name: Linux
OS architecture: amd64
OS version: 4.4.0-112-generic

Error
Python 2.7.12 (default, Nov 20 2017, 18:23:56)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import spacy
nlp = spacy.load('en_core_web_sm')
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/spacy/__init__.py", line 19, in load
return util.load_model(name, overrides)
File "/usr/local/lib/python2.7/dist-packages/spacy/util.py", line 115, in load_model
return load_model_from_package(name, *overrides)
File "/usr/local/lib/python2.7/dist-packages/spacy/util.py", line 138, in load_model_from_package
return cls.load(overrides)
File "/usr/local/lib/python2.7/dist-packages/en_core_web_sm/__init__.py", line 12, in load
return load_model_from_init_py(__file__, overrides)
File "/usr/local/lib/python2.7/dist-packages/spacy/util.py", line 177, in load_model_from_init_py
return load_model_from_path(data_path, meta, *overrides)
File "/usr/local/lib/python2.7/dist-packages/spacy/util.py", line 159, in load_model_from_path
return nlp.from_disk(model_path)
File "/usr/local/lib/python2.7/dist-packages/spacy/language.py", line 638, in from_disk
util.from_disk(path, deserializers, exclude)
File "/usr/local/lib/python2.7/dist-packages/spacy/util.py", line 522, in from_disk
reader(path / key)
File "/usr/local/lib/python2.7/dist-packages/spacy/language.py", line 625, in
('vocab', lambda p: self.vocab.from_disk(p)),
File "vocab.pyx", line 383, in spacy.vocab.Vocab.from_disk
File "vectors.pyx", line 372, in spacy.vectors.Vectors.from_disk
File "/usr/local/lib/python2.7/dist-packages/spacy/util.py", line 522, in from_disk
reader(path / key)
File "vectors.pyx", line 350, in spacy.vectors.Vectors.from_disk.load_key2row
File "vectors.pyx", line 351, in spacy.vectors.Vectors.from_disk.load_key2row
File "/home/dheeyantra/.local/lib/python2.7/site-packages/msgpack_numpy.py", line 184, in unpack
return _unpacker.unpack(stream, *kwargs)
*AttributeError: 'module' object has no attribute 'unpack'

[rawinkler]

Hi, something similar here:

Python 3.6.3 (default, Dec 12 2017, 06:40:22)
[GCC 4.9.2] on linux

import spacy
import de_core_news_sm
nlp = de_core_news_sm.load()
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.6/site-packages/de_core_news_sm/__init__.py", line 12, in load
return load_model_from_init_py(__file__, *overrides)
File "/usr/local/lib/python3.6/site-packages/spacy/util.py", line 177, in load_model_from_init_py
return load_model_from_path(data_path, meta, *overrides)
File "/usr/local/lib/python3.6/site-packages/spacy/util.py", line 159, in load_model_from_path
return nlp.from_disk(model_path)
File "/usr/local/lib/python3.6/site-packages/spacy/language.py", line 638, in from_disk
util.from_disk(path, deserializers, exclude)
File "/usr/local/lib/python3.6/site-packages/spacy/util.py", line 522, in from_disk
reader(path / key)
File "/usr/local/lib/python3.6/site-packages/spacy/language.py", line 625, in
('vocab', lambda p: self.vocab.from_disk(p)),
File "vocab.pyx", line 383, in spacy.vocab.Vocab.from_disk
File "vectors.pyx", line 372, in spacy.vectors.Vectors.from_disk
File "/usr/local/lib/python3.6/site-packages/spacy/util.py", line 522, in from_disk
reader(path / key)
File "vectors.pyx", line 350, in spacy.vectors.Vectors.from_disk.load_key2row
File "vectors.pyx", line 351, in spacy.vectors.Vectors.from_disk.load_key2row
File "/usr/local/lib/python3.6/site-packages/msgpack_numpy.py", line 179, in unpack
return _unpacker.unpack(stream, encoding=encoding, **kwargs)
AttributeError: module 'msgpack._unpacker' has no attribute 'unpack'

[ines]

Thanks for the report! Could you check which versions of msgpack-python and msgpack-numpy are installed in your spaCy environment?

pip list | grep msgpack

[iuliux]

(having the same issue)

msgpack-numpy (0.4.1)
msgpack-python (0.5.5)

Upgraded msgpack-numpy to 0.4.2. Issue still present.

[Devadath]

msgpack-numpy (0.4.1)
msgpack-python (0.5.4)

[lahloug]

Same issue here :

Python 3.5.3
spacy version : 2.0.8
msgpack (0.5.0)
msgpack-numpy (0.4.1)
msgpack-python (0.5.5)

[ines]

Thanks for updating – this is very strange!

@lahloug One possible explanation: msgpack-python is also imported as msgpack, so maybe your msgpack (0.5.0) is shadowing that? So spaCy is importing the wrong library, which doesn't have that method. What happens when you uninstall msgpack?

@Devadath @iuliux Do you also see an additional msgpack installation when you run pip list?

(If it turns out that the problem is caused by one library shadowing another, we'll have to think about how to address this problem. As far as I know, there was some reason we had to go with msgpack-python as opposed to msgpack.)

[lahloug]

Thanks Ines for your quick answer. That may be the issue. I have been looking on another direction.
msgpack-python has just released 0.5.5 2 hours ago, so I suspected that may be the problem. Indeed, when I install msgpack-python 0.5.4, loading model works well.

pip install msgpack-python==0.5.4

Can you consider locking msgpack-python version to 0.5.4? When I checked the requirements, you don't specify a version for msgpack-python.

[iuliux]

Indeed, msgpack-python==0.5.4 and no msgpack is working.

Potential pitfall: uninstalling msgpack unlinked the msgpack module name, I had to reinstall msgpack-python

[Devadath]

Thanks @lahloug your solution is working. @ines Thanks for quick response and hope you will be doing a permanent fix.

[rawinkler]

The downgrade of msgpack-python to 0.5.4 indeed is a good fix! Thanks everybody for the hints and support!

[ines]

Thanks everyone, this makes a lot more sense now. And sorry for the frustration this caused.

I just pushed a fix that locks msgpack-python to 0.5.4. I also had a look at the msgpack package and it looks like msgpack-python has actually been renamed. This also explains the pitfall described by @iuliux above, which seems to be intentional... 🤔

Package name on PyPI was changed to msgpack from 0.5. I upload transitional package (msgpack-python 0.5 which depending on msgpack) for smooth transition from msgpack-python to msgpack.

Sadly, this doesn’t work for upgrade install. After pip install -U msgpack-python, msgpack is removed and import msgpack fail.

Not sure how I feel about this. In any case, it looks like we need to work on a quick fix and move spaCy over to msgpack ...

[jiyuan312986471]

Having the same issue when I have:

• msgpack(0.5.5)
• msgpack-numpy(0.4.1)
• msgpack-python(0.5.4)

When I uninstalled msgpack I got such error:

Traceback (most recent call last):
  File "C:\Users\ji.yuan\Documents\workspace\SinistresAutomobiles\G1T\deep_learning\test.py", line 1, in <module>
    import spacy
  File "C:\Program Files\Python36\lib\site-packages\spacy\__init__.py", line 4, in <module>
    from .cli.info import info as cli_info
  File "C:\Program Files\Python36\lib\site-packages\spacy\cli\__init__.py", line 1, in <module>
    from .download import download
  File "C:\Program Files\Python36\lib\site-packages\spacy\cli\download.py", line 10, in <module>
    from .link import link
  File "C:\Program Files\Python36\lib\site-packages\spacy\cli\link.py", line 7, in <module>
    from ..compat import symlink_to, path2str
  File "C:\Program Files\Python36\lib\site-packages\spacy\compat.py", line 11, in <module>
    from thinc.neural.util import copy_array
  File "C:\Program Files\Python36\lib\site-packages\thinc\neural\__init__.py", line 1, in <module>
    from ._classes.model import Model
  File "C:\Program Files\Python36\lib\site-packages\thinc\neural\_classes\model.py", line 5, in <module>
    import msgpack
ModuleNotFoundError: No module named 'msgpack'

Does anyone fall in the same situation as mine ?

[jiyuan312986471]

Update:
Issue fixed by downgrading msgpack to 0.5.4

Sorry for bothering

[ines]

Just published spaCy v2.0.9 which uses the correct version pin and should solve the issue. Thanks again for your patience!

pip install -U spacy

(For the upcoming v2.1.0, we'll then take care of moving to msgpack properly.)

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.