Spacy: model VersionConflict on upgrading to 2.2.1

[Motorrat]

How to reproduce the problem

Use virtual environment. Upgrade spacy 2.1.8 to 2.2.0 and upgrade the models.
make symlink

ln -s /home/user/venv/full/lib/python3.6/site-packages/de_core_news_md -->
/home/user/venv/full/lib/python3.6/site-packages/spacy/data/de_core_news_md

Then try loading the model with nlp=spacy.load('de_core_news_md')

and get the following error:

pkg_resources.VersionConflict: (spacy 2.2.1 (/home/user/venv/full/lib/python3.6/site-packages), Requirement.parse('spacy<2.2.0,>=2.1.7'))

So here are detailed steps (all run in virtual environment "full"):

pip install --upgrade spacy
Successfully installed blis-0.4.1 preshed-3.0.2 spacy-2.2.1 thinc-7.1.1 urllib3-1.25.6

python -m spacy validate
package   de-core-news-md            de_core_news_md            2.1.0   --> 2.2.0

python -m spacy download de_core_news_md
...
Successfully installed de-core-news-md-2.2.0

python -m spacy download de_core_news_md
Requirement already satisfied: de_core_news_md==2.2.0 from https://github.com/explosion/spacy-models/releases/download/de_core_news_md-2.2.0/de_core_news_md-2.2.0.tar.gz#egg=de_core_news_md==2.2.0 in /home/user/venv/full/lib/python3.6/site-packages
Requirement already satisfied: spacy>=2.2.0 in /home/user/venv/full/lib/python3.6/site-packages (from de_core_news_md==2.2.0)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /home/user/venv/full/lib/python3.6/site-packages (from spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: thinc<7.2.0,>=7.1.1 in /home/user/venv/full/lib/python3.6/site-packages (from spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: blis<0.5.0,>=0.4.0 in /home/user/venv/full/lib/python3.6/site-packages (from spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /home/user/venv/full/lib/python3.6/site-packages (from spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: wasabi<1.1.0,>=0.2.0 in /home/user/venv/full/lib/python3.6/site-packages (from spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: srsly<1.1.0,>=0.1.0 in /home/user/venv/full/lib/python3.6/site-packages (from spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /home/user/venv/full/lib/python3.6/site-packages (from spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /home/user/venv/full/lib/python3.6/site-packages (from spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: numpy>=1.15.0 in /home/user/venv/full/lib/python3.6/site-packages (from spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /home/user/venv/full/lib/python3.6/site-packages (from spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /home/user/venv/full/lib/python3.6/site-packages (from thinc<7.2.0,>=7.1.1->spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: idna<2.9,>=2.5 in /home/user/venv/full/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/user/venv/full/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /home/user/venv/full/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->de_core_news_md==2.2.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/user/venv/full/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->de_core_news_md==2.2.0)
✔ Download and installation successful
You can now load the model via spacy.load('de_core_news_md')

python -m spacy validate
✔ Loaded compatibility table

====================== Installed models (spaCy v2.2.1) ======================
ℹ spaCy installation:
/home/user/venv/full/lib/python3.6/site-packages/spacy

package   de-core-news-md        de_core_news_md        2.2.0   ✔

(full) $ python
Python 3.6.8 (default, Aug 20 2019, 17:12:48) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import spacy
>>> nlp=spacy.load('de_core_news_md')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/__init__.py", line 27, in load
    return util.load_model(name, **overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 166, in load_model
    return load_model_from_package(name, **overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 187, in load_model_from_package
    return cls.load(**overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/de_core_news_md/__init__.py", line 12, in load
    return load_model_from_init_py(__file__, **overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 228, in load_model_from_init_py
    return load_model_from_path(data_path, meta, **overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 199, in load_model_from_path
    nlp = cls(meta=meta, **overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/language.py", line 175, in __init__
    user_factories = util.get_entry_points(util.ENTRY_POINTS.factories)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 286, in get_entry_points
    result[entry_point.name] = entry_point.load()
  File "/home/user/venv/full/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2433, in load
    self.require(*args, **kwargs)
  File "/home/user/venv/full/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2456, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/home/user/venv/full/lib/python3.6/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (spacy 2.2.1 (/home/user/venv/full/lib/python3.6/site-packages), Requirement.parse('spacy<2.2.0,>=2.1.7'))

Also tried installing using the code

from spacy.cli.download import download
from spacy.cli import link
download('de_core_news_md')
link('de_core_news_md', 'de_core_news_md', force=True)
spacy_pipe = spacy.load('de_core_news_md')

✔ Download and installation successful
You can now load the model via spacy.load('de_core_news_md')
✔ Linking successful
/home/user/venv/full/lib/python3.6/site-packages/de_core_news_md -->
/home/user/venv/full/lib/python3.6/site-packages/spacy/data/de_core_news_md
You can now load the model via spacy.load('de_core_news_md')

But I get the same error there as well.

Downgrading spacy to 2.1.8 fails using the new models (as expected) :

pip install spacy==2.1.8
...
      Successfully uninstalled spacy-2.2.0
Successfully installed blis-0.2.4 preshed-2.0.1 spacy-2.1.8 thinc-7.0.8

>>> nlp=spacy.load('de_core_news_md')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/__init__.py", line 27, in load
    return util.load_model(name, **overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 132, in load_model
    return load_model_from_link(name, **overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 149, in load_model_from_link
    return cls.load(**overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/data/de_core_news_md/__init__.py", line 12, in load
    return load_model_from_init_py(__file__, **overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 196, in load_model_from_init_py
    return load_model_from_path(data_path, meta, **overrides)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 179, in load_model_from_path
    return nlp.from_disk(model_path)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/language.py", line 836, in from_disk
    util.from_disk(path, deserializers, exclude)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 636, in from_disk
    reader(path / key)
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/language.py", line 831, in <lambda>
    p, exclude=["vocab"]
  File "pipes.pyx", line 641, in spacy.pipeline.pipes.Tagger.from_disk
  File "/home/user/venv/full/lib/python3.6/site-packages/spacy/util.py", line 636, in from_disk
    reader(path / key)
  File "pipes.pyx", line 629, in spacy.pipeline.pipes.Tagger.from_disk.load_tag_map
  File "morphology.pyx", line 56, in spacy.morphology.Morphology.__init__
  File "attrs.pyx", line 148, in spacy.attrs.intify_attrs
KeyError: 'PUNCTTYPE_BRCK'

Your Environment

Info about spaCy

• spaCy version: 2.2.1
• Platform: Linux-4.15.0-43-generic-x86_64-with-Ubuntu-18.04-bionic
• Python version: 3.6.8
• Models: de, fr, fr_core_news_sm, es, de_core_news_sm, la, xx, un, la_latin_spacy_model, en_core_web_sm, de_core_news_md, nl, en

[ines]

Thanks for the detailed report. It's a shame pkg_resources doesn't show more details about where exactly the version conflict occurs. But looking at the traceback, it seems to come up when spaCy is loading pipeline component factories from the entry points. Which leads me to believe that the issue might not be in the models, but in another package.

Do you have any other packages installed that register components via factories and are not compatible with v2.2 yet? For example, spacy-pytorch-transformers?

[Motorrat]

Thanks @ines. The culprit was indeed spacy-pytorch-transformers. I have removed it with pip unistall spacy-pytorch-transformers and issue got resolved. This can be closed.

[ines]

@Motorrat Thanks for checking and updating – glad you got it resolved!

Also glad this came up. I want to find a way to prevent this sort of thing, and it's frustrating because I feel like we're doing everything correctly: using appropriate version pins for our packages, using Python entry points to register extensions etc. Even just a less cryptic error message mentioning the package the conflict occurs in would be helpful. I guess I should open an issue on setuptools about this.

[petulla]

I'm having this same issue but uninstalling the past version isn't fixing. Hoping the next release resolves.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.