Spacy: ModuleNotFoundError: No module named 'tokenizations.tokenizations'

[bwholman]

How to reproduce the problem

Working in Visual Studio Code terminal window. New Python project, new virtual env:

python -m venv .env
.envScriptsactivate
pip install -U pip setuptools wheel
pip install -U spacy-nightly --pre
python -m spacy validate

output from spacy validate

Traceback (most recent call last):
File "C:Python39librunpy.py", line 188, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "C:Python39librunpy.py", line 147, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "C:Python39librunpy.py", line 111, in _get_module_details
__import__(pkg_name)
File "C:Usersbrucesourcereposedith_functions.envlibsite-packagesspacy__init__.py", line 13, in
from . import pipeline # noqa: F401
File "C:Usersbrucesourcereposedith_functions.envlibsite-packagesspacypipeline__init__.py", line 1, in
from .attributeruler import AttributeRuler
File "C:Usersbrucesourcereposedith_functions.envlibsite-packagesspacypipelineattributeruler.py", line 6, in
from .pipe import Pipe
File "spacypipelinepipe.pyx", line 8, in init spacy.pipeline.pipe
File "C:Usersbrucesourcereposedith_functions.envlibsite-packagesspacytraining__init__.py", line 1, in
from .corpus import Corpus # noqa: F401
File "C:Usersbrucesourcereposedith_functions.envlibsite-packagesspacytrainingcorpus.py", line 8, in
from .augment import dont_augment
File "C:Usersbrucesourcereposedith_functions.envlibsite-packagesspacytrainingaugment.py", line 10, in
from .example import Example
File "spacytrainingexample.pyx", line 10, in init spacy.training.example
File "C:Usersbrucesourcereposedith_functions.envlibsite-packagesspacytrainingalign.py", line 5, in
import tokenizations
File "C:Usersbrucesourcereposedith_functions.envlibsite-packagestokenizations__init__.py", line 2, in
from .tokenizations import (
ModuleNotFoundError: No module named 'tokenizations.tokenizations'

Your Environment

• Operating System: Windows 10
• Python Version Used: 3.9.0 64-bit
• spaCy Version Used: spacy-nightly-3.0.0rc2

[solarmist]

Doesn't work on Python 3.9 yet. Use 3.8.

See this issue
https://github.com/explosion/spaCy/issues/6357

[adrianeboyd]

Yes, using python <=3.8 is the right solution for now.

In case others come across this in the future:

We're dropping this dependency for the next spacy v3 rc release (will be v3.0.0rc3), so this particular problem shouldn't be an issue for the final v3 release.

We still have some outstanding issues related to other dependencies for python 3.9, so I'm not sure when we'll have python 3.9 wheels for v3 at this point, though.