Spacy: Cannot run tests on nightly

Created on 18 Aug 2020 · 8Comments · Source: explosion/spaCy

How to reproduce the behaviour

Download spacy nightly and try to run the tests after installing the requirements. Pytest gives the following complaint.

_______________________________ ERROR collecting spacy/tests/regression/test_issue5230.py ________________________________
spacy/tests/regression/test_issue5230.py:89: in <module>
    [nlp(), vectors(), custom_pipe(), tagger(), entity_linker()],
spacy/tests/regression/test_issue5230.py:61: in tagger
    tagger = nlp.add_pipe("tagger")
spacy/language.py:687: in add_pipe
    pipe_component = self.create_pipe(
spacy/language.py:585: in create_pipe
    resolved, filled = registry.resolve(cfg, validate=validate)
../../anaconda3/envs/project/lib/python3.8/site-packages/thinc/config.py:542: in resolve
    filled, _, resolved = cls._fill(
../../anaconda3/envs/project/lib/python3.8/site-packages/thinc/config.py:631: in _fill
    filled[key], validation[v_key], final[key] = cls._fill(
../../anaconda3/envs/project/lib/python3.8/site-packages/thinc/config.py:692: in _fill
    result = schema.parse_obj(validation)
pydantic/main.py:455: in pydantic.main.BaseModel.parse_obj
    ???
pydantic/main.py:344: in pydantic.main.BaseModel.__init__
    ???
pydantic/main.py:874: in pydantic.main.validate_model
    ???
E   pydantic.errors.ConfigError: field "nlp" not yet prepared so type is still a ForwardRef, you might need to call ArgModel.update_forward_refs().
================================================ short test summary info =================================================
ERROR spacy/tests/regression/test_issue5230.py - pydantic.errors.ConfigError: field "nlp" not yet prepared so type is s...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Your Environment

spaCy version: 3.0.0a7
Platform: Linux-4.15.0-112-generic-x86_64-with-glibc2.10
Python version: 3.8.5

➜  spaCy git:(5fdd6b460) pip show thinc
Name: thinc
Version: 8.0.0a27
Summary: A refreshing functional take on deep learning, compatible with your favorite libraries
Home-page: https://github.com/explosion/thinc
Author: Explosion
Author-email: [email protected]
License: MIT
Location: /home/user/anaconda3/envs/project/lib/python3.8/site-packages
Requires: numpy, murmurhash, srsly, setuptools, preshed, wasabi, pydantic, catalogue, cymem
Required-by: spacy-nightly

tests 🔜 v3.0

Source

forgetso

Most helpful comment

Thanks for the patience @svlandeg and @forgetso !

First, yeah, I have seen that error several times in other projects using Pydantic :grimacing: :sweat_smile:

Nevertheless, I started from a clean slate and I couldn't reproduce the error in spaCy :confused:

...But it took me a bit of work to actually start from a "clean slate", and I think the error could be coming from one of the several "caches" interacting in this.

@forgetso could you try this?:

// Make sure to pull the latest changes
$ git pull upstream develop

// Clean any built Cython modules/binaries
$ make clean

// Create a new venv (just to be sure)
$ python3.8 -m venv env-2020-10-15

// Activate it
$ source env-2020-10-15/bin/activate

// Update pip and the building requirements
$ python -m pip install --upgrade pip setuptools wheel

// Install the spaCy requirements, making sure to upgrade
// The upgrade here is important even in a new environment as pip could use its local cache and use an old version of some package
$ python -m pip install --upgrade -r requirements.txt

// Then install spaCy in that venv, that will trigger the Cython build, so it will take a bit
$ python -m pip install -e .

// Activate the venv again to make sure you are using the pytest from the venv
$ source env-2020-10-15/bin/activate

// Then run pytest again
$ pytest spacy/tests/regression

tiangolo on 15 Oct 2020

❤3

All 8 comments

That's strange, the current develop branch is testing just fine on my system, as well as on the CI for several open PR's ... Could you try pulling the latest?

svlandeg on 18 Aug 2020

I get the same thing when I test on develop - last commit c356e6290.

It seems like it's something to do with this and this.

➜  spaCy git:(c356e6290) pytest spacy/tests/regression
======================================== test session starts ========================================
platform linux -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/user/dev/spaCy, configfile: setup.cfg
plugins: timeout-1.4.2
collected 218 items / 1 error / 217 selected                                                        

============================================== ERRORS ===============================================
_____________________ ERROR collecting spacy/tests/regression/test_issue5230.py _____________________
spacy/tests/regression/test_issue5230.py:91: in <module>
    [nlp(), vectors(), custom_pipe(), tagger(), entity_linker()],
spacy/tests/regression/test_issue5230.py:62: in tagger
    tagger = nlp.add_pipe("tagger")
spacy/language.py:677: in add_pipe
    pipe_component = self.create_pipe(
spacy/language.py:575: in create_pipe
    resolved, filled = registry.resolve(cfg, validate=validate)
../../anaconda3/envs/project/lib/python3.8/site-packages/thinc/config.py:542: in resolve
    filled, _, resolved = cls._fill(
../../anaconda3/envs/project/lib/python3.8/site-packages/thinc/config.py:631: in _fill
    filled[key], validation[v_key], final[key] = cls._fill(
../../anaconda3/envs/project/lib/python3.8/site-packages/thinc/config.py:692: in _fill
    result = schema.parse_obj(validation)
pydantic/main.py:455: in pydantic.main.BaseModel.parse_obj
    ???
pydantic/main.py:344: in pydantic.main.BaseModel.__init__
    ???
pydantic/main.py:874: in pydantic.main.validate_model
    ???
E   pydantic.errors.ConfigError: field "nlp" not yet prepared so type is still a ForwardRef, you might need to call ArgModel.update_forward_refs().
====================================== short test summary info ======================================
ERROR spacy/tests/regression/test_issue5230.py - pydantic.errors.ConfigError: field "nlp" not yet ...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================= 1 error in 0.74s ==========================================

pip list
Package          Version             Location
---------------- ------------------- --------------------------
appdirs          1.4.4
attrs            19.3.0
backcall         0.2.0
black            19.10b0
blis             0.4.1
boto             2.49.0
boto3            1.14.38
botocore         1.17.38
catalogue        1.0.0
certifi          2020.6.20
chardet          3.0.4
click            7.1.2
cymem            2.0.3
Cython           3.0a6
decorator        4.4.2
docutils         0.15.2
en-core-web-lg   2.3.1
en-core-web-sm   2.3.1
flake8           3.5.0
future           0.18.2
gensim           3.8.3
idna             2.10
iniconfig        1.0.1
ipykernel        5.3.4
ipython          7.17.0
ipython-genutils 0.2.0
jedi             0.17.2
jellyfish        0.8.2
jmespath         0.10.0
joblib           0.16.0
jsonschema       3.0.2
jupyter-client   6.1.6
jupyter-core     4.6.3
lemminflect      0.2.1
mccabe           0.6.1
mock             2.0.0
more-itertools   8.4.0
murmurhash       1.0.2
nibabel          3.1.1
nilearn          0.6.2
nltk             3.5
numpy            1.19.1
packaging        20.4
pandas           1.1.0
parso            0.7.1
pathspec         0.8.0
pbr              5.4.5
pexpect          4.8.0
pickleshare      0.7.5
pip              20.1.1
plac             1.1.3
pluggy           0.13.1
preshed          3.0.2
prompt-toolkit   3.0.6
psutil           5.7.2
psycopg2         2.8.5
ptyprocess       0.6.0
py               1.9.0
pycodestyle      2.3.1
pycountry        20.7.3
pydantic         1.6.1
pyflakes         1.6.0
Pygments         2.6.1
pyparsing        2.4.7
pyrsistent       0.16.0
pytest           6.0.1
pytest-timeout   1.4.2
python-dateutil  2.8.1
pytokenizations  0.6.0
pytz             2020.1
pyzmq            19.0.2
regex            2020.7.14
requests         2.24.0
s3transfer       0.3.3
scikit-learn     0.23.2
scipy            1.5.2
setuptools       49.2.0.post20200714
six              1.15.0
sklearn          0.0
smart-open       2.1.0
srsly            2.2.0
symspellpy       6.5.2
thinc            8.0.0a27
threadpoolctl    2.1.0
toml             0.10.1
torch            1.6.0
tornado          6.0.4
tqdm             4.48.2
traitlets        4.3.3
typed-ast        1.4.1
typer            0.3.1
Unidecode        1.1.1
urllib3          1.25.9
wasabi           0.7.1
wcwidth          0.2.5
wheel            0.34.2

forgetso on 21 Aug 2020

@tiangolo: does this make any sense to you? The problem is I don't know where to start looking, because I can't replicate this locally. For me, the tests just run, and they've been running on spaCy's CI, too

svlandeg on 24 Sep 2020