Spacy: NoneType uncallable on spacy.load('fr') (if self.is_base_form(univ_pos, morphology))

[bartvdbraak]

How to reproduce the behaviour

I am trying to get lemmatization/stemming to work with spaCy and spaCy_lefff in French. However, everytime I try to execute this code block in iPython kernel (jupyter notebook):

import spacy
nlp = spacy.load("fr")

doc = nlp("C'est une phrase.")
print([(w.text, w.pos_) for w in doc])

I am getting the following error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
 in 
      2 nlp = spacy.load("fr_core_news_sm")
      3 
----> 4 doc = nlp("C'est une phrase.")
      5 print([(w.text, w.pos_) for w in doc])

~/anaconda3/lib/python3.7/site-packages/spacy/language.py in __call__(self, text, disable, component_cfg)
    447             if not hasattr(proc, "__call__"):
    448                 raise ValueError(Errors.E003.format(component=type(proc), name=name))
--> 449             doc = proc(doc, **component_cfg.get(name, {}))
    450             if doc is None:
    451                 raise ValueError(Errors.E005.format(name=name))

pipes.pyx in spacy.pipeline.pipes.Tagger.__call__()

pipes.pyx in spacy.pipeline.pipes.Tagger.set_annotations()

morphology.pyx in spacy.morphology.Morphology.assign_tag_id()

morphology.pyx in spacy.morphology.Morphology.lemmatize()

~/anaconda3/lib/python3.7/site-packages/spacy/lang/fr/lemmatizer.py in __call__(self, string, univ_pos, morphology)
     47             return [self.lookup(string)]
     48         # See Issue #435 for example of where this logic is requied.
---> 49         if self.is_base_form(univ_pos, morphology):
     50             return list(set([string.lower()]))
     51         index_table = self.lookups.get_table("lemma_index", {})

TypeError: 'NoneType' object is not callable

I have downloaded with python -m spacy download fr and python -m spacy download fr_core_news_sm.

Your Environment

Info about spaCy

• spaCy version: 2.3.1
• Platform: Linux-5.3.0-62-generic-x86_64-with-debian-buster-sid
• Python version: 3.7.6

• Models: fr, en

• Environment Information: iPython kernel, Jupyter Notebook, Anaconda3

Extra

I have downloaded with python -m spacy download fr and python -m spacy download fr_core_news_sm.
Extra info:

❯ python -m spacy validate
✔ Loaded compatibility table

====================== Installed models (spaCy v2.3.1) ======================
ℹ spaCy installation:
/home/bart/anaconda3/lib/python3.7/site-packages/spacy

TYPE      NAME              MODEL             VERSION                            
package   fr-core-news-sm   fr_core_news_sm   2.3.0   ✔
package   en-core-web-sm    en_core_web_sm    2.3.1   ✔
link      fr                fr_core_news_sm   2.3.0   ✔
link      en                en_core_web_sm    2.3.1   ✔

❯ python -m spacy info

============================== Info about spaCy ==============================

spaCy version    2.3.1                         
Location         /home/bart/anaconda3/lib/python3.7/site-packages/spacy
Platform         Linux-5.3.0-62-generic-x86_64-with-debian-buster-sid
Python version   3.7.6                         
Models           fr, en

The english package does work however:

import spacy
nlp = spacy.load("en")

doc = nlp("Just a phrase.")
print([(w.text, w.pos_) for w in doc])

with output

[('Just', 'ADV'), ('a', 'DET'), ('phrase', 'NOUN'), ('.', 'PUNCT')]

[adrianeboyd]

Thanks for the report!

I can fix the bug so this code works as before, but I think the French lemmatizer is unintentionally using English-specific code here that doesn't make sense for French. Let me check the details first, but I think we can probably just remove this whole function.

[SabrinaWSY]

I'm having the exact same problem with the french model, whereas my script worked fine yesterday.
I tried all 3 french models and there was the same error message TypeError: 'NoneType' object is not callable

The English model works for me as well.
Is there an update on the french models today that is causing the problem ?

[adrianeboyd]

The bug is in spacy v2.3.1. If you downgrade to v2.3.0 it should work. The models haven't changed (and won't need to change, the bug is in spacy itself).

[bartvdbraak]

Thanks, I will downgrade to version 2.3.0 for the time being, and hopefully your fix will be in the newest version soon :).

[alexispunk]

Hello,
I am sorry, I just got the same issue today with the French package 2.3.0 today, while it works perfectly with English package 2.3.1 and Japanese package 2.3.2.

Typing:
import fr_core_news_sm
nlpf = fr_core_news_sm.load()
texte3= "J'aime le vin blanc. »
doc3= nlpf(texte3)

I get:

TypeError Traceback (most recent call last)
in
----> 1 doc3= nlpf(texte3)

~/opt/anaconda3/lib/python3.7/site-packages/spacy/language.py in __call__(self, text, disable, component_cfg)
447 if not hasattr(proc, "__call__"):
448 raise ValueError(Errors.E003.format(component=type(proc), name=name))
--> 449 doc = proc(doc, **component_cfg.get(name, {}))
450 if doc is None:
451 raise ValueError(Errors.E005.format(name=name))

pipes.pyx in spacy.pipeline.pipes.Tagger.__call__()

pipes.pyx in spacy.pipeline.pipes.Tagger.set_annotations()

morphology.pyx in spacy.morphology.Morphology.assign_tag_id()

morphology.pyx in spacy.morphology.Morphology.lemmatize()

~/opt/anaconda3/lib/python3.7/site-packages/spacy/lang/fr/lemmatizer.py in __call__(self, string, univ_pos, morphology)
47 return [self.lookup(string)]
48 # See Issue #435 for example of where this logic is requied.
---> 49 if self.is_base_form(univ_pos, morphology):
50 return list(set([string.lower()]))
51 index_table = self.lookups.get_table("lemma_index", {})

TypeError: 'NoneType' object is not callable

[adrianeboyd]

The problem is not the model version, it is the spacy version. Try downgrading to spacy==2.3.0 temporarily. We will have a fix in a new version of spacy soon, which should be 2.3.2.