Flair: AttributeError: 'DistilBertConfig' object has no attribute 'return_dict'

[Y4rd13]

Environment info

• Google Colab
• transformers version: 3.1.0
• Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
• Python version: 3.6.9
• PyTorch version (GPU?): 1.6.0+cu101 (False)
• Tensorflow version (GPU?): 2.3.0 (False)
• Using GPU in script?:
• Using distributed or parallel set-up in script?:
  

  Information

Model I am using (Bert, XLNet ...):

The problem arises when using:

• [ ] the official example scripts: (give details below)
• [ ] my own modified scripts: (give details below)

import flair
flair_sent = flair.models.TextClassifier.load('en-sentiment')

def flair_lstm(text):
    sentence = flair.data.Sentence(text)
    flair_sent.predict(sentences=sentence)
    total_sent = sentence.labels
    for label in total_sent:
        value = label.value
        score = label.score
        return '1' if value == 'POSITIVE' else '-1'

The tasks I am working on is:

• I'm working with flair to get classification polarities, but the issue seems refer to transformers

To reproduce

Steps to reproduce the behavior:

1. write

def flair_lstm(text):
    sentence = flair.data.Sentence(text)
    flair_sent.predict(sentences=sentence)
    total_sent = sentence.labels
    for label in total_sent:
        value = label.value
        score = label.score
        return '1' if value == 'POSITIVE' else '-1'

df_test = "some test dataframe"
df_test['flair'] = df_test['word'].apply(lambda x: flair_lstm(x))

1. See error:

Traceback:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-26-1ee39d7138b3> in <module>()
----> 1 df_test['flair'] = df_test['word'].apply(lambda x: flair_lstm(x))

10 frames
pandas/_libs/lib.pyx in pandas._libs.lib.map_infer()

/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py in use_return_dict(self)
    217         """
    218         # If torchscript is set, force `return_dict=False` to avoid jit errors
--> 219         return self.return_dict and not self.torchscript
    220 
    221     @property

AttributeError: 'DistilBertConfig' object has no attribute 'return_dict'

Expected output:

• prediction of each word of the dataframe using the flair_lstm function

[Y4rd13]

Answer from transformers repository: https://github.com/huggingface/transformers/issues/6891

[alanakbik]

It looks like the update to transformers 3.1.0 added new variables that are missing in our serialized models that we trained using 3.0. Just adding return_dict to the serialized model config does not seem to be enough, since there are new attributes like chunk_size_feed_forward that are now part of each FFN in the transformer.

A first workaround to fix the classifier is to do this:

# load the sentiment tagger and get the serialized ("old") embeddings
classifier = TextClassifier.load("sentiment")
serialized_embeddings = classifier.document_embeddings

# load a new version of distilbert transformer embeddings with the correct config
new_embeddings = TransformerDocumentEmbeddings("distilbert-base-uncased")

# transfer weights from serialized embedding to new embedding
new_embeddings.model.load_state_dict(serialized_embeddings.model.state_dict())

# set updated embedding on classifier
classifier.document_embeddings = new_embeddings

I'll take a closer look to see if there is a better fix.

[Y4rd13]

@alanakbik how this will change to the current classifier?

import flair
from flair.data import Sentence
from flair.models import TextClassifier

classifier = TextClassifier.load('sentiment')
sentence = Sentence(text)

classifier.predict(sentences=sentence)
result = sentence.labels
for label in result:
    value = label.value
    score = label.score

A: I just needed to update
!pip install --upgrade git+https://github.com/flairNLP/flair.git