Flair: How to improve the speed classifier.predict(sentence) sentiment analysis ?

[Rahulvks]

Describe the bug
classifier.predict(sentence) is taking lot of time for predicting the sentiment.

To Reproduce

def getPolarity(sen):
    try:
        sentence = Sentence(sen)
        classifier.predict(sentence)
        if sentence.labels[0].value=='NEGATIVE':
            score = -1*(sentence.labels[0].score)
        else :
            score = sentence.labels[0].score
        return  score 
    except Exception as e:
        return  "Error"

data['review_sentiment'] = data.Clean_Review.apply(lambda x: getPolarity(x))

• IOS:
  Kindly assist how can we make it faster ?

[alanakbik]

Hello @RVKRM the best way for speeding up processing is to use mini-batching, i.e. always send batches for Sentence objects through the classifier at once. We typically use mini-batches of 32 documents, but depending on your setup you might want to use fewer (8 or 16 depending on how long your documents are) or more (64 or 128 if you have a good GPU).

The.predict() method can consume a list of Sentence and has the parameter mini_batch_size, so you can set this parameter directly:

# your sentences
sentence_1 = Sentence("I love Berlin .")
sentence_2 = Sentence("I love New York .")
[...]
sentence_32 = Sentence("I love Paris .")

# make a batch (list) of sentences
batch = [sentence_1, sentence_2, ... , sentence_32]

# predict for the whole batch at once for speed ups
classifier.predict(sentence, mini_batch_size=8)

Hope this helps!

[pinkal08cece]

In real time, we can't wait for the next 31 sentences, we need to pass only one sentence then what happens like the question-answering application? There is required one more function which handles a single sentence and it is faster too.

Can you provide any idea for this?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.