Rasa: Run evaluation fails every time

[nadabou]

Rasa NLU version:
0.14.3
Operating system (windows, osx, ...):
Windows 10
Content of model configuration file:

language: "en"

pipeline:
- name: "nlp_spacy" 
- name: "tokenizer_spacy" 
- name: "intent_featurizer_spacy" 
- name: "intent_classifier_sklearn" 

Issue:
After training my model and parsing utterances with it, I try to evaluate it using:

from rasa_nlu.evaluate import run_evaluation
run_evaluation("nlu.md", model=model_directory)

Where nlu.md contains my test data and model_directory is the path to my model.
Evaluation starts and then fails every time with the same error:
TypeError: classification_report() got an unexpected keyword argument 'output_dict'

[akelad]

Thanks for raising this issue, @erohmensing will get back to you about it soon.

[erohmensing]

Hi @nadabou, what version of scikit-learn are you running? It looks like the output_dict parameter of classification_report() didn't show up until 0.20.0. The scikit-learn requirement for rasa nlu is 0.20.2, so if you're not running that version, I'd recommend reinstalling the requirements.

[nadabou]

Hi @erohmensing, thank you for the answer. It seems I am running version '0.19.2' of sklearn. I'll try upgrading to 0.20.2 without reinstalling all requirements as it might mess up my environment.

Edit: Problem was solved by upgrading to 0.20.2 ! Thank you @erohmensing !

[erohmensing]

Glad it helped! Just letting you know, you might run into other issues if other requirements are out of date -- I'd recommend creating a virtual environment for your rasa project so you can have the correct requirements without messing up your default. 🙂