Rasa: Option to run nlu test against intents in e2e stories

Created on 5 May 2020  路  5Comments  路  Source: RasaHQ/rasa

Description of Problem:
When I run e2e tests and there errors associated with the nlu, I'd like to see the nlu confidence scores but the generated intent related results files (intent_report.json, intent_success.json, etc) appear to be the results from a cross validation test.

If I want to see the confidence levels for the utternaces in my e2e stories, I have to pull them out into a separate NLU test file and run the NLU test.

Overview of the Solution:
Add an option to run the NLU tests against the utterances in the e2e test file or add those utterances automatically to the list of selected cross-validation utterances.

Definition of Done:

  • [ ] Tests are added
  • [ ] Feature described the docs
  • [ ] Feature mentioned in the changlog
area normal type

All 5 comments

@rgstephens make sure to put them on a sprint board, otherwise no one will see them (i've done it this time)

@federicotdn @degiz Would appreciate input on priority / whether this is a loop thing? 馃檹

That doesn't seem to be a request for the @RasaHQ/loop-engineers squad IMO.

@RasaHQ/production-squad does it fit to your board? If no, then it's research squad 馃槃

Will put it in our board. However, I think it makes sense to do this after 2.0

Closing this as duplicate of #5817

Was this page helpful?
0 / 5 - 0 ratings