Rasa: Warn for invalid keys for classifiers

[melindaloubser1]

Rasa version: 2.0.0

Issue:

If you pass an invalid key to a classifier no warning or error is thrown during training

Command or request that led to lack of error:

rasa train

Content of configuration file (config.yml) (if relevant):

pipeline
- name: FallbackClassifier
   confidence_threshold: 0.7

(should be threshold)

[rgstephens]

Here's a similar issue with responses. button_type is not valid at this location, but validate and train don't throw an error.

  utter_ask_feedback:
  - buttons:
    - button_type: inline
    - payload: /feedback{"feedback_value":"positive"}
      title: 👍
    - payload: /feedback{"feedback_value":"negative"}
      title: 👎
    text: How is this conversation going?

[melindaloubser1]

The trouble with responses is depending on the channel, those keys may be valid or not. With built in classifiers, the possible keys are known.

[wochinge]

would be great to find a general solution which also works for other NLU components than classifiers

[melindaloubser1]

@wochinge one way would be if each component class had an options or settings property that was a list of legitimate keys for that component. Then it could be validated like the name property is validated. wdyt?

[wochinge]

The defaults should already give us that information 🤔

[melindaloubser1]

Do all possible options have defaults?

[melindaloubser1]

looks like it. I'll use that then

[wochinge]

wait - so you're working on this?

[melindaloubser1]

I'm about to unless someone else started already?

[wochinge]

Fine for me - it's just for the 2.1 release plan 🚀