Rasa: Rules: false contradiction with active_loop: null condition

[NetBKarine]

Rasa version: 2.1.0

Rasa SDK version (if used & relevant):2.0.0

Rasa X version (if used & relevant): not used

Python version: 3.7.9

Operating system (windows, osx, ...): Ubuntu 18.04

Issue:
Training fails when there is both a rule with one condition on active_loop: null and a rule with similar steps but no conditions.

I want to have a different behaviour for a simple query when there is an active form and when there is not, so I tried:

Content of data file (rules.yml):

version: "2.0"

...

rules:
  - rule: List merchants
    condition:
      - active_loop: null
    steps:
      - intent: list_merchants
      - action: action_list_merchants
      - action: utter_ask_main_menu.reentry.initial

  - rule: List merchants as form digression
    steps:
      - intent: list_merchants
      - action: action_list_merchants
...

I got:

Error (including full traceback):

InvalidRule:
Contradicting rules or stories found 🚨

- the prediction of the action 'action_listen' in rule 'List merchants as form digression' is contradicting with rule(s) 'List merchants' which predicted action 'utter_ask_main_menu.reentry.initial'.
Please update your stories and rules so that they don't contradict each other.
You can find more information about the usage of rules at https://rasa.com/docs/rasa/rules.

Command or request that led to error:

rasa train

Content of configuration file (config.yml) (if relevant):

policies:
  ...
  - name: RulePolicy
    core_fallback_threshold: 0.3
    core_fallback_action_name: action_default_fallback
    check_for_contradictions: true

However, when switching off check_for_contradictions, training passes and the model differentiates the rules; the first one applies when there is no active form/loop, and the second one the rest of the time, as expected. The check works fine if the condition has the name of a form instead of null. Of course there are other solutions, like listing every form possible in a different rule, or using stories, but the error returned is still incorrect, as those two rules are not in contradiction with one another.

[sara-tagger]

Thanks for the issue, @degiz will get back to you about it soon!

You may find help in the docs and the forum, too 🤗

[nbeuchat]

My understanding of your second rule is that it should apply _anytime_, not just when there is an active form.

I have not tried it but maybe that works? I saw in another issue someone doing something like that for a slot.

  - rule: List merchants as form digression
    condition:
      - active_loop
    steps:
      - intent: list_merchants
      - action: action_list_merchants

[NetBKarine]

My understanding of your second rule is that it should apply _anytime_, not just when there is an active form.

I have not tried it but maybe that works? I saw in another issue someone doing something like that for a slot.

  - rule: List merchants as form digression
    condition:
      - active_loop
    steps:
      - intent: list_merchants
      - action: action_list_merchants

@nbeuchat What you saw was for text slots, since the actual value doesn't matter. I did try it even so (you never know), but it doesn't work, active_loop has to have a value:

YamlValidationException: Failed to validate 'data/rules.yml'. Please make sure the file is correct and all mandatory parameters are specified. Here are the errors found during validation:
  in data/rules.yml:88:
      Value 'active_loop' is not a dict. Value path: '/rules/7/condition/0'

It could be useful though... In my use case the precedence of the first rule causes the second one to apply only when there is a form, and it works because I only have two scenarios: in a form or not, with one different step only. If there were more conditions or a bigger difference, it might not work as well, or be messier and error-prone.

[nbeuchat]

Arf, that's too bad! I have the same use case as you have, hope there is or will be a way to have a condition for the active loop being anything but null

[Ghostvv]

hope there is or will be a way to have a condition for the active loop being anything but null

you can put active_loop: some_form in condition. But it needs to be a concrete form, it cannot be arbitrary

[Ghostvv]

@NetBKarine why do you think the contradiction is false? Your rules are indeed contradicting

[Ghostvv]

the problem is that RulePolicy wouldn't know what to predict after action_list_merchants if no active_loop is set

[nbeuchat]

you can put active_loop: some_form in condition. But it needs to be a concrete form, it cannot be arbitrary

I think the point is that it would be great if it could be arbitrary (ie: the rule should apply if there is _any_ active loop set). For example, you could have an intent that would always do the same thing and deactivate the form if you are within a form (irrespective of which form we are talking about); if you have five forms, you need to write five times the same rules just to do that. It doesn't sound very DRY

[NetBKarine]

@NetBKarine why do you think the contradiction is false? Your rules are indeed contradicting

@Ghostvv They are not in contradiction as the first one is more precise than the second and will only match if there is no active loop, whereas the second one could match anytime (if the first one was not there). Of course, the second rule alone makes no sense, but that doesn't mean they are contradictory.

And again, it would be cleaner to have a way to make a rule matching any active loop as mentioned by @nbeuchat than to have to do this; maybe this issue can be turned into a feature request instead, but I thought it would be simpler to allow this way, which is technically correct even though not very clean, than to create something to match any form.

[Ghostvv]

the first one is more precise than the second and will only match if there is no active loop

the problem is that theoretically the second one will also match if there is no active loop, it works only due to the fact that we pick a longer rule.
I agree that in practice they are not contradicting, however from looking at them they're contradicting (because they're both applied in the same situations). For these rule snippets, the goal is to be as explicit as possible to avoid standard confusions that state machine based dialogue system suffer.

it would be cleaner to have a way to make a rule matching any active loop

having no value to be interpreted as any rule seems to be a good idea, however, I'm not sure how confusing will it be for users.

  - rule: List merchants as form digression
    condition:
      - active_loop
    steps:
      - intent: list_merchants
      - action: action_list_merchants

@akelad what do you think about the rule above?

[akelad]

I think we should allow for this option, but we should maybe phrase these conditions differently. E.g. active_loops: any, active_loops: loop_1, active_loops: loop_1, loop_2. Something like that?

[nbeuchat]

I think we should allow for this option, but we should maybe phrase these conditions differently. E.g. active_loops: any, active_loops: loop_1, active_loops: loop_1, loop_2. Something like that?

It would be great to have the same syntax for slots too (especially for non-text slots like float, boolean, etc.)!

[akelad]

@nbeuchat can I ask what you mean by that? The condition already supports slot set events I believe

[nbeuchat]

@nbeuchat can I ask what you mean by that? The condition already supports slot set events I believe

What I'm referring to is the syntax with the any keyword. If my understanding is correct, you cannot have a condition "the slot was set to any value except null" for slots like float, bool, custom, etc. Isn't it? It might be a misunderstanding on my side but my feeling is that if you could have:

condition:
- slot_was_set:
  - my_float_slot: any

It would be very explicit that this slot is set to something but the value does not matter.

[akelad]

ah right, you are correct, thanks for explaining!

[Ghostvv]

@nbeuchat we cannot do it for slots like categorical, bool, etc. Because the value of these slots affect their features, so we need the value to create training data.

@akelad any for these slots then becomes effectively an or for slots 😉

[akelad]

i need github to add a 🙃 reaction please