Rasa: Automatically migrate MappingPolicy + FallbackPolicies

[wochinge]

This issue may be split in three parts

Motivation

• this feedback
• there is actually no need to manually migrate FallbackPolicy, TwoStageFallbackPolicy or MappingPolicy. We can migrate all of them automatically.

Ideas

• check out the migration guide and automate it as part of some CLI command (rasa data convert?)
• ?

Definition of Done

• [ ] Feature implemented
• [ ] Migration guide updated

[wochinge]

@degiz Do you think we can integrate it into rasa data convert? There is no rasa migrate, right?

[wochinge]

It might not be possible to migrate the TwoStageFallbackPolicy automatically as users might have customized the deny_suggestion_intent_name intent and we can't migrate custom actions of theirs which are using this intent.

[degiz]

There is no rasa migrate, right?

We were talking about possibly introducing rasa migrate, but wanted to use it to migrate the whole project from to 2.0. So back then, when we only wanted to convert training data, we didn't do that.

Now, if you have an idea of how to do that, that makes sense! But Do you think we can do that before next week?

[wochinge]

I mean it's just for these polices above. We can't migrate the whole thing (e.g. forms).
The alternative based on the feedback would be to add a default rule for the two-stage-fallback but I don't think this is ideal as it's not transparent and users might wanna change the rule for the two stage fallback somehow. (+ it doesn't cover MappingPolicy etc.)

We could also do a rasa data convert config which would be a bit more selective than introducing a rasa migrate.

But Do you think we can do that before next week?

It's blocking feedback and directly relevant to our okrs (smooth transition). I think it's definitely doable with a bit of force.

cc @m-vdb @tmbo Wdyt?

[m-vdb]

It might not be possible to migrate the TwoStageFallbackPolicy automatically as users might have customized the deny_suggestion_intent_name intent and we can't migrate custom actions of theirs which are using this intent.

If we can get a command that simplifies the migration for a good share of users that's great. What about something along those lines:

1. show a very visible warning about the fact that the rule is missing (can we detect it?)
2. have a command that converts the config to 2.0 format
3. have users run this command themselves (it is not run automatically)

That way it's not _too_ smart and doesn't try to automatically migrate. But it's still simple enough for users.

I am not sure it's a good idea to name it migrate because the command will currently only migrate one thing (whereas users could expect migrate to migrate everything).

Regarding prioritization: definitely in line with this quarter prios. I'm afraid we won't have time to ship it but we could try, and worst case ship it in a patch right after (2.0.1)? But I think we do not have other alternative at this point, right?

[wochinge]

show a very visible warning about the fact that the rule is missing (can we detect it?)

I think we can't reliable detect it. What we could do is a check to see if the FallbackClassifier and a story / rule handles the nlu_fallback intent. It wouldn't be great performance wise as we'd have to go through O(n) stories (worst case scenario) to validate this.

I am not sure it's a good idea to name it migrate because the command will currently only migrate one thing (whereas users could expect migrate to migrate everything).

This could also integrate the training data conversion. However, time and implementation wise a rasa data convert config is quicker as we can just focus on making one thing work.

I'm afraid we won't have time to ship it but we could try

I'll think about it and then give an estimate. I also think we have no alternative 👍 I think we could split this issue btw.

[m-vdb]

@wochinge if you have a short implementation plan and we can split it, i'm happy to help

[wochinge]

On it 👍

[wochinge]

@m-vdb @ricwo @alwx Did an implementation proposal .

It's splittable in 4 tasks, but probably doesn't make sense that more than 2 people work on it. I think it's realistic that we can get out something working by tomorrow evening and then review on Friday and merge?

[wochinge]

should we a parameter for the version which is migrated to the command? I'd say we can skip this for now and then add it in later versions

[m-vdb]

works for me

[wochinge]

@Ghostvv @ricwo @m-vdb @b-quachtran
We currently have this check to ensure that the RulePolicy cannot be used in conjunction with MappingPolicy, FallbackPolicy or TwoStageFallbackPolicy. I'd vote for relaxing this and converting this to a warning. The problem is that with the current check it's not possible to migrate a 1.x bot step by step. As a user I'd personally would have the following workflow:

1. Migrate one policy
2. Train
3. Test
4. Migrate next policy

This is currently not possible due to our check. This makes it hard for users to find out where in the process of the migration a potential error happened. What do you think?

[Ghostvv]

@wochinge I think, it should be ok, then you need to create new highest priority to make sure RulePolicy overrides all other policies. Also, I'd be a bit worried that users wouldn't finish migration, but rather start to pile up different policies

[b-quachtran]

I like the idea of relaxing it to a warning, at least for 2.0, while users are migrating from 1.X.