Rasa: Run training fingerprinting via the CLI

[b-quachtran]

Description of Problem:

Request to be able to create fingerprint.json file via the CLI without having to run rasa train. Feature will make it easier to run CI checks against Rasa assistant code changes.

Overview of the Solution:

Exposing a CLI command to generate fingerprint.json, something along the lines of rasa fingerprint or rasa train --fingerprint

Examples (if relevant):

Blockers (if relevant):

Definition of Done:

[TyDunn]

@tmbo Is this something we could include in 2.0? A customer has been asking for this for a while, and I think something like it makes sense. The workflow for checking if a model needs to be trained is cumbersome right now because you have to diff check the config, training data, domain file, and Rasa version today

[tmbo]

yes I think this should be easy enough to add and I'd agree this does seem useful.

[m-vdb]

decided with Ty to add this to 2.1 milestone, as it's a new feature and shouldn't be release blocking for 2.0

[b-quachtran]

@TyDunn Is there any update on this? The same customer is asking about this feature again, but it looks like it's been pushed to the 2.2 milestone.

[TyDunn]

@b-quachtran There were quite a bit of improvements we needed to make to smoothen out friction caused by 2.0, so we decided to do that in 2.1 before we adding new functionality in 2.2

[wochinge]

@b-quachtran How about something like rasa train --dry-run which would give you a different status code in case the model needs re-training?

[b-quachtran]

@wochinge Yes, that would be exactly what they're looking for 😄

[wochinge]

Ok, so they have access to previously trained models when running rasa train --dry-run?

[b-quachtran]

Yeah, they have previously trained models stored in S3 and are able to download that for dry runs.