Rasa: Rasa X 0.19.1 -- interactive learning broken?

[PhilippeBaert]

Rasa version: 1.1.3

Rasa X version: 0.19.1

Python version: 3.6.7

Operating system (windows, osx, ...): Windows 10

Issue:

Hello,

I updated Rasa X to 0.19.1. It has a lot of new cool features (automatically updating domain file is very nice!) but unfortunately I ran into a new problem:

When I talk to the bot in interactive learning mode, Core is not suggesting me any action.

For example:

I say hi: NLU detects the greet intent:

I click next, Rasa X is not proposing me any actions! Instead, it just runs actions_listen again:

The shell based interactive mode however, works perfectly:

Talking to the bot in Rasa X's talk mode, also works fine:

I didn't have this problem with Rasa X version 0.19.0.

A bug?

Cheers,
Philippe

[akelad]

@PhilippeBaert can you try installing 0.19.2? we just released that with a few more fixes

[PhilippeBaert]

@akelad Hi, I updated to 0.19.2 but I'm still experiencing the same issues...

Really strange!

[akelad]

that's strange, i'm running 0.19.2 on a server and interactive learning works fine. @imLew do you think you could try and reproduce this behaviour locally with rasa-x 0.19.2?

[PhilippeBaert]

@akelad, @imLew

Thanks again for your reply.

What I did (you can maybe reproduce this behavior more easily):

1. create a new directory
2. do a "rasa init" and train the models
3. launch Rasa x with "rasa x"
4. go to "Talk to your bot" => "Mode: interactive learning"
5. say "hello" and press enter
6. intent "greet" gets detected, I press enter again, no action gets proposed, the bot listens again

Of course the bot works perfect in shell or Rasa X "talk mode".

I tested this in both Chrome and Firefox.

[EDIT] I re-installed Rasa X v 0.19.0 and retried all the previous steps, and there were no issues ! [/EDIT]

Cheers and thanks for your help,
Philippe

[imLew]

Hi @PhilippeBaert,

unfortunately I cannot reproduce your problem. Using the the same versions of rasa and rasa-x on Ubuntu the interactive learning correctly predicts actions.
Have you tried reproducing it in a completely new environment with just rasa-x installed?

[PhilippeBaert]

Hi @imLew,

Thank you for looking into my issue.

I retried with a fresh environment, I still have the same issue.
When I try again with v 0.19.0, no issues.

I'm using Anaconda on Windows 10, I install Rasa X from the tar.gz file I download from https://pypi.rasa.com/simple/rasa-x/

Cheers,
Philippe

[imLew]

Just to check: did you retrain the bot at all? Is the model tagged as production in the models tab?

[PhilippeBaert]

@imLew, I tried re-training the bot (and tagging the new model as production) but it still doesn't work.

Really strange ... "rasa interactive" (in command line) works like a charm...

Could it have something to do with me running on Windows 10 and you running on Ubuntu?

[JEM-Mosig]

Ok, I can reproduce this on my Windows 10 laptop, following these points:

• download rasa-x-0-19-2.tar.gz from https://pypi.rasa.com/simple/rasa-x/
• open Anaconda Prompt
• conda create -n rasa-iss3802 python=3.6
• conda activate rasa-iss3802

• d:\Apps\Anaconda3\envs\rasa-iss3802\python.exe -m pip install rasa-x-0.19.2.tar.gz

• create a new directory

• do a "rasa init" and train the models
• launch Rasa x with "rasa x" (starts in the Brave browser)
• go to "Talk to your bot" => "Mode: interactive learning"
• say "hello" and press enter
• intent "greet" gets detected, I press enter again, no action gets proposed, the bot listens again

Looking into it...

[JEM-Mosig]

• 0.19.0 also works fine on my machine
• running rasa x --debug with 0.19.2 and looking into devtools/network/requests shows an error 500 (Failed to load resource: {"reasons":["Authorization header not present."],"exception":"Unauthorized"})
• On the Anaconda Console, rasa x --debug with 0.19.2 gives the following output when I repeat the procedure:

(rasa-iss3802) D:\rasa\support\iss3802\model>rasa x --debug
2019-06-20 11:01:53 DEBUG    rasa.cli.utils  - Parameter 'endpoints' not set. Using default location 'endpoints.yml' instead.
Starting Rasa X in local mode... 🚀
2019-06-20 11:01:54 DEBUG    sanic_jwt.configuration  - validating provided secret
2019-06-20 11:01:54 DEBUG    sanic_jwt.configuration  - validating keys (if needed)
2019-06-20 11:01:54 DEBUG    sanic_jwt.configuration  - loading secret and/or keys (if needed)
2019-06-20 11:02:06 DEBUG    rasa.cli.utils  - Parameter 'credentials' not set. Using default location 'credentials.yml' instead.
2019-06-20 11:02:06 DEBUG    rasa.core.utils  - Available web server routes:
/conversations/<conversation_id>/messages          POST                           add_message
/conversations/<conversation_id>/tracker/events    POST                           append_events
/webhooks/rasa                                     GET                            custom_webhook_RasaChatInput.health
/webhooks/rasa/webhook                             POST                           custom_webhook_RasaChatInput.receive
/webhooks/rest                                     GET                            custom_webhook_RestInput.health
/webhooks/rest/webhook                             POST                           custom_webhook_RestInput.receive
/model/test/intents                                POST                           evaluate_intents
/model/test/stories                                POST                           evaluate_stories
/conversations/<conversation_id>/execute           POST                           execute_action
/domain                                            GET                            get_domain
/                                                  GET                            hello
/model                                             PUT                            load_model
/model/parse                                       POST                           parse
/conversations/<conversation_id>/predict           POST                           predict
/conversations/<conversation_id>/tracker/events    PUT                            replace_events
/conversations/<conversation_id>/story             GET                            retrieve_story
/conversations/<conversation_id>/tracker           GET                            retrieve_tracker
/status                                            GET                            status
/model/predict                                     POST                           tracker_predict
/model/train                                       POST                           train
/model                                             DELETE                         unload_model
/version                                           GET                            version
2019-06-20 11:02:06 INFO     root  - Starting Rasa Core server on http://localhost:5005
2019-06-20 11:02:06 DEBUG    asyncio  - Using selector: SelectSelector
2019-06-20 11:02:06 INFO     root  - Enabling coroutine debugging. Loop id 2420073607856.
2019-06-20 11:02:06 DEBUG    root  - Could not load interpreter from 'None'
2019-06-20 11:02:06 DEBUG    rasa.core.tracker_store  - Attempting to connect to database via "sqlite:///tracker.db"
2019-06-20 11:02:06 DEBUG    rasa.core.tracker_store  - Connection to SQL database 'tracker.db' successful
2019-06-20 11:02:06 DEBUG    rasa.core.agent  - Requesting model from server http://localhost:5002/api/projects/default/models/tags/production...
2019-06-20 11:02:08 DEBUG    rasa.core.agent  - Tried to fetch model from server, but couldn't reach server. We'll retry later... Error: Cannot connect to host localhost:5002 ssl:None [Connect call failed ('127.0.0.1', 5002)].
2019-06-20 11:02:08 DEBUG    rasa.core.agent  - No new model found at URL http://localhost:5002/api/projects/default/models/tags/production
2019-06-20 11:02:10 DEBUG    rasa.core.agent  - Requesting model from server http://localhost:5002/api/projects/default/models/tags/production...

The server is running at http://localhost:5002/login?username=me&password=hB65OeuttXTZ

2019-06-20 11:02:12 DEBUG    rasa.core.agent  - Unzipped model to 'C:\Users\jm271\AppData\Local\Temp\tmpuyy8vx85'
2019-06-20 11:02:13 DEBUG    rasa.core.agent  - Found new model with fingerprint e0f7e931a9f56454a32474952e9dbb73. Loading...
2019-06-20 11:02:23 DEBUG    rasa.core.agent  - Finished updating agent to new model.
2019-06-20 11:02:23 DEBUG    rasa.core.agent  - Requesting model from server http://localhost:5002/api/projects/default/models/tags/production...
2019-06-20 11:02:24 DEBUG    rasa.core.agent  - Model server returned 204 status code, indicating that no new model is available. Current fingerprint: e0f7e931a9f56454a32474952e9dbb73
2019-06-20 11:02:24 DEBUG    rasa.core.agent  - No new model found at URL http://localhost:5002/api/projects/default/models/tags/production
2019-06-20 11:02:24 DEBUG    rasa.core.agent  - Requesting model from server http://localhost:5002/api/projects/default/models/tags/production...
2019-06-20 11:02:25 DEBUG    rasa.core.agent  - Model server returned 204 status code, indicating that no new model is available. Current fingerprint: e0f7e931a9f56454a32474952e9dbb73
2019-06-20 11:02:25 DEBUG    rasa.core.agent  - No new model found at URL http://localhost:5002/api/projects/default/models/tags/production
INFO:apscheduler.scheduler:Scheduler started
2019-06-20 11:02:26 DEBUG    rasa.core.agent  - Requesting model from server http://localhost:5002/api/projects/default/models/tags/production...
2019-06-20 11:02:27 DEBUG    rasa.core.agent  - Model server returned 204 status code, indicating that no new model is available. Current fingerprint: e0f7e931a9f56454a32474952e9dbb73
2019-06-20 11:02:27 DEBUG    rasa.core.agent  - No new model found at URL http://localhost:5002/api/projects/default/models/tags/production
2019-06-20 11:02:28 DEBUG    rasa.core.tracker_store  - Recreating tracker from sender id 'me'
2019-06-20 11:02:28 DEBUG    rasa.core.agent  - Requesting model from server http://localhost:5002/api/projects/default/models/tags/production...
2019-06-20 11:02:29 DEBUG    rasa.core.agent  - Model server returned 204 status code, indicating that no new model is available. Current fingerprint: e0f7e931a9f56454a32474952e9dbb73
2019-06-20 11:02:29 DEBUG    rasa.core.agent  - No new model found at URL http://localhost:5002/api/projects/default/models/tags/production
2019-06-20 11:02:30 DEBUG    rasa.core.agent  - Requesting model from server http://localhost:5002/api/projects/default/models/tags/production...
2019-06-20 11:02:31 DEBUG    rasa.core.agent  - Model server returned 204 status code, indicating that no new model is available. Current fingerprint: e0f7e931a9f56454a32474952e9dbb73
2019-06-20 11:02:31 DEBUG    rasa.core.agent  - No new model found at URL http://localhost:5002/api/projects/default/models/tags/production
2019-06-20 11:02:31 DEBUG    rasa.core.tracker_store  - Can't retrieve tracker matchingsender id '4e06cb95d25840b18b3d934fb17cb553' from SQL storage.  Returning `None` instead.
2019-06-20 11:02:31 DEBUG    rasa.core.tracker_store  - Tracker with sender_id '4e06cb95d25840b18b3d934fb17cb553' stored to database
...

The last statements keep repeating as long as the application is running.

[ricwo]

Hey @PhilippeBaert I just published a debug version of rasa-x which should print the exact exception occurring in the /domainActions endpoint. Would you mind installing 0.19.3.dev1+g2bde944 from https://pypi.rasa.com/simple/rasa-x/ and posting the command line output when performing interactive learning?

Here's a direct link to the .tar.gz: https://pypi.rasa.com/api/package/rasa-x/rasa-x-0.19.3.dev1+g2bde944.tar.gz

[PhilippeBaert]

Hi @ricwo ,

I'm getting the following output:

```Starting Rasa X in local mode... 🚀

The server is running at http://localhost:5002/login?username=me&password=U0vD9joXGer5

INFO:apscheduler.scheduler:Scheduler started
threw exception in /domainActions
Object of type 'set' is not JSON serializable
```

Cheers,
Philippe

[ricwo]

Thanks @PhilippeBaert - this was very helpful. The endpoint tries to serialise a set which it shouldn't. This serialisation seems to work anyway in some json.dumps() implementations, but fails on Windows. We'll fix this asap - I'll get back to you once it's released

[PhilippeBaert]

Hello @ricwo , without wanting to sound pushy, do you have any idea on when a fix will be implemented? :-)

Thanks!

Cheers,
Philippe

[ricwo]

@PhilippeBaert the fix has already been implemented, and will be part of the next release. We're working on it and it should be out in the next few days

[ricwo]

@PhilippeBaert 0.19.3 has been released today 🙂

[ricwo]

closing this as it's fixed as of 0.19.3

[cyrilthank]

can you please share where i may download 0.19.5 (or the latest recommended version) from
requesting since
pip install rasa-x --extra-index-url https://pypi.rasa.com/simple
still installed 0.19.0

[JEM-Mosig]

You can download all versions from https://pypi.rasa.com/simple/rasa-x/

[cyrilthank]

Thank you
but now on http://localhost:5008/api/version
i get the below can you please advise
{
"rasa": {
"production": "1.1.4",
"worker": "1.1.4"
},
"rasa-x": "UNDEFINED - this package is not installed"
}

[ricwo]

@cyrilthank the undefined rasa X version is a bug that occurs when rasa X has been installed in editable mode (pip install -e). It's already been fixed and will be part of the next release

[cyrilthank]

Thanks @ricwo can you please advise in this situation how i may confirm/check the version, please do let me know if there is another(maybe the fixed) version to install along with the command

[ricwo]

Hi @cyrilthank, you could try installing in non-editable mode, so just pip install https://pypi.rasa.com/api/package/rasa-x/rasa-x-0.19.5.tar.gz. In that case the version should be fetched correctly

[cyrilthank]

Thank you @ricwo