Rasa: 403 client error when talking to the bot in command line

[alecigne]

Rasa Core version: 0.11.3 (same bug in all 0.11 versions, no bug in 0.10.4)

Python version: 3.6

Operating system (windows, osx, ...): Windows 7

Issue:

Hi,

After updating to Rasa Core 0.11, I am confronted with an error message in ALL my Rasa projects (including the Rasa Core Quickstart) when talking to the bot in the command line.

Steps:

1. Reproduce the Quickstart by copying and pasting.
2. Train the model (same command in Quickstart): python -m rasa_core.train -d domain.yml -s stories.md -o models/dialogue
3. Talk to the bot (same command in Quickstart): python -m rasa_core.run -d models/dialogue

Result:

2018-09-12 09:24:34 INFO     root  - Rasa process starting
2018-09-12 09:24:34 WARNING  py.warnings  - C:\Users\a.lecigne\.virtualenvs\rasa-quickstart-kxlx6eU3\lib\site-packages\pykwalify\core.py:99: UnsafeLoaderWarning:
The default 'Loader' for 'load(stream)' without further arguments can be unsafe.
Use 'load(stream, Loader=ruamel.yaml.Loader)' explicitly if that is OK.
Alternatively include the following in your code:

  import warnings
  warnings.simplefilter('ignore', ruamel.yaml.error.UnsafeLoaderWarning)

In most other cases you should consider using 'safe_load(stream)'
  data = yaml.load(stream)

2018-09-12 09:24:35 INFO     root  - Rasa Core server is up and running on http://localhost:5005
Bot loaded. Type a message and press enter (use '/stop' to exit):
/greet
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\a.lecigne\AppData\Local\Continuum\anaconda3\Lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\Users\a.lecigne\AppData\Local\Continuum\anaconda3\Lib\threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\a.lecigne\.virtualenvs\rasa-quickstart-kxlx6eU3\lib\site-packages\rasa_core\channels\console.py", line 110, in record_messages
    for response in bot_responses:
  File "C:\Users\a.lecigne\.virtualenvs\rasa-quickstart-kxlx6eU3\lib\site-packages\rasa_core\channels\console.py", line 69, in send_message_receive_stream
    r.raise_for_status()
  File "C:\Users\a.lecigne\.virtualenvs\rasa-quickstart-kxlx6eU3\lib\site-packages\requests\models.py", line 939, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://localhost:5005/webhooks/rest/webhook?stream=true&token=

The issue is gone when using the exact same project but downgrading to Rasa Core 0.10 and retraining the dialogue model.

Many thanks,

Anthony

[akelad]

I think I already replied in the forum post, but i'll ask here again: do you have any restrictions on running things on localhost?
Otherwise, maybe @tmbo has some more ideas of what could be going wrong

[tmbo]

Yes, this very much sounds like e.g. the windows firewall or some other firewall is not allowing you to connect to that server.

[tmbo]

Can you add --enable_api to your call to rasa_core.run and once started, try to open http://localhost:5005/ in your browser, what does happen?

[alecigne]

Hi,

Can you add --enable_api to your call to rasa_core.run and once started, try to open http://localhost:5005/ in your browser, what does happen?

The browser displays: hello from Rasa Core: 0.11.3 (but the command line bot still throws the same error).

The Windows firewall is configured to let me connect to that address (i.e. the virtualenv's Python executable is allowed to access all local addresses on all ports for the TCP and UDP protocols).

If relevant, a minimal Flask program in that project (thus using the same virtualenv's Python executable) runs well on that port:

from flask import Flask, request
from flask_cors import CORS, cross_origin

app = Flask(__name__)
CORS(app)

@app.route('/', methods=['GET'])
def main():
    return 'hey'


if __name__ == '__main__':
    app.run(port=5005)

The browser displays hey.

[tmbo]

I honestly don't know what the issue is. We are starting a normal webserver listening on port 5005 and then send web requests to that. I doubt that these forbidden responses come from the wwebser we spawned, at least I couldn't find any information about it in the flask documentation.

[alecigne]

Thank you for taking the time to look at it. I will investigate the issue and stick to Rasa Core 0.10 in the meantime.

[tmbo]

Ok, if you have any more insights let us know and I'd be happy to take another look

[REEDOOX]

Hello @alecigne ,

I have the same problem and the same problem as you, turned out the problem was with the with statement in console.py:

with requests.post(...) as r:


the with keywork somehow doesn't work as expected, am just a newbie in python but I think it's something lile using() in C#, but I doubt it has something to do with the yield at the end being asynchrounous (this might be non sense).

Whatever, I just edited the py file manually waiting for a fix and just used the result r as a variable directly:
r = requests.post(...)<br/> r.raise...

[tmbo]

mhm that is the normal way to use a streaming response in requests: http://docs.python-requests.org/en/master/user/advanced/#body-content-workflow

[REEDOOX]

You are right, but as I said I think the yield is what throwing the error here: check this stackoverflow thread, The solution I provided works for me but it is quite dirty.
UPDATE: This happens when the request to the action server fails. I had to remove the wait so I can take a look at the exception message from the action server and not only the http status code.

[alecigne]

Hi @REEDOOX and thank you for your comment.

Whatever, I just edited the py file manually waiting for a fix and just used the result r as a variable directly:
r = requests.post(...)<br/> r.raise...

If I understand you well, the send_message_receive_stream function in console.py looks like this after editing:

def send_message_receive_stream(server_url, auth_token, sender_id, message):
    payload = {
        "sender": sender_id,
        "message": message
    }

    r = requests.post("{}/webhooks/rest/webhook?stream=true&token={}".format(
        server_url, auth_token),
        json=payload,
        stream=True)

    r.raise_for_status()

    if r.encoding is None:
        r.encoding = 'utf-8'

    for line in r.iter_lines(decode_unicode=True):
        if line:
            yield json.loads(line)

If that's the case, then it doesn't fix the error on my side, which stays exactly the same.

Thank you,

Anthony

[tmbo]

@REEDOOX are you saying that it might be because requests to the action server fail?

[REEDOOX]

Yes. When the request to the action server fails it throws an unknown error with no stack trace. But once I removed the with keyword. It showed me the exception details. I thought this might be the problem with this issue but no. My bad.

[alecigne]

Update - the issue is solved when the http_proxy environment variable is unset. Thanks to ashukrishna100 on the Rasa community forums (link).

[tmbo]

Oh very interesting, glad you found the solution :+1: