Rasa: Timeout error on rasa shell

[tabergma]

Rasa version: 1.3.9

Python version: 3.6

Issue:
https://forum.rasa.com/t/timeout-error-on-rasa-shell/19638

Error can be reproduced on examples/restaurantbot. Add a time.sleep(15) to any action and run the bot.

Error (including full traceback):

2019-10-14 10:18:37 ERROR    asyncio  - Task exception was never retrieved
future: <Task finished coro=<configure_app.<locals>.run_cmdline_io() done, defined at /Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/rasa/core/run.py:123> exception=TimeoutError()>
Traceback (most recent call last):
  File "/Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/rasa/core/run.py", line 127, in run_cmdline_io
    server_url=constants.DEFAULT_SERVER_FORMAT.format("http", port)
  File "/Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/rasa/core/channels/console.py", line 138, in record_messages
    async for response in bot_responses:
  File "/Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/async_generator/_impl.py", line 366, in step
    return await ANextIter(self._it, start_fn, *args)
  File "/Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/async_generator/_impl.py", line 205, in throw
    return self._invoke(self._it.throw, type, value, traceback)
  File "/Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/async_generator/_impl.py", line 209, in _invoke
    result = fn(*args)
  File "/Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/rasa/core/channels/console.py", line 103, in send_message_receive_stream
    async for line in resp.content:
  File "/Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/aiohttp/streams.py", line 40, in __anext__
    rv = await self.read_func()
  File "/Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/aiohttp/streams.py", line 329, in readline
    await self._wait('readline')
  File "/Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/aiohttp/streams.py", line 297, in _wait
    await waiter
  File "/Users/tabergma/.virtualenv/rasa/lib/python3.6/site-packages/aiohttp/helpers.py", line 585, in __exit__
    raise asyncio.TimeoutError from None
concurrent.futures._base.TimeoutError

Command or request that led to error:

rasa shell

[ravikrcs]

same problem on
Rasa version : 1.5.1
python 3.6.8

[wochinge]

This is due to https://github.com/RasaHQ/rasa/blob/f950f7c734e2482fd49bd97fb5a955d0708a1287/rasa/core/channels/console.py#L101 . We are doing this to avoid freezing of the shell when the rasa server crashes in background.

For me the question is, why is the action running so long?

[ravikrcs]

Please check this thread, in which i am calling an action for http connection.

[Thien223]

It' because default time out (10 seconds) is not enough for your action to be executed.

Increasing the default timeout in "/rasa/core/channels/console.py" file, line 22 might help solving problem. Or you can check which makes the action taking time as @wochinge comment and optimize the executing time.

[ravikrcs]

Here is my action file that gives timeout error:

[wochinge]

@ravikrcs Can you mock the human handoff server for the rasa shell tests so it returns sth immediately?

[ravikrcs]

could you rephrase?

This is my api.py file

i am testing through Postman. But
It doesn't return any response.

[ravikrcs]

getting this error on rasa shell:

[wochinge]

could you rephrase?

I meant if you can replace the handoff interaction for testing the bot with something which returns an answer immediately? As a workaround you could also use the rest input channel for testing (in case you want to manually answer to the handoff requests)

[ravikrcs]

I meant if you can replace the handoff interaction for testing the bot with something which returns an answer immediately?

Yes @wochinge ,
other action class are working fine. only problem in human-handoff class that gives timeout error that means i am not getting response from api.

[ravikrcs]

i got this issue in human - handoff implementation. Below i attached screenshots.....

actions.py

api.py

action server:

flask server:

rasa shell:

[wochinge]

mhm, ok, then let's

• check if there is a better way to communicate errors in Rasa to the console connector than the timeout
• if not: make the timeout configurable

[mtrworld]

Is there any update or workaround for this issue?

[wochinge]

A workaround for now would be to not use rasa shell but e.g. Rasa X or the rest api

[rbasto1]

I'm having the same issue. We use rasa shell in development in order to have a lighter environment (just two containers needed, rasa-worker and the actions app).
When we load a new model the API call to PUT /model takes more than 10 seconds, and the console crashes.
Can you please add a config variable that overrides DEFAULT_STREAM_READING_TIMEOUT_IN_SECONDS?
Thanks!

[jutinguely]

I get the same issue here! Any update?

[wochinge]

@jutinguely This is still on our todo list. If you want, you can also pick it up? I'd be happy to give you feedback / support you on the way.

[TKueng]

Did you already do that? Because I could adjust the default stream time to 25 seconds in console.py file:
DEFAULT_STREAM_READING_TIMEOUT_IN_SECONDS=25
and I don't get a timeout.error anymore!

mhm, ok, then let's

• check if there is a better way to communicate errors in Rasa to the console connector than the timeout
• if not: make the timeout configurable

But my big question is can I adjust this default stream error time on RASA-X as well? @wochinge

Because I really need to share my bot for my Masterthesis asap and I think it's the last step, from preventing me to do so!

[wochinge]

But my big question is can I adjust this default stream error time on RASA-X as well? @wochinge

Rasa X communicates with Rasa Open Source in a different way, so you should be good

Did you already do that?

No, we haven't done it yet.

[nbeuchat]

Just bumped into the same issue with rasa==1.9.5. I just wanted to use the console locally for testing (we usually only use the Facebook channel).

I had to override DEFAULT_STREAM_READING_TIMEOUT_IN_SECONDS in the rasa code itself (which is ok locally but obviously not good).

It would be fantastic to be able to set DEFAULT_STREAM_READING_TIMEOUT_IN_SECONDS (or any other constant for that matter) through environment variable. I could have then just run:

DEFAULT_STREAM_READING_TIMEOUT_IN_SECONDS=30 rasa shell

[IgNoRaNt23]

Got a similar problem when I start a bot in rest-mode with rasa==1.9.5

Traceback (most recent call last):
  File "<local>/lib/python3.6/site-packages/rasa/core/processor.py", line 617, in _run_action
    events = await action.run(output_channel, nlg, tracker, self.domain)
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 129, in throw
    return self.gen.throw(type, value, traceback)
  File "<local>/lib/python3.6/site-packages/rasa/core/actions/action.py", line 549, in run
    json=json_body, method="post", timeout=DEFAULT_REQUEST_TIMEOUT
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 129, in throw
    return self.gen.throw(type, value, traceback)
  File "<local>/lib/python3.6/site-packages/rasa/utils/endpoints.py", line 146, in request
    **kwargs,
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 129, in throw
    return self.gen.throw(type, value, traceback)
  File "<local>/lib/python3.6/site-packages/aiohttp/client.py", line 1012, in __aenter__
    self._resp = await self._coro
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 129, in throw
    return self.gen.throw(type, value, traceback)
  File "<local>/lib/python3.6/site-packages/aiohttp/client.py", line 504, in _request
    await resp.start(conn)
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 129, in throw
    return self.gen.throw(type, value, traceback)
  File "<local>/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 847, in start
    message, payload = await self._protocol.read()  # type: ignore  # noqa
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 129, in throw
    return self.gen.throw(type, value, traceback)
  File "<local>/lib/python3.6/site-packages/aiohttp/streams.py", line 591, in read
    await self._waiter
concurrent.futures._base.CancelledError

My action calls another REST-service and does several retries when it fails. This may take some. I could of course change this, but it would be way more convenient to have a configurable timeout. And also a possibility to inform the user somehow, see https://github.com/RasaHQ/rasa/issues/5588

[wochinge]

I made a PR to fix this: https://github.com/RasaHQ/rasa/pull/5686

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed due to inactivity. Please create a new issue if you need more help.