Python-slack-sdk: Jupyter lab - "RuntimeError: Cannot run the event loop while another loop is running"

[ecatkins]

Bug Report

Reproducible in:

slackclient version: 2.0.1

python version: 3.7

OS version(s): MACOSX

Description

The code below, produces an error (see screenshot) when run in Jupyter Lab, and not when run from a simple python script. This comes after trying to upgrade slack client from a previous version, when I was successfully able to run from my Jupyter lab notebook

from slack import WebClient
sc = WebClient(token='<my-token>')
sc.chat_postMessage(channel='<my-channel>', text='testing2')

[RodneyU215]

Thanks for reporting this! I believe the reason you're seeing this error is due to the fact that Jupyter lab creates and runs the main event loop. Therefore all async functions (i.e. "API calls") must be scheduled for execution on the event loop instead of attempting to create a new loop in the same thread.

I'm currently working on a patch to resolve this. I'll update this once it's shipped.

[c-goosen]

Hi can confirm this in the sanic framework as well: https://github.com/huge-success/sanic.

I am looking into this as well. Could possibly pass the main event loop as a kwarg to slackClient?

[c-goosen]

@ecatkins && @RodneyU215 I think you can solve this the following way:

`
import asyncio
from slack import WebClient
loop = asyncio.get_event_loop()

sc = WebClient(token='')

sc.chat_postMessage(channel='', text='testing2', loop=loop, run_async=True)
`

This worked for me.

An addition in my code
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

I am using uvloop as the default event loop so I need to set it so that slackClient uses it too.

We could document this for users, otherwise a fix could be to default to using asyncio.get_event_loop()

[c-goosen]

Actually its not a full fix: ver.py:365> got Future > attached to a different loop

[c-goosen]

@RodneyU215 : https://github.com/slackapi/python-slackclient/pull/448

[RodneyU215]

@c-goosen Thanks for diving into this issue. Your troubleshooting helped guide me to an underlying issue where I attempted to create an event loop when there was one running already. I've since removed this code in #466 (specifically this code.).

@ecatkins I believe if you update to 2.1.0 it'll resolve your issues. One thing to keep in mind is that if Jupyter lab runs and manages the event loop then running the WebClient in async mode may be necessary. I'd try it without it first though.

from slack import WebClient
sc = WebClient(token='<my-token>', run_async=True)
post_my_message('<my-channel>', 'testing2')

async def post_my_message(channel, msg):
    await sc.chat_postMessage(channel=channel, text=msg)

I'll keep this issue open for a little while to confirm it's fixed.

[c-goosen]

@RodneyU215 no problem. Glad it was sorted out. Just a reminder on this specific issue, Jupyter notebook had to fix a similar issue itself, make sure you have the correct version of Jupyter installed as well @ecatkins

[ecatkins]

@c-goosen @RodneyU215 Thanks! That seems to work when running in aysnc mode

[vtrokhymenko]

so, webClient word at slack the last version?
because I get next error: <Task pending coro=<BaseClient._send() running at /Users/.../slack/web/base_client.py:182>>
doing with run_async=True

[jlmcgehee21]

I'm seeing this issue with latest release of the slack client (2.1) and Jupyter notebook. Stack trace below

~/code/work/very_gerbil/gerby/data/users.py in get_all_slack_users(self)
     26     def get_all_slack_users(self):
     27         # get all the users
---> 28         users_info = self.__s.api_call("users.list")
     29         # cram into dataframe
     30         df = pd.DataFrame(users_info["members"])

~/Library/Caches/pypoetry/virtualenvs/gerby-py3.7/lib/python3.7/site-packages/slack/web/base_client.py in api_call(self, api_method, http_verb, files, data, params, json)
    152             return future
    153 
--> 154         return self._event_loop.run_until_complete(future)
    155 
    156     def _validate_xoxp_token(self):

~/.asdf/installs/python/3.7.4/lib/python3.7/asyncio/base_events.py in run_until_complete(self, future)
    564         future.add_done_callback(_run_until_complete_cb)
    565         try:
--> 566             self.run_forever()
    567         except:
    568             if new_task and future.done() and not future.cancelled():

~/.asdf/installs/python/3.7.4/lib/python3.7/asyncio/base_events.py in run_forever(self)
    519         self._check_closed()
    520         if self.is_running():
--> 521             raise RuntimeError('This event loop is already running')
    522         if events._get_running_loop() is not None:
    523             raise RuntimeError(

RuntimeError: This event loop is already running

[stevenmanton]

I'm running into similar issues in Jupyter notebooks with no clear workaround. It seems that the default way to use the client (i.e. run_async=False) should completely sidestep any async operations. Instead, it seems like the client is trying to define all operations as async operations and then manage the loop itself when run_async=False. Unfortunately, there are all kinds of issues when a library tries to manage the loop itself because that often conflicts with parent packages that also manage the loop. (This is an annoying aspect of Python, I think.)

For what it's worth, I've run into this before and found that nest_asyncio handles this situation quite nicely. For packages that I manage that use asyncio I just stick nest_asyncio.apply() in the __init__.py and things mostly just work, including in notebooks.

Alternatively, for users who don't control the slack package, the following will likely work

import asyncio

import nest_asyncio
import slack

nest_asyncio.apply()

token = ...
client = slack.WebClient(token=token)

client.chat_postMessage(channel="channel", text="hello")

[dougan778]

I am also seeing this issue in 2.1. For me it has nothing to do with Jupyter but I think it's already known that this is more about the event loop handling than Jupyter itself.

I think it's because I'm starting an RTMClient before calling the API to post a message. I think the RTMClient is controlling the event loop in a weird way, but it's odd because the standard asyncio methods don't seen to know about it. I am not sure though.

[RodneyU215]

I appreciate everyone's feedback. I'm jumping back on this today!

[leorochael]

I can confirm @stevenmanton's workaround works for me:

Alternatively, for users who don't control the slack package, the following will likely work

import asyncio

import nest_asyncio
import slack

nest_asyncio.apply()

token = ...
client = slack.WebClient(token=token)

client.chat_postMessage(channel="channel", text="hello")

[c-goosen]

@dougan778 had the same issue.

Which API framework/library are you using?

It can help to pass a loop to slack first, then set the loop for the API.

Ive looked at nest_asyncio, but wasn't sure if I wanted to go down that route already.

[c-goosen]

@dougan778 had the same issue where I checked if slack connection was working before api start, for sanic solved it with before_server_start which passes a event loop in for use.

@app.listener('before_server_start')
async def setup_db(app, loop):
    app.db = await db_setup()

@app.listener('after_server_start')
async def notify_server_started(app, loop):
    print('Server successfully started!')

@app.listener('before_server_stop')
async def notify_server_stopping(app, loop):
    print('Server shutting down!')

@app.listener('after_server_stop')
async def close_db(app, loop):
    await app.db.close()

[dougan778]

@dougan778 had the same issue.

Which API framework/library are you using?

It can help to pass a loop to slack first, then set the loop for the API.

Ive looked at nest_asyncio, but wasn't sure if I wanted to go down that route already.

No frameworks of note here, just a very vanilla python script using the 2.1 slack library. The nest_asyncio route that was laid out by @stevenmanton does resolve the problem and is just fine in my case, but probably isn't going to be for everyone, depending on how their other code interacts with the event loop.

[RodneyU215]

I apologize it's taken me a while to get this resolved. In the initial design I've attempted to "help" folks that weren't familiar with Async by creating/managing the loop within the SDK, this was a mistake. This lack of context for folks has caused more confusion than I anticipated. I'm going to remove this run_async option in the next major version. This SDK is built on top of aiohttp which is an async library and I will be explicit about this going forward.

[TTTPOB]

（same problem here and I understand whats going on now

[rbahumi]

Hi @RodneyU215
I am also experiencing the same issue running in Jupyter Lab
python version 3.7.4
slackclient==2.5.0

Thanks for your help.

[juan-vg]

FYI: nest_async doesn't work if using uvloop

I'm currently trying to mix slackclient==2.5.0 RTM with rasa==1.8.0 (which uses uvloop as well), and it's becoming a big headache

Python 3.7.6

[seratch]

Let me share the updates on this issue. My pull request #662 is going to be the solution to this issue. The fix will be included in the next minor version - 2.6.0. The version will be out within two weeks. Refer to https://github.com/slackapi/python-slackclient/issues/476#issuecomment-620645518 for more info.

[seratch]

Let me close this issue now as #662 is resolving it.

[seratch]

👋 slackclient 2.6.0rc1 is out. The pre-release version contains fixes for your issue described here.
https://pypi.org/project/slackclient/2.6.0rc1/

One week later from now, we'll be releasing version 2.6.0 to PyPI.

If you have a chance, could you try the release candidate version out and let us know your feedback? Thank you very much for being patient with this issue.

[JohnPaton]

I just tried out 2.6.0rc1 from both Jupyter and the python interpreter and it works as expected! Great work

[adir1661]

my version is 2.9.3 ? and I am having the same issue... when trying to run code that uses asyncio in my jupyter-lab... any solution yet?