Python-slack-sdk: Intermittent concurrent.futures._base.TimeoutError with run_async=True

[smslack]

Description

When i try to post a message to slack channel using one of the basic example to post a message to a channel, i see an error saying "concurrent.futures._base.TimeoutError". I am using python 3.6.7 on Ubuntu 18.0.4 with slackclient 2.

What type of issue is this? (place an x in one of the [ ])

• [x] bug
• [ ] enhancement (feature request)
• [x] question
• [ ] documentation related
• [ ] testing related
• [ ] discussion

Requirements (place an x in each of the [ ])

• [x] I've read and understood the Contributing guidelines and have done my best effort to follow them.
• [x] I've read and agree to the Code of Conduct.
• [x] I've searched for any related issues and avoided creating a duplicate issue.

---

Bug Report

Filling out the following details about bugs will help us solve your issue sooner.

Reproducible in:

slackclient version: 2

python version: 3.6.7

OS version(s): Ubuntu 18.0.4

Steps to reproduce:

1. Set up virtualenv with python 3.6.7
2. Example code from tutorial
3. python3 testnotify.py

My Code:

import slack

client = slack.WebClient(token='SLACK_API_TOKEN')

response = client.chat_postMessage(
    channel='#sm-test',
    text="Hello world!")
assert response["ok"]
assert response["message"]["text"] == "Hello world!"

Expected result:

I am expecting to see the message 'Hello World!' posted to the slack channel '#sm-test'

Actual result:

I am seeing the following Error:

Traceback (most recent call last):
  File "testnotify.py", line 9, in <module>
    text="Hello world!")
  File "/usr/local/lib/python3.6/dist-packages/slack/web/client.py", line 332, in chat_postMessage
    return self.api_call("chat.postMessage", json=kwargs)
  File "/usr/local/lib/python3.6/dist-packages/slack/web/base_client.py", line 154, in api_call
    return self._event_loop.run_until_complete(future)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 473, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.6/dist-packages/slack/web/base_client.py", line 211, in _send
    http_verb=http_verb, api_url=api_url, req_args=req_args
  File "/usr/local/lib/python3.6/dist-packages/slack/web/base_client.py", line 240, in _request
    async with session.request(http_verb, api_url, **req_args) as res:
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/client.py", line 1005, in __aenter__
    self._resp = await self._coro
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/client.py", line 575, in _request
    break
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/helpers.py", line 585, in __exit__
    raise asyncio.TimeoutError from None
concurrent.futures._base.TimeoutError

Attachments:

Logs, screenshots, screencast, sample project, funny gif, etc.

[Marc-Girard]

I have the same error messages, been looking for a fix for the last two days, no success.

Anyone else heard/knows about this? Mine are trigged by "dialog_open" and "conversations_info" but since smslack have the same behaviour with "chat_postMessage".

[Marc-Girard]

OK, I've got random concurrent.futures._base.TimeoutError from many scripts I wrote in the last year. I've ported everything to the Slack API 2.1 (using Python 3.6) and I got no luck. Is there any development on this? Thanks in advance.

[Marc-Girard]

Hi Slack Dev People,

Would it be possible to have an update on this issue? Thanks.

[devonmurphy]

I am running into this issue randomly also

[RodneyU215]

@Marc-Girard I apologize it's taken me so long to get to this issue. Can you confirm if you're using the RTMClient at all? Or are you seeing this issue only when working with the WebClient?

If you're following the tutorial which uses the RTMClient, I believe there's a bug in how the current threading handles timeouts. I'm currently working on an update to remove this. In the meantime, you can take a look at this async example to run the app without this threading logic.

[Marc-Girard]

@Marc-Girard I apologize it's taken me so long to get to this issue. Can you confirm if you're using the RTMClient at all? Or are you seeing this issue only when working with the WebClient?

I'm using the WebClient exclusively. So yes, I'm experiencing those issues with WebClient.

[HSDen]

@Marc-Girard HI, I am facing the same issue as well.
import slack

client = slack.WebClient(token='SLACK_API_TOKEN')
response = client.chat_postMessage(
channel='#mychannel',
text="Hello world!")
assert response["ok"]
assert response["message"]["text"] == "Hello world!"

version:
Python: 3.7.x
slack: 2.3.1
websocket-client: 0.54.0

Please help, as I am not able to proceed on further development. TIA.

[ram373737]

Hi Slack Dev Team,
Any update on this issue.
Please help me, Looking for this issue to be fixed

[pybala]

Same issue, irrespective of whether run_async true or not.. hitting the same error.
Going back to rest api and handle from my end.

[hbandlamudi]

Any update on this issue? I've started experiencing this behavior for the auth_test() API call.

[HSDen]

Downgrade the version it worked for me. I forgot the library think its
webclient if am not wrong.

On Tue, Feb 18, 2020 at 7:43 PM hbandlamudi [e-mail]
wrote:

Any update on this issue? I've started experiencing this behavior for the
auth_test() API call.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/slackapi/python-slackclient/issues/476?email_source=notifications&email_token=ABUXQFEFQSISCBPQQKVWWYLRDR6BVA5CNFSM4H7AWLBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMF5BWQ#issuecomment-587976922,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABUXQFGMAW7J3IXSKJZHM33RDR6BVANCNFSM4H7AWLBA
.

>

Thanks&Regards, Hari,Gnanaprakasam Ph#: 804-292-4894

[hbandlamudi]

Downgrade the version it worked for me. I forgot the library think its webclient if am not wrong.
On Tue, Feb 18, 2020 at 7:43 PM hbandlamudi @.*> wrote: Any update on this issue? I've started experiencing this behavior for the auth_test() API call. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#476?email_source=notifications&email_token=ABUXQFEFQSISCBPQQKVWWYLRDR6BVA5CNFSM4H7AWLBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMF5BWQ#issuecomment-587976922>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUXQFGMAW7J3IXSKJZHM33RDR6BVANCNFSM4H7AWLBA .
-- Thanks&Regards, Hari,Gnanaprakasam Ph#: 804-292-4894

@HSDen Do you remember which version you downgraded to?

[hbandlamudi]

@HSDen @RodneyU215 I downgraded to 2.4.0 but still see the error occurring.

[seratch]

I tried to reproduce this issue but I haven't managed to see the error yet. I'm wondering if this issue may be related to a specific version of aiohttp library (and possibly its dependencies). If someone who encountered this issue could share pip freeze output, that may be helpful.

[jeznag]

aiohttp==4.0.0a1
async-timeout==3.0.1
attrs==19.3.0
certifi==2020.4.5.1
chardet==3.0.4
idna==2.9
multidict==4.7.5
psycopg2-binary==2.8.5
quotes==0.0.3
requests==2.23.0
slackclient==2.5.0
tabulate==0.8.7
typing-extensions==3.7.4.2
urllib3==1.25.9
yarl==1.4.2

[seratch]

aiohttp==4.0.0a1

Sorry for the confusion but this library doesn't work with the alpha version of aiohttp. From the next release coming soon, the install_requires will be more specific ("aiohttp>3.5.2,<4.0.0"). For now, could you specifically install aiohttp 3.6.2 instead?

[jhofeditz]

just noticed this error in webclient conversations_replies on Python 3.6.9

aiodns                   2.0.0
aiohttp                  3.6.2
async-timeout            3.0.1
requests                 2.22.0
slackclient              2.5.0

[Shiwoo-Park]

I get this error too

package versions

aiohttp                        3.6.2
slackclient                    2.5.0

error stack

File "/home/deploy/git/exid/core/utils/slack_util.py", line 86, in send_message
    response = client.chat_postMessage(channel=channel, text=message)
  File "/home/deploy/.pyenv/versions/3.7.6/envs/py37/lib/python3.7/site-packages/slack/web/client.py", line 559, in chat_postMessage
    return self.api_call("chat.postMessage", json=kwargs)
  File "/home/deploy/.pyenv/versions/3.7.6/envs/py37/lib/python3.7/site-packages/slack/web/base_client.py", line 171, in api_call
    return self._event_loop.run_until_complete(future)
  File "/home/deploy/.pyenv/versions/3.7.6/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
    return future.result()
  File "/home/deploy/.pyenv/versions/3.7.6/envs/py37/lib/python3.7/site-packages/slack/web/base_client.py", line 214, in _send
    http_verb=http_verb, api_url=api_url, req_args=req_args
  File "/home/deploy/.pyenv/versions/3.7.6/envs/py37/lib/python3.7/site-packages/slack/web/base_client.py", line 244, in _request
    async with session.request(http_verb, api_url, **req_args) as res:
  File "/home/deploy/.pyenv/versions/3.7.6/envs/py37/lib/python3.7/site-packages/aiohttp/client.py", line 1012, in __aenter__
    self._resp = await self._coro
  File "/home/deploy/.pyenv/versions/3.7.6/envs/py37/lib/python3.7/site-packages/aiohttp/client.py", line 582, in _request
    break
  File "/home/deploy/.pyenv/versions/3.7.6/envs/py37/lib/python3.7/site-packages/aiohttp/helpers.py", line 596, in __exit__
    raise asyncio.TimeoutError from None
concurrent.futures._base.TimeoutError

[seratch]

Sorry for the delay on this issue for a long time.

My pull request #662 (submitted yesterday) is the solution to this issue. The change will be included in the next minor version 2.6.0. If everything goes well, it will be released within two weeks.

Since this version, the default synchronous WebClient (=run_async=False) doesn't use asyncio and aiohttp internally. Instead, it just uses urllib, the standard Python module for HTTP communications. This means you no longer encounter the timeout error caused by asyncio's event loop.

If you prefer using aiohttp properly with run_async=True, it has been working fine and we haven't observed any issues from the beginning. Of course, the async mode will be available without any changes from now on.

But I think the troubles/concerns shared here are all about run_async=False use cases (=synchronous way that is more familiar to most people). So, version 2.6.0 will be the one for you.

[hbandlamudi]

@seratch I use the asynchronous version of the Slackclient and have been getting these errors for the past 2-3 months however freezing aiohttp to 3.6.2 did not help resolve the issue.

[seratch]

@hbandlamudi hmm, thanks for sharing this. May I ask you to provide a minimum working example that reproduces the issue easily? Or, does it occasionally occur when running the app for a while?

[seratch]

I tried to reproduce this issue with run_async=True but I was not able to. If someone could help us identify the reproduction steps, it would be greatly appreciated. As for version 2.6.0 development, I will focus on other remaining issues.

[hbandlamudi]

@seratch It's an intermittent issue but I've noticed it primarily for the api_test() and auth_test() calls.

[jhofeditz]

I reran my Web API synchronous processes, mostly conversations_replies() and conversations_history(), with this patch for the last two days and the issue appears to be fixed for me. (Previously the TimeoutError would occur within an hour). Looking forward to v2.6. Thanks!

[romutchio]

@seratch Well, how to fix this problem? Which version of slackclient i should use?

[seratch]

@romutchio If you want to use run_async=True (= use aiohttp), this issue is not yet fixed. If you don't have the intention to use aiohttp/asyncio for API requests, try the latest one (v2.7.1).

[NateVolt]

@seratch I am experiencing the same issue on 2.7.2. Has there been any progress on this issue?

Error

<class 'concurrent.futures._base.TimeoutError'>
Traceback (most recent call last):
 File "/home/pi/Documents/Python Files/Temp_Monitor_Project/slackmessaging.py", line 124, in StartRtm
   self.rtm_client.start()
 File "/home/pi/.local/lib/python3.7/site-packages/slack/rtm/client.py", line 206, in start
   return self._event_loop.run_until_complete(future)
 File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
   return future.result()
 File "/home/pi/.local/lib/python3.7/site-packages/slack/rtm/client.py", line 357, in _connect_and_read
   proxy=self.proxy,
 File "/home/pi/.local/lib/python3.7/site-packages/aiohttp/client.py", line 1012, in __aenter__
   self._resp = await self._coro
 File "/home/pi/.local/lib/python3.7/site-packages/aiohttp/client.py", line 728, in _ws_connect
   proxy_headers=proxy_headers)
 File "/home/pi/.local/lib/python3.7/site-packages/aiohttp/client.py", line 582, in _request
   break
 File "/home/pi/.local/lib/python3.7/site-packages/aiohttp/helpers.py", line 596, in __exit__
   raise asyncio.TimeoutError from None
concurrent.futures._base.TimeoutError

Slack code

import sys
import os
import traceback
import multiprocessing
from multiprocessing import Pipe
from slack import WebClient
from slack import RTMClient
from slack.errors import SlackApiError

class SlackMessaging():

    def handle_command(self, **payload):
        data = payload['data']
        web_client = payload['web_client']
        #Local Command List
        reset = 'reset'
        off = 'off'
        start = 'start'
        status = 'status'
        stop = 'stop'
        # get message text
        msgtext = data['text']
        returnvalue = -1
        response = ''

        #while self.data_pipe.poll() is True:
        #    self.current_task = self.data_pipe.recv()

        print("Slack message received: %s" % msgtext)
        if self.user_id not in msgtext:
            print("Message not addressed to us")
            return

        if reset.lower() in msgtext.lower():
            print("reset")
            response = "Reset command received. It may take several seconds for the system to respond to the command.\r\n"
            #response += f"Current task: {self.current_task}"
            returnvalue = 1
        elif off.lower() in msgtext.lower():
            print("off")
            response = "Off command received. It may take several seconds for the system to respond to the command.\r\n"
            #response += f"Current task: {self.current_task}"
            returnvalue = 2
        elif start.lower() in msgtext.lower():
            print("start")
            response = "Start command received. It may take several seconds for the system to respond to the command.\r\n"
            #response += f"Current task: {self.current_task}"
            returnvalue = 3
        elif status.lower() in msgtext.lower():
            print("status")
            response = "Status command received. It may take several seconds for the system to respond to the command.\r\n"
            #response += f"Current task: {self.current_task}"
            returnvalue = 4
        elif stop.lower() in msgtext.lower():
            print("stop")
            response = "Stop command received. It may take several seconds for the system to respond to the command.\r\n"
            #response += f"Current task: {self.current_task}"
            returnvalue = 5
        else:
            print("Not a valid command")
            response = self.GetCommandListResponse()

        channel_id = data['channel']
        user = data['user'] # This is not username but user ID (the format is either U*** or W***)
        web_client.chat_postMessage(
            channel=channel_id,
            text=f"<@{user}> {response}"
            )

        if returnvalue is not -1:
            self.data_pipe.send((returnvalue, channel_id, user))

    def GetCommandListResponse(self):
        response = "Use Instructions:\r\n"
        response += "Send only one of the commands listed below. Commands are case insensitive. If a command is submitted incorrectly this message will be sent again.\r\n\r\n"
        response += "Command List:\r\n"
        response += '`reset` - Executes a hardware and software reset of the monitor. New logfiles will be started and all the previous logfile will need to be manually retreived from memory.\r\n'
        response += '`off` - Turns off the RPDU output to transmitter.\r\n'
        response += '`start` - Enables monitoring of temperatures and enables sending of daily status messages. Off by default.\r\n'
        response += '`stop` - Disables monitoring of temperatures and disables sending of daily status messages.\r\n'
        response += '`status` - Replies with an message that contains output status of RPDU to transmitter as well as the current temperatures.\r\n'
        return response

    def GetMainChannel(self):
        data = self.slack_client.users_conversations()
        # print(data)
        channelid = data['channels'][0]['id']
        return channelid

    def ReplyToUser(self, channel_id, user_id, msg):
        self.SendMessage(f"<@{user_id}> {msg}", channel_id)

    def DirectMessageMaintainer(self, message):
        self.DirectMessageUser(message, self.maintainer_user_id)

    def DirectMessageUser(self, message, userid):
        self.SendMessage(message, userid)

    def SendMessage(self, message, channelid):
        print(f"Sending: {message}")
        self.slack_client.chat_postMessage(
        channel=channelid,
        text=message
        )

    def PostToMainChannel(self, message):
        self.SendMessage(message, self.main_channel)

    def NotifyAll(self, message):
        self.SendMessage(f"<!here> {message}", self.main_channel)

    def StartRtm(self):
        try:
            # create an instance of the real time messaging interface
            self.rtm_client = RTMClient(token=self.slack_token)
            # set the callback for when a new message is posted
            self.rtm_client.on(event="message",callback=self.handle_command)
            # start the real time messenger
            # this is a blocking call and runs forever
            self.rtm_client.start()
        except (KeyboardInterrupt, SystemExit):
            raise
        except:
            e = sys.exc_info()[0]
            t = traceback.format_exc()
            self.DirectMessageMaintainer(f"There was an unhandled error.\r\n\r\n{str(e)}\r\n\r\n{t}")
            raise e

    def __init__(self, pipe):
        self.current_task = ""
        # If the maintainer is no longer Nate, replace this user ID with the new maintainer's user ID.
        self.maintainer_user_id = 'Uxxxxxxx'
        # keep a reference to the pipe we will use to send data back to the main app with
        self.data_pipe = pipe
        # our slack api token for this bot. It has limited permissions.
        # this is still not a best practice and should be fixed later.
        self.slack_token = 'xoxb-xxxxxx'
        # create an instance of the slack client
        self.slack_client = WebClient(token=self.slack_token)
        # get the bot's user id
        self.user_id = self.slack_client.auth_test(token=self.slack_token)['user_id']
        # find the channel that users will use to converse with this bot.
        self.main_channel = self.GetMainChannel()

Snippet of the calling code.

import slackmessaging
import multiprocessing
from multiprocessing import Pipe

our_connection, their_connection = Pipe()
self.slack_comm = slackmessaging.SlackMessaging(their_connection)
self.slack_pipe = our_connection
slack_rtm_process = \
  multiprocessing.Process(\
  target=self.slack_comm.StartRtm)

slack_rtm_process.start()

[seratch]

@NateVolt Your issue looks a bit different from the one discussed here. Your issue is about WebSocket connection establishment in RTMClient and its underlying aiohttp library while this issue is about WebClient in run_async=True mode. I don't have any clear answer for your question yet but according to the stack trace, it may be related to your proxy settings.

[tgwaste]

Is there a solution for this?

[seratch]

@cyphr3x
The current WebClient doesn't provide coroutines. Instead, it generates a Future for every single method call. I think this may be the cause of the intermittent issue.

In v2.8.0, we've introduced a more straight-forward async client named AsyncWebClient. All the methods in this class provide coroutines. https://slack.dev/python-slackclient/changelog.html#v2-8-0-2020-08-06

You can use the new class this way:

import asyncio
import os
from slack import AsyncWebClient

client = AsyncWebClient(token=os.environ["SLACK_BOT_TOKEN"])

async def main():
    new_message = await client.chat_postMessage(channel="#general", text="Hi")

asyncio.run(main())

• https://github.com/slackapi/python-slackclient/blob/v2.8.1/integration_tests/web/test_async_web_client.py
• https://github.com/slackapi/python-slackclient/blob/v2.8.1/tests/web/test_async_web_client.py

If you still encounter this (or a similar) issue with AsyncWebClient, please let us know the details of it. In this case, breaking down this issue should be much easier. The cause can be the configuration issue on this library side or something in aiohttp library side.

[tgwaste]

Traceback (most recent call last):
  File "./t.py", line 24, in <module>
    asyncio.run(main())
AttributeError: module 'asyncio' has no attribute 'run'

Tried substituting with this

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

got the same timeout error
concurrent.futures._base.TimeoutError

The answer (before you ask) is: No I can not simply upgrade to python 3.7.0. Thats not an acceptable solution.

[seratch]

Thanks for sharing this. If you don't see any issue when you directly use aiohttp (the underlying HTTP library) and this library does something wrong, please let us know by creating a new ticket in this project. We'll be checking the issue.

[tgwaste]

im so glad we pay you so many thousands of dollars.

Thanks for sharing this. If you don't see any issue when you directly use aiohttp (the underlying HTTP library) and this library does something wrong, please let us know by creating a new ticket in this project. We'll be checking the issue.

HUH??????????

[seratch]

My comment was not great. I just wanted to say, if you could provide us with information, it'd be greatly appreciated. We're still looking for ways to completely address this issue.

[tgwaste]

I did provide you with information. I cant believe how difficult it is to read and reply to messages via your API.