Discord.py: Bot exits with "Task was destroyed but it is pending!" inside client.py

[ams2990]

Full error, skimpy though it is:

Task was destroyed but it is pending!
task: <Task pending coro=<_run_event() done, defined at /usr/local/lib/python3.5/dist-packages/discord/client.py:304> wait_for=<Future pending cb=[BaseSelectorEventLoop._sock_connect_done(6)(), Task._wakeup()]>>

When the crash happens, I've always just invoked this line:

await bot.send_message(destination=channel, content=output)

However, it seems to depend on the contents of the output string. I haven't been able to figure out exactly what it is about the string that causes a crash.

[ams2990]

Okay, my claim about it depending on the value of output is wrong. I've run the same command multiple times (and being deterministic, it prints the same output every time). The crash only happens sometimes. This seems like a good old-fashioned race condition.

[Phxntxm]

The error only means that there was a task pending that was killed, it's not enough information to go off of anything. This is probably not relevant to this send_message line that you're referring to

[ams2990]

It doesn't reproduce every time I call send_message, but every time it crashes its while waiting for send_message to return. When the bot crashes, the message is never delivered, but the log line immediately preceeding the call is always present.

I don't have any background tasks of my own, just two command handlers. I recognize this bug report is pretty bereft of useful information -- how can I get you something more useful?

[rce]

Apparently the websocket gets closed (opcode=8) and it kills the async event loop or something even stranger. That causes the gateway to fail (bot user goes offline) and all background tasks to be destroyed (Task pending errors). This issue appeared after updating from v0.14.3 to 0.15.1 and the same code has been working for days straight without issues on v0.14.3.

Dec 14 03:47:02 lemon-bot python3[1956]: DEBUG:websockets.protocol:client << Frame(fin=True, opcode=8, data=b'\x03\xe8')
Dec 14 03:47:02 lemon-bot python3[1956]: DEBUG:websockets.protocol:client >> Frame(fin=True, opcode=8, data=b'\x03\xe8')
Dec 14 03:47:02 lemon-bot python3[1956]: ERROR:asyncio:Task was destroyed but it is pending!
Dec 14 03:47:02 lemon-bot python3[1956]: task: <Task pending coro=<task() running at /home/lemon/src/feed.py:80> wait_for=<Future pending cb=[Task._wakeup()]>>
Dec 14 03:47:02 lemon-bot python3[1956]: ERROR:asyncio:Task was destroyed but it is pending!
Dec 14 03:47:02 lemon-bot python3[1956]: task: <Task pending coro=<task() running at /home/lemon/src/archiver.py:115> wait_for=<Future pending cb=[Task._wakeup()]>>

[Rapptz]

The websocket code has been unchanged since v0.14.3 to v0.15.1.

What opcode 8 means is that the websocket is requesting closure. This can be from any reason -- either Discord is terminating your connection, CloudFlare is requesting a reset, or you're requesting closure. This has nothing to do with what version you're using, especially considering the code for this type of logic hasn't changed.

The message "task was destroyed" does not signify anything of importance, all things considered. It just means that a task's __del__ was called without being it being complete. This could be a poorly written Task that wasn't cleaned up, or something else entirely. As it stands, "Task was destroyed" by itself means nothing and it's tempting to shut it off entirely.

If you guys have more information that would actually help solve this issue then I welcome it, but as it stands now there is not enough information to fix this, track it down, or do anything with it and can be chalked up as "Discord or CloudFlare are being weird".

[ams2990]

I have no reason to believe this is a recently introduced issue - I'm writing new code, rather than finding an issue in existing code after upgrading.

Also, I think Henry's issue is different than mine because his is failing in his own code, whereas mine is failing inside discord.py's code.

I agree with you about the Task being at fault. My point with this bug is, the Task in question is inside discord.py. It doesn't seem to me that either Discord or CloudFlare are at fault. However, if I've missed something please let know as I'm going to start reading discord.py's code tonight in hopes of finding what I suspect is a missing state transition.

[ccubed]

Which do you think is more likely? That this library has completely and utterly failed to do the one thing it was made to do or that user error is responsible?

[Rapptz]

I don't think there's much to look at. The task you mention that is "inside discord.py" is the wrapped up task that executes the user callbacks. Ergo, things like on_message and friends.

Those tasks dying mean nothing to me.

[ams2990]

Alright, I was able to make a fairly minimal test case. Make a command that takes a while to do work. To avoid figuring out what the precise amount of time required is, I just slept for two minutes. After waking up, try to send a message to the channel.

If I had to guess, the bot is running the on command callback on the same thread it executes heartbeats on. I also noticed when testing that it executes all commands on the same thread. I would have thought the point of making the callbacks async was so that they could be executed asynchronously. Is this working as intended?

[Rapptz]

Make a command that takes a while to do work. To avoid figuring out what the precise amount of time required is, I just slept for two minutes. After waking up, try to send a message to the channel.

And..? People have made thousands of commands that do this with no issues -- myself included. This is telling me nothing.

If I had to guess, the bot is running the on command callback on the same thread it executes heartbeats on.

No. https://github.com/Rapptz/discord.py/blob/master/discord/gateway.py#L57-L87

I also noticed when testing that it executes all commands on the same thread. I would have thought the point of making the callbacks async was so that they could be executed asynchronously.

From asyncio docs:

This module provides infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives. Here is a more detailed list of the package contents:

Emphasis mine.

Is this working as intended?

Yes.

Please familiarise yourself with asyncio and how it works.

[serfma]

As someone who uses discord.py and is also new to asyncio and Python in general, I've ran into this same kind of pickle (bot crashing randomly every now and again due to task destroyed). After reading through the comments I wanted to ask; Is there any way to narrow down what's calling __del__ for a Task? And, is it a bad idea/bad practice to have a Task which contains a while loop that checks something every X seconds?

I understand it's not a lib issue and a user issue, especially as I'm not incredibly familiar with asyncio.

[acffordyce973]

I have a similar issue where the bot will exit for no reason with the same messages and would appreciate it if someone can tell me any way to narrow it down as well.

[TheSilvus]

I have basically the same issue and don't know why. It's probably a user mistake, but it'd help a lot if the library would be able to output a useful error message. I don't know a lot about Asyncio (or Python in general), would it be possible to print which task did something wrong? Or another way that can help users to find the bug?

[ghost]

This isn't related to the library, it's more with your code. Pretty sure it's when a task times out, or the loop is closed.