Sanic: Run a task in background

[aqeelahmad]

I am trying to figure out an example which explains how to run a coroutine in the background. I haven't found anything significant. Any help will be great.

[iasemin]

try this:

import asyncio
asyncio.ensure_future(func_name(params), loop=loop)

[dalianzhu]

i made a consumer in sanic like this:

@app.listener('before_server_start')
async def setup(app, loop):
    ......
    logging.info("add task maintanance_count")
    from asyncio import Queue
    app.maintanance_count_queue = Queue(loop=loop)
    app.maintanance_count_dict = {}
    from handlers.count_click import maintanance_count
    app.add_task(loop.create_task(maintanance_count(app)))

Maybe this helps.

[aqeelahmad]

@SwordFighter I have already been trying this with no help yet. It kind of gets lost somewhere. I am printing some statements in that func_name but that never runs it seems.

@dalianzhu , The example you have given seems to be adding the task before starting the server. I am looking for something while the server is running. I have seen examples similar to above in Sanic docs too but that is somewhat different behaviour.

[iasemin]

@aqeelahmad if you print full code with your call ensure_future i can help to fix.

ensure_future is standard python way to start func https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future

[r0fls]

@aqeelahmad if your task doesn't exit it will keep running after the server starts when using app.add_task. Have you tried something like this?

```python
async def notify_every_five_seconds():
while True:
await asyncio.sleep(5)
print('it's been five seconds!')

app.add_task(notify_server_started_after_five_seconds())

... other routes, start server

````

[r0fls]

I'm closing this for now but feel free to reopen if you're still having issues

[rcarmo]

I'd just like to comment that doing app.add_task(foo()) doesn't work when invoked from inside something like this:

@app.listener('after_server_start')
async def init_connections(sanic, loop):
    """Bind the database and Redis client to Sanic's event loop."""

    global redis, db
    redis = await connect_redis()
    motor = AsyncIOMotorClient(MONGO_SERVER, io_loop=loop)
    db = motor[DATABASE_NAME]
    log.debug("adding task")
    app.add_task(monitor_loop(db))

This is a bit of a pain because I'd like my monitor_loop to have the database connection...

[Hubro]

Starting a background task using app.add_task(...) results in this error when the app is shut down (using Ctrl+c):

sys:1: RuntimeWarning: coroutine 'test_device_connections' was never awaited

I'm not running Python in debug mode, so this is always printed on shutdown.

I could definitely be wrong about this, but shouldn't Sanic await all the started tasks before shutting down to ensure they are all properly cleaned up? What if I have a task running that looks like this:

def my_background_task():
    try:
        while True:
            asyncio.sleep(60)
            do_background_task_stuff()
    except asyncio.CancelledError:
        do_cleanup()

Will my app be cleaned up in this case, if I run app.add_task(my_background_task())? Or is my background task abruptly halted as the process is terminated?

The only documentation about add_task I've been able to find is at the bottom of this page: https://sanic.readthedocs.io/en/latest/sanic/middleware.html, and it doesn't really mention anything about cleanup.

[ahopkins]

First, as a matter of housekeeping, rather than reviving a 3+ year old issue, it is generally better to open something new and link to it. I get it, every project has different preferences.

Out of the box, Sanic does not do any management for background tasks for you. It is a fairly simple wrapper around create_task meant to make it easier to attach to the existing loop, and ensure that the task is started after the server has begun.

With that said, perhaps this is functionality that could be added, and I welcome you to propose it if you think this is a worthwhile feature.

Sanic will perform a graceful shutdown of all running route handlers to make sure that all connections are resolved and closed properly. It does not await the completion of tasks though by design. It is not necessarily known whether the task will complete, how long a timeout should be given, or (as in your example) the task will never complete. Perhaps there are some sensible defaults, and if you propose this feature we should discuss what they should be.

If you would like to implement something yourself, here is a simple working example:

async def my_background_task():
    print("START TASK")
    try:
        while True:
            print(".", end="", flush=True)
            await asyncio.sleep(1)
    except asyncio.CancelledError:
        print("DO CLEANUP")
    finally:
        print("TASK DONE")

@app.listener("after_server_start")
def task_start(app, loop):
    app.add_task(my_background_task())


# after_server_stop because all HTTP connections have been closed
@app.listener("after_server_stop")
async def stop_bg(app, loop):
    current = asyncio.current_task()
    for task in asyncio.all_tasks():
        if task is not current:
            task.cancel()
            await task

Logs:

[2021-02-01 10:33:14 +0200] [354794] [INFO] Goin' Fast @ http://127.0.0.1:9999
[2021-02-01 10:33:14 +0200] [354794] [INFO] Starting worker [354794]
START TASK
...^C[2021-02-01 10:33:17 +0200] [354794] [INFO] Stopping worker [354794]
DO CLEANUP
TASK DONE
[2021-02-01 10:33:17 +0200] [354794] [INFO] Server Stopped

[Hubro]

@ahopkins First of all my apologies, my instinct was that it was better to ask a question on an existing, relevant issue rather than creating a (possibly) duplicate issue.

In any case, I did try to implement the task lifecycle manually, but the only way I found to actually start the task was to use add_task, and it didn't give me the task object back, so I had no reference to the task when cleaning up. I didn't know about the all_tasks function, that definitely helps.

Would it be a breaking change to modify the add_task function to return the created task? I believe it just returns None currently.

If not, I will create a new issue for that and see if I can implement it myself.

[ahopkins]

No apologies necessary.

I agree that it probably does make sense to return the task, but that creates a potential problem: Sanic attempts to catch the situation when a task is added too soon by not scheduling it until after the server is running. I _think_ that potential solution that would not be breaking would be to store a reference to those on some object at the app level: Sanic.background_tasks.

Open up a new issue and we can continue the conversation there to keep it separate from this issue.