Uvicorn: How to support server side events (sse)

[almarklein]

I'm experimenting with SSE, and run into the problem that the server keeps on sending data even when the client is gone (or closes the connection). This is problematic, because over time, these zombie tasks will pile up. The same situation would occur with long-polling.

So at first I spawned a task that will call receive() until it gets "http.disconnect" and then cancels the task corresponding to the request (the one that sends messages to the client).

But this seems rather awkward, so I took a look at send(), and saw that this line means that my coroutine happily keeps sending data ... into the void, forever.

Would it make sense to replace that return with a RuntimeError? Or is there another solution that I overlooked?

[almarklein]

Actually, it would be better if it were an exception time that is easy to distinguish from other exceptions that the coro might raise, so one can do some appropriate cleanup too.

[tomchristie]

So, if the ASGI spec mandated a particular exception type to indicate disconnects, then that could work.

I’m not sure if I’d be in favour of that or not.

As it stands you’ll want to structure things so that the application code listens to and handles the http.disconnect event.

[tomchristie]

If you take a look at Starlette’s WebsocketEndpoint, or similar stuff in channels that should give you a good starting point on how to structure SSE implementations.

[almarklein]

Ok, I now see that indeed the spec explicitly specifies this behavior:

Note that messages received by a server after the connection has been closed are not considered errors. In this case the send awaitable callable should act as a no-op.

The consequence is that in a situation where the server is only sending data, and not receiving any, the server will never know if the client closes the connection. For the server to be notified, it would need a separate task that awaits receive(). This feels awkward to me, especially since at the server level (the internals of the receive() func in h11_impl.py) this information is in fact available.

If you take a look at Starlette’s WebsocketEndpoint

Correct me if I'm wrong, but I think the same applies to a ws: a disconnect from the client is only detected if you call ws.receive().

[rob-blackbourn]

I've tried SSE out with uvicorn and it worked well.
Some examples here, here and here.

[almarklein]

@rob-blackbourn Ah, nice! I ran your example and it indeed works fine. It looks like this part of your source code does the trick - you're using one task for sending, and one for receiving.

I suppose that's what I'll do (for now). I still wonder if its worth discussing to communicate to a send() caller that the connection has dropped, since send() internally knows this. Or is the overhead of that extra task (for each connection) negligible?

[rob-blackbourn]

@almarklein Yes. Implementing SSE exposed a lot of problems in my implementation :P

I'm don't know enough about the internals of uvicorn to have an opinion about how send() should work, but I just tested it with Hypercorn to see if the treatment was consistent, and it seems so.

Here's what I did to try it with Hypercorn.

# uvicorn.run(app, host='localhost', port=9009)

from hypercorn.asyncio import serve
from hypercorn.config import Config

config = Config()
config.bind = ["127.0.0.1:9009"]
asyncio.run(serve(app, config))

[tomchristie]

@almarklein If you take a look at how channels works wrt. websockets and broadcast groups you'll see that the application code doesn't end up having to deal with creating an extra task, because instead, everything's driven by the events that occur. (And each event driven bit of logic is an isolated bit of "do something, then return".

You'll want to follow similar patterns, although eg. Starlette could do with a little more work in this area, in order to handle the frameworky bits for broadcast and SSE, so that the application developer doesn't need to think about it.

(And yeah, task overhead is negligable.)

I'm sure this'll keep coming up for discussion, and you'd be very welcome to raise it over on the ASGI spec for further talking through.

One thing I do think that servers should probably do on send is raise errors against sending to disconnected channels after a timeout period. That'd give application code the neccessary buffer period to react cleanly to the http.disconnect event, while still eventually erroring out if absolutely required.

[almarklein]

I'll have a look at what channels does then. But first holidays :)

[almarklein]

I recently found the time to look into this again, and added proper support for SSE to Asgineer. As for my original question, the pattern to follow is to let a task receive, as to detect the client from disconnecting. I now see how this is better :)

For those interested: in Asgineer I added support to be able to sleep the connection: until the connection closes, a timeout expires, or it is woken up. This allows using SSE (and e.g. long-polling) in various ways.

Anyway, this can be closed.

[euri10]

Looks great @almarklein !!
Ping @florimondmanca for https://github.com/florimondmanca/awesome-asgi 😋

[almarklein]

Ping @florimondmanca

Already contributed ;)