Sentry-python: Aiohttp support

[mike-hart]

I don't see a clear way to integrate with aiohttp (server) just yet. I can still use raven at present, but I imagine sentry-python/sentry-sdk is the proposed way forward longer term?

[untitaker]

Hi, we currently don't have any integration with aiohttp-server on the new sdk. If anybody is interested in this feature please just vote with reactions on the OP and we'll prioritize accordingly.

[mitsuhiko]

@mike-hart what do you have in mind there? Afaik the old SDK also did not have an aiohttp integration (just an aiohttp transport which we do not believe we will provide going forward).

[mike-hart]

@mitsuhiko - well, right now I'm just looking for a way to replace the error-reporting middleware I have that currently uses raven. Making sure I don't end up "stuck" using an unsupported/non-current framework. Maybe a middleware isn't the best approach, if not then some documentation or a basic recipe would be a great start.

Right now I don't know enough about sentry-sdk to know whether it'll stall an async process while contacting sentry itself, how to replace its transport if it does, whether it's threaded, whether there will be unexpected or undesired outcomes from using its defaults. All I really want is a mechanism to report uncaught exceptions (basically something that would result in a 5xx status code) to Sentry for my service.

[mitsuhiko]

@mike-hart calling any of the capture methods on the hub/client/transport are non blocking operations. They enqueue into a background queue.

[mike-hart]

@mitsuhiko - I note in https://docs.sentry.io/platforms/python/ that there's mention of httplib_request hints, is this the mechanism I should be using to add additional information to the reported exception for a service?

Raven had a semi-standard structure (the http context) you'd build to feed to the capture so that you knew URL, query params etc. Is this is the same thing, and are there any documented examples of how to build something similar for sentry-sdk? Or am I going down completely the wrong path?

[mitsuhiko]

@mike-hart i'm not entirely sure what you mean but i would propose to look at the sanic, flask and django integrations to see how this works. Integrations are pretty straightforward and can emit any kind of data. The structure from raven is very much still there, we just simplified it.

[mitsuhiko]

Eg: this is how sanic works: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/sanic.py

[mike-hart]

Thanks, I'll take a further look at the integrations and see if I can come up with something. The mechanism seems a little... invasive (at least when looking at sanic, django, and flask integrations, where they appear to monkey-patch). Is there a reason (say, in sanic's case) not to tie in to the inbuilt error-handler mechanism for example?

Anyway, thanks for the pointers. I'll have a further play around and see if I can get something together, sorry for taking up so much of your time.

[mitsuhiko]

We intentionally patch the core handlers now as there were too many issues with working with the individual error handlers that are already in place. Both in reliability and usability. More importantly we already in the past had to do invasive patching for breadcrumbs so the behavior was wildly inconsistent.

Generally the builtin error handlers are typically compromising so that they can still render error pages which swallows quite a few errors we are still interested in.

[untitaker]

So this seems to have a lot of support. Is this something that used to work just fine with raven for you? Is there a third-party package that enabled this for you?

[decaz]

@untitaker if your question is what do people use within aiohttp, I can answer for myself - raven-aiohttp and hope sentry-python will support aiohttp as it is critical feature and it's absence prevents me from using sentry-python.

[untitaker]

@decaz how does its absence prevent you from using the new SDK? Surely spawning threads like the new SDK does would still work even if it's not the optimal way to do things? I haven't tried it but it should work out of the box.

Note that as far as I can tell this is only about the transport used internally. Perhaps I don't have the full context as to why people use raven-aiohttp in the first place

[decaz]

I think people use raven-aiohttp because new SDK does not contain integration with aiohttp. And yes, this is also about the transport.

[mike-hart]

@untitaker - For me, it would just be nice if aiohttp (and related asyncio-based systems) had some kind of first-class support is all. Instead of each of us coming up with out own (possibly wrong) recipe to integrate Sentry.

We're not _prevented_ from using sentry-python, we just don't necessarily have the confidence to integrate something as invasively as this appears to require in such a manner that it's done properly. Doing it properly is pretty high on my list when it comes to adding an exception handling framework to my services, especially if that service might be handling hundreds of open connections in-process, and doubly-so to avoid reporting on the wrong async stream/connection as being the cause.

[untitaker]

@mike-hart I understand all of your points. I am just very confused about statements like this:

I think people use raven-aiohttp because new SDK does not contain integration with aiohttp.

raven-aiohttp has nothing but a custom transport.

[mike-hart]

raven-aiohttp has nothing but a custom transport.

You might want to ask @decaz more about that in particular, but my take on it is this:
That transport is the way to avoid the Sentry client stalling the eventloop. I suspect most people just add the client.captureException call in a piece of middleware post-request and be done with it. If something more specific is needed by them then I guess they would go from there (doing something more aggressive), adding context details, breadcrumbs etc. etc.

[decaz]

I meant that if a user who is using raven-aiohttp meets the SDK and sees that it does not have integration with aiohttp, then he probably will not think about transport or anything else, for him it looks like a library without support aiohttp, nothing more.

[untitaker]

That transport is the way to avoid the Sentry client stalling the eventloop

The eventloop is not stalled either way unless you explicitly use a blocking transport, which this SDK does not provide. The default transport in this SDK just spawns a new thread which should not be a problem even with async code.

I suspect most people just add the client.captureException call in a piece of middleware post-request and be done with it. If something more specific is needed by them then I guess they would go from there (doing something more aggressive), adding context details, breadcrumbs etc. etc.

Exactly. I don't know of prior art that does this out of the box for aiohttp.

[mike-hart]

I guess part of this is that people love to hate threads 😉

More seriously though, some proper integration would be lovely. There are a number of asyncio-based frameworks now becoming much more popular, and getting it right would be useful. aiohttp and perhaps starlette might be good ones to start on so that people know how to do it correctly.

[untitaker]

aiohttp-server support will be implemented at #189. It can capture basic request info and deals with unhandled exceptions

[untitaker]

Docs here: https://docs.sentry.io/platforms/python/aiohttp/

[cancan101]

Any plans to support (automatic) performance monitoring of aiohttp client calls?