Sentry-python: Raven python processors API alternative

Created on 16 Dec 2018 · 13Comments · Source: getsentry/sentry-python

In Raven python there was a processors param that could sanitize data client-side before sending to the server https://docs.sentry.io/clients/python/advanced/#sanitizing-data . After upgrading, I noticed that the processors API has been removed and replaced with before_send and a send_default_pii option. This has similar power, but requires a lot more setup than just adding processors raven.processors.SanitizePasswordsProcessor, etc.

What is the correct way to sanitize passwords, etc. now? It looks like passwords are not sanitized and sent to the Sentry server by default in the new version, which is not acceptable in my environment.

enhancement help wanted

Source

taylor-cedar

👍1

Most helpful comment

I migrated from raven to sentry-sdk and decided to still use raven processors. Here is the code if it is useful to anyone reaching this issue.

from raven.processors import SanitizeKeysProcessor, SanitizePasswordsProcessor

class FakeRavenClient:
    sanitize_keys = [
        'card_number',
        'card_cvv',
        'card_expiration_date',
    ]

processors = [
    SanitizePasswordsProcessor(FakeRavenClient),
    SanitizeKeysProcessor(FakeRavenClient),
]

def before_send(event, hint):
    for processor in processors:
        event = processor.process(event)
    return event

sentry_sdk.init(
    before_send=before_send,
)

iurisilvio on 23 May 2020

👍6

All 13 comments

Yes, the way to go here is before_send. The main reason is that people sent us a lot of pull requests slightly adjusting the regex(es) to fit their data, which would break other people's code. We'd be happy to add code snippets to the docs that show how to do this with before_send such that it's easy to get started.

untitaker on 19 Dec 2018

That is fine. I think it should be heavily stated in the docs that upgrading will remove all filters of data because I didn't notice it until I was looking through the code. There was a really nice utility class called SanitizeKeysProcessor that went through the event dictionary and obfuscated all the data. It would be really nice to supply a replacement or link to a third party replacement.

before_send: sanitize_keys(
  'password',
  'secret',
  'key',
  # ...
)

taylor-cedar on 19 Dec 2018

Oh yeah the argument with the many pull requests doesn't really work for sanitize_keys. I would have this as a sort of integration for sure.

untitaker on 19 Dec 2018

Ok great! I was trying to build a replacement for the SanitizeKeys processor and noticed no documentation around the Event object shape (all the fields). Is it documented anywhere? I want to make sure my obfuscation code works correctly with all types of events.

taylor-cedar on 19 Dec 2018

event payload is documented here: https://docs.sentry.io/development/sdk-dev/interfaces/

untitaker on 19 Dec 2018

@untitaker, I would like to remove items from event['request']['data'], but when I do del event['request']['data']['field], then I get the Invalid interface data in sentry. Do you have any idea why?

gabor-boros on 7 Jan 2019

@gabor-boros no, sorry. I would create a separate issue anyways

untitaker on 7 Jan 2019

FYI: @untitaker
The issue is solved. The root cause was that the initialization of the SDK happened on request time, instead of application load. The issue can be closed.

gabor-boros on 18 Jan 2019

@gabor-boros That's nice to hear but this issue was not about your issue.

untitaker on 18 Jan 2019

I migrated from raven to sentry-sdk and decided to still use raven processors. Here is the code if it is useful to anyone reaching this issue.

from raven.processors import SanitizeKeysProcessor, SanitizePasswordsProcessor

class FakeRavenClient:
    sanitize_keys = [
        'card_number',
        'card_cvv',
        'card_expiration_date',
    ]

processors = [
    SanitizePasswordsProcessor(FakeRavenClient),
    SanitizeKeysProcessor(FakeRavenClient),
]

def before_send(event, hint):
    for processor in processors:
        event = processor.process(event)
    return event

sentry_sdk.init(
    before_send=before_send,
)

iurisilvio on 23 May 2020

👍6

@iurisilvio thanks for the snippet!

@untitaker the https://docs.sentry.io/development/sdk-dev/interfaces/ link seems to be dead - where is the up-to-date description of the event payload? TIA!