Sentry-python: Django integration can't handle psycopg2's SQL string `.format`

[mlennon-lumere]

Relates to #169

psycopg2 2.7 introduces the psycopg2.sql module, which "contains objects and functions useful to generate SQL dynamically, in a convenient and safe way."

http://initd.org/psycopg/docs/sql.html

Unfortunately, format_sql() doesn't know how to work with a 'Composed' object. So when we try to make use of this in a Django application, we get tracebacks from the sentry_sdk.

Here's a simple management command that illustrates the problem.

from django.core.management.base import BaseCommand
from django.db import connection
from psycopg2 import sql


class Command(BaseCommand):
    def handle(self, *args, **options):
        cursor = connection.cursor()

        cursor.execute('drop table if exists my_test_table')
        cursor.execute('create table my_test_table (id int)')
        cursor.execute(sql.SQL("""
            SELECT
                1
            FROM
                {my_table} 
            """).format(my_table=sql.Identifier('my_test_table')), {
            'foo': 'foo', #This is needed to trigger the bug.  In our prod code we use this to format into our 
            # query qith %(foo)s syntax, omitted here for clarity 
        })

Running it with sentry_sdk's DjangoIntegration() enabled produces the following error:

$ ./manage.py test-sentry
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/ph/ph/app/management/commands/test-sentry.py", line 18, in handle
    'foo': 'foo',
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/django/__init__.py", line 273, in execute
    record_sql(sql, params)
  File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/django/__init__.py", line 240, in record_sql
    real_sql, real_params = format_sql(sql, params)
  File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/django/__init__.py", line 222, in format_sql
    sql = sql % conv
TypeError: unsupported operand type(s) for %: 'Composed' and '_FormatConverter'
$

[untitaker]

Got it. Next time pls use your own words, I was about to close this as "invalid" because at first glance it seemed like you just copypasted the old issue entirely.

[mlennon-lumere]

Update: here's another case where the sentry_sdk will currently fail:

from django.core.management.base import BaseCommand
from django.db import connection
from psycopg2 import sql


class Command(BaseCommand):
    def handle(self, *args, **options):
        cursor = connection.cursor()
        names = ['foo', 'bar', 'baz']
        cursor.execute('drop table if exists my_test_table')
        cursor.execute('create table my_test_table (foo text, bar text, baz text)')

        q2 = sql.SQL("insert into my_test_table ({}) values ({})").format(
            sql.SQL(', ').join(map(sql.Identifier, names)),
            sql.SQL(', ').join(map(sql.Placeholder, ['first_var', 'second_var', 'third_var'])))

        cursor.execute(q2, {'first_var': 'fizz', 'second_var': 'buzz', 'third_var': 'fizzbuzz'})

[untitaker]

@mlennon-lumere Yeah. I am currently trying to figure out how to support and write tests for all of this without installing postgres in Travis. Problem is that the current tests don't actually have/use a psycopg cursor so your PR doesn't pass tests.

[mlennon-lumere]

Fixed in #203