Pytest-django: pytest is unable to catch django's IntegrityError

[vanyakosmos]

pytest is unable to catch django's IntegrityError

# models.py
class A(models.Model):
    val = models.BooleanField(default=True)

class B(models.Model):
    a = models.ForeignKey(A, on_delete=models.CASCADE)

# tests.py
from django.db import IntegrityError
import pytest

@pytest.mark.django_db
def test_integrity():
    with pytest.raises(IntegrityError):
        # using object A with id=1 before creating it 
        B.objects.create(a_id=1)

After running pytest it will show two errors for test_integrity function:

• FAILURES: "DID NOT RAISE "
• ERROR at teardown: "django.db.utils.IntegrityError: insert or update on table "core_b" violates foreign key constraint "core_b_a_id_2a5b83ce_fk_core_a_id"

Which is basically: "here is an error about not raising IntegrityError in test_foo, and here is an error for raising IntegrityError in test_foo, deal with it"

I can catch Integrity error w/o problems outside of pytest:

root@1f28fc583eed:/app# ./manage.py shell
Python 3.6.7 (default, Oct 24 2018, 22:47:56)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from core.tests import test_integrity
>>> test_integrity()
>>> exit()

pip list

Package             Version
------------------- ---------
apipkg              1.5
asn1crypto          0.24.0
atomicwrites        1.3.0
attrs               19.1.0
certifi             2019.3.9
cffi                1.12.3
chardet             3.0.4
codecov             2.0.15
coverage            4.5.4
cryptography        2.6.1
dj-database-url     0.5.0
Django              2.2.3
django-dbbackup     3.2.0
django-extensions   2.1.7
emoji               0.5.2
execnet             1.6.1
future              0.17.1
gunicorn            19.9.0
idna                2.8
more-itertools      7.0.0
pip                 18.1
pluggy              0.11.0
psycopg2-binary     2.8.2
py                  1.8.0
pycparser           2.19
pytest              4.5.0
pytest-cov          2.7.1
pytest-django       3.4.8
pytest-forked       1.0.2
pytest-mock         1.10.4
pytest-xdist        1.29.0
python-telegram-bot 12.0.0b1
pytz                2019.1
redis               3.2.1
regex               2019.4.14
requests            2.22.0
schedule            0.6.0
setuptools          40.4.3
six                 1.12.0
sqlparse            0.3.0
tornado             6.0.2
urllib3             1.25.3
wcwidth             0.1.7
wheel               0.32.2
whitenoise          4.1.3

Versions:

pytest: 4.5.0
django: 2.2.4

• [x] a detailed description of the bug or suggestion
• [x] output of pip list from the virtual environment you are using
• [x] pytest and operating system versions
• [x] minimal example if possible

[blueyed]

How does the stacktrace for django.db.utils.IntegrityError look like?
Please also try it with the latest pytest and pytest-django.

[vanyakosmos]

I already deleted example project, but here are error logs from another project where I test essentially the same thing: object created with id of related object that doesn't currently exist.

IntegrityError logs

______________________________ ERROR at teardown of TestReactionModel.test_safe_create_without_user ______________________________

self = <django.db.backends.utils.CursorWrapper object at 0x7f7c8e278898>, sql = 'SET CONSTRAINTS ALL IMMEDIATE', params = None
ignored_wrapper_args = (False, {'connection': <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f7c95001898>, 'cursor': <django.db.backends.utils.CursorWrapper object at 0x7f7c8e278898>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
>               return self.cursor.execute(sql)
E               psycopg2.errors.ForeignKeyViolation: insert or update on table "core_reaction" violates foreign key constraint "core_reaction_user_id_968339ea_fk_core_user_id"
E               DETAIL:  Key (user_id)=(111) is not present in table "core_user".

/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:82: ForeignKeyViolation

The above exception was the direct cause of the following exception:

self = <django.test.testcases.TestCase testMethod=__init__>

    def _post_teardown(self):
        """
        Perform post-test things:
        * Flush the contents of the database to leave a clean slate. If the
          class has an 'available_apps' attribute, don't fire post_migrate.
        * Force-close the connection so the next test gets a clean cursor.
        """
        try:
>           self._fixture_teardown()

/usr/local/lib/python3.6/site-packages/django/test/testcases.py:1009:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.6/site-packages/django/test/testcases.py:1177: in _fixture_teardown
    connections[db_name].check_constraints()
/usr/local/lib/python3.6/site-packages/django/db/backends/postgresql/base.py:246: in check_constraints
    self.cursor().execute('SET CONSTRAINTS ALL IMMEDIATE')
/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:67: in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:76: in _execute_with_wrappers
    return executor(sql, params, many, context)
/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
/usr/local/lib/python3.6/site-packages/django/db/utils.py:89: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <django.db.backends.utils.CursorWrapper object at 0x7f7c8e278898>, sql = 'SET CONSTRAINTS ALL IMMEDIATE', params = None
ignored_wrapper_args = (False, {'connection': <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f7c95001898>, 'cursor': <django.db.backends.utils.CursorWrapper object at 0x7f7c8e278898>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
>               return self.cursor.execute(sql)
E               django.db.utils.IntegrityError: insert or update on table "core_reaction" violates foreign key constraint "core_reaction_user_id_968339ea_fk_core_user_id"
E               DETAIL:  Key (user_id)=(111) is not present in table "core_user".

/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:82: IntegrityError

Updated pytest gang:

But the error is still present.
Also, there is another thing that I forgot to mention.

class A(Model):
    pass

class B(Model):
    a = ForeignKey(A, on_delete=CASCADE)

assert A.objects.all().count() == 0  # ok
try:
    b = B.object.create(a_id=1)
except IntegrityError:
    # will go here only if I run this code outside of pytest
    print('error')
    b = None
    # ... another logic to deal with situation
assert b is None  # will fail in pytest but will work if I run this code normally
assert b.a_id == 1  # will work in pytest
assert A.objects.count() == 0  # will work in pytest

Expected behavior

If there is no object A -> IntegrityError is thrown -> deal with it (eg create object A with id=1 and rerun).

Reality

Object B is created anyway despite having a foreign key constraint, IntegrityError (Key (a_id)=(1) is not present in table "app_a") is thrown but is not fetched by try/except. Everything is messed up.

[graingert]

have you tried:

# tests.py
from django.db import transaction, IntegrityError
import pytest

@pytest.mark.django_db
def test_integrity():
    with pytest.raises(IntegrityError):
        with transaction.atomic():
            B.objects.create(a_id=1)

[graingert]

https://docs.djangoproject.com/en/2.2/topics/db/transactions/#controlling-transactions-explicitly
Note particularly the admonition:

Avoid catching exceptions inside atomic

I believe you're seeing this because by default @pytest.mark.django_db tests run in an implicit atomic block

[vanyakosmos]

I've tried to wrap code into @transaction.atomic() like in your example, but it didn't work.

Errors are the same.

[blueyed]

@vanyakosmos
btw: pasting code is usually preferred to screenshots.

I think you need to use @pytest.mark.django_db(transaction=True) then probably. Otherwise you still have the outer atomic transaction.
You could also try to close that one before instead (since transactional tests are slow in general).

[vanyakosmos]

@pytest.mark.django_db(transaction=True) fixed everything. I didn't even need to add inner @trasaction.atomic. Thank you.

[blueyed]

For reference, this can be tested without transaction=True (which is slow):

@pytest.mark.django_db
def test_request_integrity():
    from django.db import IntegrityError
    from django.db import connection

    B.objects.create(a_id=1)
    with pytest.raises(IntegrityError) as excinfo:
        connection.check_constraints()
    assert 'Key (a_id)=(1) is not present in table "app_a"' in str(excinfo.value)

Django calls this itself in _fixture_teardown (for all DB connections): https://github.com/django/django/blob/2c66f340bb50ed6790d839157dff64456b497a43/django/test/testcases.py#L1171-L1177