Mongoengine: How to mock mongodb with mongoengine in unit test?

[zhaozhiming]

HI ALL,
I use mongoengine in my project, it's pretty good.
But when I write unit test with mongoengine and use mongomock to mock mongodb, I find they did not work together.
Could someone tell me how to mock mongodb with mongoengine, or what mock framework should be use?
Thanks.

[MRigal]

Hi @zhaozhiming
I must admit that I always start a local mongo for my tests and never thought about mocking it... I doubt it is really implemented or working right now. Let's see if some other users have different experiences.
I guess it would be worth writing a new module to integrate to mongoengine a wrapper of mongomock. I don't think so much work would be required. In case you are interested, it would be a great contribution!
Cheers

[gbrennon]

Someone got an answer about how to test mongoengine apps using some mock stuff?

[moonso]

:+1:

[andresdouglas]

Is the integration with mongomock still working/supported? Couple issues:

If you import get_connection as instructed in the docs, you get all sorts of errors in your tests.
Simply doing a
connect('mongoenginetest2', host='mongomock://localhost') doesn't seem to reset the connection, and data is persisted between tests. Maybe I'm not using it properly?

[thedrow]

Data is persisted between tests in mongomock.
Drop the databases as you would do with a live MongoDB connection as well.

[codermpl]

I'm having trouble getting this to work as well. I try to recreate my database after every test. I can drop the database once, but if I recreate the database and try to drop it again, nothing happens. It seems like after calling drop_database() the first time, the databases are never re-added to the _databases dictionary in the collection.

I've tried manually calling disconnect() or close(), but it doesn't change anything. If I run with a real mongo instance, it drops the database every time.

@pytest.fixture(scope="function")
def mongoengine_connection(request):
    """Create MongoEngine connection to MongoMock"""
    connection = connect(db='mongotest', host='mongomock://localhost')

    def fin():
        """Drop the database at the end of the fixture"""
        print("drop database")
        connection.drop_database('mongotest')
        return

    request.addfinalizer(fin)
    return connection

[thedrow]

Is it a bug on our side or mongomock's side?

[ericchapman]

The following reproduces the issue. Will see if I can debug further.

test.py

from mongoengine import *

class TestObject(Document):
    param = StringField()

connection = connect('mongoenginetest', host='mongomock://localhost')

test_object = TestObject(param="value")
test_object.save()

print "Count should be 1, was " + str(TestObject.objects.count())

connection.drop_database('mongoenginetest')

print "Count should be 0, was " + str(TestObject.objects.count())

test_object = TestObject(param="value")
test_object.save()

print "Count should be 1, was " + str(TestObject.objects.count())

connection.drop_database('mongoenginetest')

print "Count should be 0, was " + str(TestObject.objects.count())

response

%> python test.py 
Count should be 1, was 1
Count should be 0, was 0
Count should be 1, was 1
Count should be 0, was 1

versions

%> pip freeze | grep -e mongomock -e pymongo -e mongoengine
mongoengine==0.10.6
mongomock==3.4.0
pymongo==3.2.2

[ericchapman]

I got it. The issue is in MongoMock. The problem is that in "database.py" and "mongo_client.py" they are clearing the documents on "drop_database" and THEN deleting the collections and the database objects. The issue is that the mongengine "_collections" attribute in the "Document" object has handles to the original DB and Collections so the objects are holding on to stale data.

I observed when the normal "mongodb" connection is used the DB is sent a "drop_database" command but the object remains. I hacked my local copy of the mongomock library and was able to get this working properly

mongo_client.py

• Remove line 71 "del self._databases[database_name]"
• Remove line 77 "del self._databases[name_or_db]"

*_database.py_

• Remove line 51 "del self._collections[name]"
• Remove line 58 "del self._collections[name_or_collection]"

[thedrow]

@ericchapman Can you create a new issue about this or a PR so we can discuss this further?

[ericchapman]

@thedrow Yeah, I was in the middle of doing a PR last night but having trouble getting the tests to run. Opened the following issue

https://github.com/vmalloc/mongomock/issues/238

[Folyd]

It seems like this is the same bug what i met. here is the issue https://github.com/mongomock/mongomock/issues/281

[corydodt]

I just realized this is probably the same bug I also opened, https://github.com/mongomock/mongomock/issues/233