Mongoengine: Transaction API

[bagerard]

I'd like to open the discussion on the interface that we want to use for supporting the use of transactions/sessions within MongoEngine.

Items to keep in mind:

• Session needs to be used in all queries (write/update obviously but also any read queries)
• There can be multiple sessions used concurrently, (e.g: 1 per database, not sure we want to support multiple sessions within 1 database)
• Need to keep in mind the signals (pre/post_save, etc) that could benefit from this
• Need to keep in mind the deletion rules (CASCADE, etc) that could benefit from this
• Need to keep in mind that different databases can be connected to different models (+things like switch_database)

Assuming a test class like

class Person(Document):
    name = StringField()

I see 3 options:
1) Explicitly pass the session object to each and every queryset/queries

with start_session_with_transaction() as session:
    person = Person()
    person.save(session=session)

    Person.objects(name='test', session=session)
    Person.objects.aggregate([a_pipeline], session=session)
    Person.objects().update(name="Updated Test User", session=session)
    Person.objects.insert(Person(), session=session)

Pros:

• Explicit

Cons:

• Users are forced to pass the session around in the whole code base
• Person.objects(name='test', session=session) looks a bit weird
• Means that we need to account for contradictory cases like Person.objects(name='test', session=session1).update(name='test2', session=session2)

2) Allow to bind a Queryset to a session with Document.objects().session(session)

with start_session_with_transaction() as session:
    person = Person()
    person.save(session=session)

    # Binds a session to a queryset
    qs = Person.objects().session(session)
    qs.filter(name='test')
    qs.count()
    qs.aggregate([your_fancy_pipeline])

    Person.objects().session(session).update(name="Updated Test User")
    Person.objects().session(session).insert(Person())

Pros:

• more or less explicit

Cons:

• Users are forced to pass the session around in the whole code base, every class involved in a request needs to be manually bound to the session
• I find it error prone, its very easy to forget it
• Person.objects().session(session).insert(Person()) looks quite ugly

3) Implicitly bind all underlying operations to a session (à la django)

with run_in_transaction(alias="default"):
    person = Person()
    person.save()

    qs = Person.objects()
    (...)

Pros:

• simple to use
• compatible with existing code base - i.e: easy adoption by users
• mimic django (django being an inspiration for this project)
• less error prone since user only has to define a

Cons:

• very implicit
• does not comply with a repository pattern / unit of work approach

- unclear how we'll manage different databases

Of course, we would provide a decorator in addition to the context manager.

Personally I initially thought that 2) would be the best but I quickly came across a few examples that looked unnatural, so I'd suggest to go for 3)

Let me know what you think! (especially @wojcikstefan @erdenezul but happy to gather more feedbacks)

[angieduan]

Option 3 looks better.

[xqzhou]

Option 3 looks good

[luking]

I like option 3

[SuCice]

I like option 3 too

[ajdas-code]

I vote for option 3 too

[ibmboy19]

Vote for option 3!

[coughlinjake]

Is there any coding progress? I'm REALLY interested in using MongoDB's transactions.... so interested that I'd offer to code it but I'm still fairly new to Python.

[bagerard]

Thanks for your interest @coughlinjake so far, I've only do a small proof of concept. I was in fact waiting to have some decisions from the maintainer (ping @wojcikstefan). I'm pretty busy now but if I manage to start at some point on this, I'll check if we can share the workload

[ajdas-code]

I can help you there. If you are ok I can share the load. Do we have any design or spec available or need to write one?

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Jason Coughlin [e-mail]
Sent: Thursday, August 6, 2020 5:22:10 AM
To: MongoEngine/mongoengine [e-mail]
Cc: Das, Ajitesh [e-mail]; Comment [e-mail]
Subject: Re: [MongoEngine/mongoengine] Transaction API (#2248)

Is there any coding progress? I'm REALLY interested in using MongoDB's transactions.... so interested that I'd offer to code it but I'm still fairly new to Python.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/MongoEngine/mongoengine/issues/2248#issuecomment-669894053, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AO6S4FH7KMHPIKNJBSM5YSDR7KN7FANCNFSM4KI3XZAQ.

[ajdas-code]

looks like, we have some helps at our disposals. If we have decided on the approach, we can go ahead start the project.
Who takes the decision here? Can he please fill in.
Thanks!
Ajitesh

--

A person should not be too straight or too honest. Straight trees are cut first and honest people are screwed first.
-Chanakya

---

From: Henry [e-mail]
Sent: Monday, July 13, 2020 10:29 PM
To: MongoEngine/mongoengine [e-mail]
Cc: Das, Ajitesh [e-mail]; Comment [e-mail]
Subject: Re: [MongoEngine/mongoengine] Transaction API (#2248)

Vote for option 3!

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/MongoEngine/mongoengine/issues/2248#issuecomment-657975452, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AO6S4FAL6WRWG64I2MVK7I3R3PUKBANCNFSM4KI3XZAQ.

[rjrakshit24]

Hi @bagerard , I am not sure if I am on right track or not, but I modified the library for transaction support. Following is the snippet for using the session.

k=get_connection()
  with k.start_session() as mongosession:
    with mongosession.start_transaction():
      prevData=getToLogValue(dtype=cls._ParentClass._fields.get('XYZ'),value=p['XYZ'])
      prevData={cls._ParentClass._toForm.get('XYZ'):prevData}
      item.update(session=mongosession, **n)
      newData=getToLogValue(dtype=cls._ParentClass._fields.get('XYZ'),value=n['XYZ'])
      newData={cls._ParentClass._toForm.get('XYZ'):newData}
      itemLog=cls._ItemLog(user=user, action='updated', newData=newData, prevData=prevData, recId=item)
      itemLog.save(session=mongosession)

P.S.:

• It is not mandatory to provide a session variable when you don't want sequential execution (transaction execution).
• CASCADE operation is also handled in case of delete
• Have used for read as well.

[Baloneo]

Is there any coding progress? Hope to realize this feature. Thanks

[shellcodesniper]

option 3 will be gooooooood

[Baloneo]

Will this feature be merged?