Mongoose: Mongoose hooks triggered before transaction commit

[fr6nco]

bug

Post hooks on object are executed when creating an object to the database before the transaction is committed.

const session = await this.organizationModelService.getModel().db.startSession();
    try {
      await session.withTransaction(async () => {
        //Create organization
        orgObj = await this.organizationModelService.getModel().create([org], { session: session });

        const roles = [
          {
            name: `user-role-${context._id}`,
            organization: orgObj[0]._id,
            users: [context._id],
            userRole: true,
            locked: false,
            permissions: [],
          },
        ];

        const roleResult = await this.roleModelService.getModel().create(roles, { session: session });
      });

On creation of an object I want to create creation of corresponding objects, which have a post save hook attached. As soon as create is called, the hooks are fired, while the objects are not yet saved in DB.

Is this a desired behaviour in such case? Any ideas how to force hooks after the transaction was commited to the DB?

Im using mongoose version 5.8.5

Thanks

[vkarpov15]

Yes this is desired behavior - await Model.create(doc) waits until post hooks are done.

If you want to run operations after a transaction is committed, I'd recommend wrapping those operations in an instance method on your roleModel, and run that method after withTransaction() is done.

    let roleResult;
    try {
      await session.withTransaction(async () => {
        //Create organization
        orgObj = await this.organizationModelService.getModel().create([org], { session: session });

        const roles = [
          {
            name: `user-role-${context._id}`,
            organization: orgObj[0]._id,
            users: [context._id],
            userRole: true,
            locked: false,
            permissions: [],
          },
        ];

        roleResult = await this.roleModelService.getModel().create(roles, { session: session });
      });

      await roleResult.myCustomMethod();

[buccfer]

What happens if the model already has post hooks that are being executed outside transactions? Should you remove those hooks because in a transaction they don't work that way?

Despite one could workaround this issue, it makes sense that post hooks execute after the transaction was committed and not before. So definitely @fr6nco 's use case is a common one.

[vkarpov15]

@buccfer what are some cases where you want to execute a post hook after a transaction is committed?

[buccfer]

For example:

If you have a transaction that creates documents of different collections, and one of those documents belongs to an Activity collection.

If the Activity model has some post save hooks related to sending notifications, those notifications will be sent before the transaction commits. This leads to 2 possible undesired situations:

1) The transaction fails and rolls back: You sent notifications of an operation that never took place.

2) The transaction succeeds: If any of the post save hooks depend on the resources created by the transaction to exist, then the post save hook will fail. To make things work as expected, one should pass the session object wherever a query needs to be executed. So this approach doesn't scale.

These are the main reasons that I see that could be a good addition to the library :)

[guillegette]

Joining this thread as I am as well quite surprised to see the post hook being execute before the transaction was committed. Here are my reasons:

1. Model post hooks exist before transactions were supported in Mongo, so as developers we are already counting on those where a document is created/saved/etc, something else happen that depends on that specific document details, like _id. For this reason calling a post hook before the transaction, in my opinion, is breaking the original implementation of post hooks.

2. Transactions can fail, so a roll back will happen, which will cause the post hooks to behave very unpredictable. If post hooks are called after the transaction is committed then there is nothing to worry.

Finally, if calling post hooks on all models involve in the transaction it's a pain or difficult to implement, then we could just don't call any hooks within transactions, which will allow the developer to make the necessary changes. The current implementation makes it harder for developer to switch to transactions as we have an unexpected behavior.

Hope all this makes sense and you guys can re-consider this implementation.

[vkarpov15]

There are also cases where you would want post save hooks to execute within the transaction. Say, if updating a denormalized property.

I still think the right solution here is to implement your transaction as a model static, and put any “post transaction” hooks as post hooks for your custom static.

[buccfer]

There are also cases where you would want post save hooks to execute within the transaction. Say, if updating a denormalized property.

@vkarpov15 Suppose I'd like to execute a post hook within the transaction. How would I pass the session to the post hook operations so they are actually part of the transaction?

[guillegette]

I still think the right solution here is to implement your transaction as a model static, and put any “post transaction” hooks as post hooks for your custom static.

If you were designed a solution from scratch I agree, but if you already have models with post hooks implemented and you are now adding a transaction, you would need to do quite a bit of refactor to adapt to this.

Would you mind sharing any of the reasons behind implementing this in this way ? Is it a particular technical limitation?

[vkarpov15]

@buccfer you can access the current session using the Document#$session() functiomn in middleware: https://mongoosejs.com/docs/api/document.html#document_Document-$session

schema.post('save', function(doc) {
  // Get the current session
  const session = this.$session();

  // Another way to get the current session
  const session = doc.$session();
});

@guillegette there are numerous reasons. One of the most glaring ones is that the promise save() returns doesn't resolve until all post hooks are done. So if we wanted to, say, make post save hooks run after the current transaction is done, we would have to break the assumption that await doc.save() waits for all post hooks to finish. Either way, you would have to audit all your post hooks to ensure you know which ones should run after the transaction, versus which ones should run after the individual save() operation.