Mongoose: ParallelSaveError: Can't save() the same doc multiple times in parallel

[noviewpoint]

Do you want to request a feature or report a bug?
feature

What is the current behavior?
When saving the same model instance from multiple different contexts (multiple different parts of application) setting different values to this model instance before calling save, mongoose produces ParallelSaveError.

I'm writing rather complicated logic in my application that is based of working with mongoose models and using the save() method. Recently I found out that even if mongoose produces ParallelSaveError when trying to save same model in parallel, only the last updated value is saved aftermath (as if the ParallelSaveError errors are ignored).

If you run this sample code, the values setted to the model from the last call of saveNew function ( // step 5 ) will be stored to DB taking precedence to previous steps:

```const mongoose = require("mongoose");
mongoose.connect(
"mongodb://localhost:27017/test",
{ useNewUrlParser: true }
);
const Schema = mongoose.Schema;
const exampleSchema = new Schema({
keyA: Number,
keyB: Number
});
const exampleModel = mongoose.model("example", exampleSchema);
const newDocument = exampleModel({ keyA: 1, keyB: 2 });
newDocument.save().then(() => {
saveNew({ doc: newDocument, keyA: 4, keyB: 0 }); // step 1
saveNew({ doc: newDocument, keyA: 6, keyB: 8 }); // step 2
saveNew({ doc: newDocument, keyA: 9, keyB: 0 }); // step 3
saveNew({ doc: newDocument, keyA: 7, keyB: 3 }); // step 4
saveNew({ doc: newDocument, keyA: 6, keyB: 0 }); // step 5
async function saveNew({ doc, keyA, keyB }) {
try {
if (keyA) {
doc.keyA = keyA;
}

  if (keyB) {
    doc.keyB = keyB;
  }
  await doc.save();
  console.log("end of saveNew for", keyA, keyB, doc.keyA, doc.keyB);
  // end of saveNew for 4 0 6 3 gets logged
} catch (err) {
  if (err.name === "ParallelSaveError") {
    console.log("There was a parallel save error for", keyA, keyB);
  }
}

}
});
```
So even though only the first step gets resolved ok and the others fall into catch, values from the last step are retained in the model instance and in the DB.

What is the expected behavior?

For me that IS the desired behaviour but I wonder if it is correct. Why the error warnings if the last updated value gets written to database as you would expect? Is there some kind of queue mechanism in mongoose to retain all model values before separate .save() calls?

Can someone please clear some confusion on this topic?

What are the versions of Node.js, Mongoose and MongoDB you are using? Note that "latest" is not a version.

Mongoose: ^5.6.12
Node.js: v8.9.0
MongoDB: pre v4

[vkarpov15]

The issue is that you're making all the changes to the document in the same tick of the event loop. doc.save() doesn't start until the next tick of the event loop after you call save(), so you can still make further changes to the document in the same tick. I admit this behavior is confusing, and it's something that we'll figure out how to improve.

But then the question is why do you need to call save() on the same document multiple times in the same tick of the event loop? That seems unnecessary...

[vkarpov15]

I thought more about this and I haven't been able to come up with a way to improve this behavior yet. Admittedly it is a bit confusing that you can continue to change the document after you call save(). But then again, you should be able to modify the document in pre save middleware...

I'm going to push this back a couple of milestones and see if I can come up with something.

[noviewpoint]

The issue is that you're making all the changes to the document in the same tick of the event loop. doc.save() doesn't start until the next tick of the event loop after you call save(), so you can still make further changes to the document in the same tick. I admit this behavior is confusing, and it's something that we'll figure out how to improve.

But then the question is why do you need to call save() on the same document multiple times in the same tick of the event loop? That seems unnecessary...

My application is a TCP client reacting to TCP server messages. Based on these messages I update documents using mongoose .save(). Sometimes I get 2 or even more messages in a very short period of time informing me to update different parts of the same document. There is also a global timer present in my application that is updating time field on documents every few seconds.

So in theory multiple .save()-s on the same documents can happen at the same time... and so do the ParrallelSaveError-s :) Is mongoose suitable for such tasks? Is it better to ditch mongoose documents and use plain pojos with native mongo findOneAndUpdate queries?

[vkarpov15]

Why do multiple messages update the same document instance at the same time though? It is generally better to either:

1) Load the document from the database, modify it, and then save() it. Or,
2) Use findOneAndUpdate() to modify the document as you suggested

You can read more about updating documents in mongoose here.

Mongoose isn't really designed to call save() in parallel multiple times on the same document instance. We can make some improvements, but that isn't something Mongoose does well right now.

Out of curiosity, why don't you load a separate copy of each document in response to each message? Is that for performance?

[noviewpoint]

Out of curiosity, why don't you load a separate copy of each document in response to each message? Is that for performance?

Because I'm pulling mongoose documents from memory and not from database. This "in-memory cache" is nothing but a Javascript variable and is intended to optimize reads.

My superior told me that this cache and database should be as synchronized as possible. I really like working with mongoose documents instead of pojos, so it was my decision to keep mongoose documents inside the cache. Also, there is no danger that someone else would do database operations on documents present in my cache. My app is the only one accessing the database.

The more I think about this problem, the more I feel like I can live with ParrallelSaveError if it only happens every once in a while ...

Otherwise I'm very grateful for your help.

[lacivert]

I have a similar need. I want to store a status history in db and there are lots of status change on the document. So I get parallel save error. I understand the race condition but there is no need to race and it can wait for the "next tick" to get the new member of the array like this:

{
  // _id: 
  history: [
    status1, status2, status3, // and in a very short time many more statuses would come
  ]
}

How can I save this kind of document without getting parallel save error?

[vkarpov15]

@lacivert either load a separate document from the database each time, or wait until all the changes are done before calling save().

[anup36]

@vkarpov15 am also facing a similar kind of issue. So I have a web app where users can create their blogs. On the signup API I'm doing a Find query by a mobile number if user found returning if not creating a new user.
The only issue is when Am getting parallel calls span of milliseconds which leading to duplicate records creation.

const user = await Customer.findOne({ mobile });
    if (user) {

      throw new ServiceError('user exist');
    }

    const customer = new Customer(data);
    await customer.save();`

[vkarpov15]

@anup36 that shouldn't be a problem as long as you're loading the document from the database each time, like what's shown below.

app.post('/signup', async (req, res) => {
  const user = await Customer.findOne({ mobile });
    if (user) {

      throw new ServiceError('user exist');
    }

    const customer = new Customer(data);
    await customer.save();
});

This error only occurs if the exact same document instance (same reference) is saved multiple times in parallel.

If you're still facing this issue, please open a new issue and follow the issue template.

[SorinGabriel02]

Had the same problem and solved it by using .findOneAndUpdate() instead of .save()

[karankumarshreds]

Even I am facing this issue. I have to make changes to 2 documents and make a new one in a singe function. Used .save() and .findOneAndUpdate() for first two and now I am unable to create+save a new entry in the end.