Mongoose: `nModified: 1` returned by `.update()` when $addToSet makes no modification

[RobinJayaswal]

current behavior

User with id 1 has foo: ['bar'] currently in db. I run:

User.update({_id: 1}, {$addToSet: {foo: 'bar'}}).exec().then(res => {
   console.log(res)
}

And it outputs { n: 1, nModified: 1, ok: 1 }. But the document was not actually modified since it did not add anything to foo, as bar was already in the array.

expected behavior

I want to be able to check whether foo array already contained 'bar', as this determines whether I continue. In a more concrete example, if the user gets points for doing an action, and completed actions are stored in an array, I want to be able to check if the user had already taken the action so I can avoid giving them points. I am aware I could just fetch the user from the db and check, but then if a user hits the api with concurrent requests using a script, they could get points multiple times. The $addToSet method with a check for updated or not seems like a great atomic way to do this, but it always returns nModified: 1 so I am unable to currently.

Versions
Mongoose 4.13.9, MongoDB 3.6.2

[lineus]

@RobinJayaswal do you have timestamps turned on by chance? or something else that might be updating a path in the document besides what your example shows? I tried to replicate what you're seeing with the following code, but it doesn't show the same behavior:

6861.js

#!/usr/bin/env node
'use strict';

const assert = require('assert');
const mongoose = require('mongoose');
mongoose.Promise = global.Promise;
mongoose.connect('mongodb://localhost:27017/test', { useMongoClient: true });
const conn = mongoose.connection;
const Schema = mongoose.Schema;

const schema = new Schema({
  foo: [String]
});

const Test = mongoose.model('test', schema);

const test = new Test({ foo: ['bar'] });

async function run() {
  await conn.dropDatabase();
  await test.save();
  let res = await Test.update({ _id: test._id }, { $addToSet: { foo: 'bar'} });
  assert.strictEqual(res.nModified, 0);
  console.info(`Test passed on ${mongoose.version}`);
  return conn.close();
}

run();

Output:

issues: ./6861.js
Test passed on 4.13.9
issues:

[RobinJayaswal]

@lineus i do in fact have timestamps on. I suppose I expected the timestamps to only update in a real update, but now that I think about it they are probably implemented in a pre hook.

[vkarpov15]

@RobinJayaswal that's right, timestamps are implemented using a pre hook. That means that updatedAt represents the last time you tried to update the document, updatedAt will get bumped even if nothing changed.

[dandv]

I just ran into the same issue today. Quite a bit counter-intuitive, and it also means

• we can't depend on the .nModified to decide if an $addToSet or $pull actually changed the array
• we can't rely on updatedAt since it will falsely indicated updates that didn't change anything

Are there any workarounds?

[vkarpov15]

@dandv the only workaround right now would be to skip timestamps for the updates that you want to use nModified for, like Model.findOneAndUpdate(filter, update, { timestamps: false }). We'll reopen this and consider it for a future release.

[vovarevenko]

I faced this problem here: #9452

@AbdelrahmanHafez left a good comment:

The reason for that being we don't have access to the document in the database by the time we do an updateOne or updateMany, so we can't tell what actually changed without degrading the performance by finding the whole document and comparing the state before/after the update.

Maybe you should separate the targeting update fields and the companion update fields at the mongodb level?

Current usage works like this:

db.cats.updateMany({}, {
  '$setOnInsert': { createdAt: new Date("Sat, 26 Sep 2020 17:56:54 GMT") },
  '$set': {
    updatedAt: new Date("Sat, 26 Sep 2020 17:56:54 GMT"),
    name: 'Zildjian',
  },
})

New usage could be like this:

db.cats.updateMany({}, {
  '$setOnInsert': { createdAt: new Date("Sat, 26 Sep 2020 17:56:54 GMT") },
  '$setOnUpdate': { updatedAt: new Date("Sat, 26 Sep 2020 17:56:54 GMT") },
  '$set': { name: 'Zildjian' },
})

Thе $setOnUpdate field means what data needs to be updated along with the data updated by the $set field.

• n = number of matched docs
• nModified = number of modified by the $set field

What do you think?

[AbdelrahmanHafez]

@vovarevenko $set and $setOnInsert are native MongoDB operators, what you're proposing sounds reasonable, but I am not sure such a $setOnUpdate operator exists on MongoDB.

[vovarevenko]

but I am not sure such a $setOnUpdate operator exists on MongoDB

These are just thoughts on how the functionality could be expanded. Maybe, we can report this to MongoDB developers?

[AbdelrahmanHafez]

This would be reasonable if such an operator existed, feel free to open a ticket on MongoDB's jira with that suggestion.
Meanwhile, I think things will remain the same on mongoose side of things.

[vovarevenko]

I created a ticket. You can vote for it if you are interested: https://jira.mongodb.org/browse/SERVER-51208

[vovarevenko]

Solution for a similar problem from JIRA MongoDB:

db.runCommand({
  update: "test",
  updates: [{
    q: { a: 1 },
    u: [{ $set: {
      a: 1,
      b: 2,
      created_at: {
        $cond: [ { $eq: [ { $type: "$_id" }, "missing" ] }, ISODate(), "$created_at" ]
      },
      updated_at: {
        $cond: [ { $or: [ $ne: [ "$a", 1 ], $ne: [ "$b", 2 ] ] }, ISODate(), "$updated_at" ]
      }
    } }],
    upsert: true
  }],
  writeConcern: { w: "majority", wtimeout: 5000 }
})

I have not run tests.

@AbdelrahmanHafez, is this a decision you had in mind when you talked about performance? Do I need to test this, or will such a solution not be added to Mongoose?

Docs for this: https://docs.mongodb.com/master/reference/command/update/index.html#syntax
Tickets i use: SERVER-13578, SERVER-42084, SERVER-51208.

Some fields like $setOnUpdate can only be added as syntactic sugar. (comment)

Thank you!

[AbdelrahmanHafez]

@vovarevenko There are two issues with that approach:

The first is that update pipelines are supported on MongoDB 4.2+, and building that to be backward compatible in mongoose can be a bit confusing for users, and tricky for the developers.

The second is that I can't see how we'd use that as a generic solution, the example above adds fields if they're missing; we could, of course, check if the values are equal for primitive types, but it would be tricky to implement with $pushing objects into an array or $addToSet, or using the positional operator.

IMHO the only reliable way of achieving that without the support of a native $setOnModified operator from MongoDB, is by finding the document(s) before the update, after the update, convert both into JSON, compare them, and do another update with the updatedAt field. This approach has a terrible impact on performance.

Out of curiosity, why is this important to you? What's your use case?

[vovarevenko]

Out of curiosity, why is this important to you? What's your use case?

Actually, it doesn't really matter to me. I ran into this problem when I saw that the n and nModified fields are always equal (and useless) when using timestamps. I knew that there is a $setOnInsert functionality, and it seemed illogical to me that MongoDB does not do a similar functionality for updating fields.

Anyway, the discussions from JIRA MongoDB and your comments were helpful. Thanks!