Mongoose: Model.update with $rename does not seem to work

[JedWatson]

note: I'm using 3.8.33, so if this is resolved in 4.x my apologies, I'm not able to upgrade the project I'm working on yet...

I've got a field in a collection that I want to rename, and have been able to do it successfully from the mongo cli using the following:

db.agents.update({}, { $rename: { "state": "contactStatus" } }, { multi: true })

However, I want to write this into an update script so it is applied to my staging and production environments when the new version of my app is released.

Since I'm using mongoose, it seems I should be able to use the Model.update method to achieve the same:

AgentModel.update(
  { },
  { $rename: { state: 'contactStatus' } },
  { multi: true },
  function(err, raw) { console.log(err, raw); }
);

This doesn't seem to have any effect on the data, however, and the value of raw when logged to the console is 0.

Should this update query be supported by Mongoose? I know that the update query is rewritten to { $set: { update } } when overwrite: false (default), but the docs say:

All top level keys which are not atomic operation names

... which I had expected would mean $rename would work as expected.

Not sure if this is related to #3027? No error is being thrown.

Any guidance would be appreciated, and if there's further debug information I can provide please let me know.

[JedWatson]

Update: I followed the thread through to #1845 and the solution mentioned in that issue, using AgentModel.collection.update, _does_ work. So this seems like a bug.

[vkarpov15]

That fix was added in 4.0.5, but wasn't backported to 3.8.x. Will backport it for you :)

[JedWatson]

Thanks @vkarpov15!

[vkarpov15]

Fixed in https://github.com/Automattic/mongoose/commit/3edbae5eea8f43642d738caa1f41b59d2b13a3d1

[JedWatson]

@vkarpov15 cheers for the backport, much appreciated! :smile:

[vkarpov15]

Always happy to help :) let me know if you need help migrating to 4.0.x - would be great to get keystone on the new version, especially since we're planning on stopping regular 3.8.x releases in September.

[JedWatson]

@vkarpov15 thanks! good to know the 3.8.x releases will be ending in September. We're hoping to get our major UI rebuild out later this month, which will include some breaking changes, I was planning to bundle the Mongoose 4.x update into that (since a mongoose major really needs a keystone major)

If we're not on track by the end of august I'll push it out without all the updates I've got planned, so everyone can get onto the latest 4.x version of Mongoose.

[vkarpov15]

No worries. If there are any bugs reported with 3.8.x I'll fix them as necessary, that's just the last time we'll be actively working on 3.8.x. Let me know if you need help with migrating, always happy to help.

[yaring]

Here with [email protected] $rename doesn't works, maybe bug came back?
with Model.collection.update works find.

[vkarpov15]

@yaring can you show some code that shows what you're trying to do?

[yaring]

@vkarpov15 Sure.

    BuildingBlock.update({}, { $rename: { layout: 'content' } }, { multi: true }, function(err, blocks) {
      if(err) { throw err; }
      console.log('done!');
      process.exit();
    });

[vkarpov15]

Is it giving you an error? Or just not updating properly?

[yaring]

No error, just not updating properly.
I'm using Model.collection and it works.

[vkarpov15]

Hmm can you double-check your version of mongoose for me and run mongoose debug mode require('mongoose').set('debug', true); to show me what query is being sent to mongodb?

[akoskm]

I can confirm this bug. Here is the output from one of my migrations script after turning on .set('debug', true);.

1. with .collection:

./node_modules/.bin/migrate up : 0020-vendor.js Mongoose: users.update({}) { '$rename': { customRequests: 'vendorRequests' } } { multi: true } rename finished { result: { ok: 1, nModified: 24, n: 26 }

1. without .collection:

./node_modules/.bin/migrate up : 0020-vendor.js rename finished { ok: 0, n: 0, nModified: 0 }

Here is the relevant part of the migration script:

  app.db.models.User.collection.update({}, {
    $rename: {
      customRequests: 'vendorRequests'
    }
  }, { multi: true }, function (err, users) {
    if (err) {
      console.log(err);
    }
    console.log('rename finished', users);
    next();
  });

Tested with 4.4.3.

[akoskm]

With 4.4.10 the migration script containing .collection simply hangs.

Now $rename doesn't work at all.

[akoskm]

@vkarpov15 can we expect this fix in the next release?

[vkarpov15]

Yep this fix was released with 4.4.12.

[apitts]

I experienced this same issue with 4.4.19 - without .collection $rename had no effect but with .collection it succeeded.

[akoskm]

@apitts Could you create a test and reopen this issue if the bug still exists?

[vkarpov15]

Can you provide some sample code? We have tests in place for $rename and they're passing...

[apitts]

Apologies for the delay - yes, will try to replicate what I experienced with a simplified test case. If the tests are passing it is more likely my error though.

[apitts]

I have not been able to replicate what I experienced with a simple test case, all my attempts have passed. I can only conclude that I must have been using an older version (though I had printed the version to make sure). My apologies for the false alarm.

[vkarpov15]

No worries, thanks for taking the time to try to repro

[valVk]

I have had that issue mongoose: 4.7.4.

import * as db from './../models';

//this works
exports.up = function(next) {
  db.User.update({}, { $rename: { password: 'hashedPassword' } }, { multi: true }, function(err, blocks) {
      if (err) { throw err; }
      console.log('done!');
      process.exit();
    });
  next();
};

```bash
$ npm run migrate up [11:43:33]

tsc -p ./server && ./node_modules/.bin/migrate -c ./dist/server "up"

Connection established
Mongoose: devicetokens.ensureIndex({ deviceToken: 1, user: 1 }, { unique: true, background: true })
Mongoose: questions.ensureIndex({ title: 1 }, { unique: true, background: true })
Mongoose: surveyresults.ensureIndex({ survey: 1, user: 1, question: 1 }, { unique: true, background: true })
Mongoose: userprofiles.ensureIndex({ user: 1 }, { unique: true, background: true })
Mongoose: users.ensureIndex({ mobile: 1 }, { background: true })
Mongoose: questions.ensureIndex({ survey: 1, sort: 1 }, { unique: true, background: true })
Mongoose: users.ensureIndex({ zip: 1 }, { background: true })
up : 1484125955304-change-password-filed.js
Mongoose: users.update({}, { '$rename': { password: 'hashedPassword' } }, { multi: true })
Mongoose: users.ensureIndex({ isAdmin: 1 }, { background: true })
Mongoose: users.ensureIndex({ email: 1 }, { unique: true, background: true })
migration : complete
```

//this not works
exports.down = function(next) {
  db.User.update({}, { $rename: { hashedPassword: 'password' } }, { multi: true }, function(err, blocks) {
      if (err) { throw err; }
      console.log('done!');
      process.exit();
    });
  next();
};

$ npm run migrate down                                                                                                         [11:44:43]
> tsc -p ./server && ./node_modules/.bin/migrate -c ./dist/server "down"

Connection established
Mongoose: devicetokens.ensureIndex({ deviceToken: 1, user: 1 }, { unique: true, background: true })
Mongoose: questions.ensureIndex({ title: 1 }, { unique: true, background: true })
Mongoose: surveyresults.ensureIndex({ survey: 1, user: 1, question: 1 }, { unique: true, background: true })
Mongoose: userprofiles.ensureIndex({ user: 1 }, { unique: true, background: true })
Mongoose: users.ensureIndex({ mobile: 1 }, { background: true })
Mongoose: questions.ensureIndex({ survey: 1, sort: 1 }, { unique: true, background: true })
Mongoose: users.ensureIndex({ zip: 1 }, { background: true })
  down : 1484125955304-change-password-filed.js

But it works as expected if I'm adding collection reference

[vkarpov15]

Try doing db.User.update({}, { $rename: { hashedPassword: 'password' } }, { multi: true, strict: false }, function(err, blocks) { }); . I presume hashedPassword is not defined in your schema so mongoose will strip that out by default unless you do strict: false

[natanavra]

Mongoose was deleting the field from my $rename query since I already renamed the field in the Schema.
So I used {strict: false}, that way it works with Model.update.
I would think this should be the default behavior, since we're not planning on leaving both fields in the Schema.

P.S. works well with Model.collection.update but that's deprecated.