Mongoose: Using String for _id locally, but ObjectId in Database

[bendytree]

Is there a way to have _id be a string on the model, but an ObjectId in MongoDB?

Basically it's a bit painful to do equality with ObjectId, and I don't get much out of it being an ObjectId on my side. For example, finding a document in a collection:

Using ObjectId

var match = _.find(documents, function(document){
  return String(document._id) === String(someId);
}); 

Using String

var match = _.find(documents, {_id:someId});

[vkarpov15]

Hmm is there a bug with ObjectID equality check? There was one in some earlier versions of 3.8.x but it should be fixed.

To get the above, you can always just use a virtual whose getter calls toString() on the document's _id.

[Spekpannenkoek]

By default, your schema should already have an id virtual (read why it exists here). You can use it to do the following: document.id === someIdAsAString.

The ObjectId object also has a method named equals: document._id.equals(someIdAsAStringOrObjectId) but this method throws an exception when the argument is not, well, a 24 character string or ObjectId.

You could technically define your id as a String in your model, add a pre-init hook to cast the DB's value to a string and a pre-save hook to cast it back to an ObjectId using something like doc.set("_id", ObjectId(doc._id), {strict: false}) (ought to work) but I wouldn't recommend it.

[bendytree]

Yeah, the problem is that .equals(...) may throw plus it doesn't mesh well with libraries like lodash.

id As A Virtual

The id virtual is a good idea, but I frequently use .lean() so I think I'd need to look at id on mongoose objects and _id on lean objects (plus _id is an ObjectId in lean queries). It ends up that lean() is the bane of my efforts.

Hooks For Conversion

I tried using the pre init/save hooks to convert between ObjectId and String (here's the full gist). Despite using strict:false my value was always forced back into a string in applySetters. I tried to fix it but started to feel like I was going too far against the grain of mongoose, plus I'm worried about edge cases like a failed save leaving me with an ObjectId.

schema.pre("save", function(next){
  // this._id is a string
  this.set("_id", mongoose.Types.ObjectId(this._id), {strict: false});
  // this._id is still a string
});

Custom Schema Type

Finally, I made a custom schema type called ObjectIdAsString. It works except that .lean() queries naturally leave the _id as an ObjectId (which basically ruins everything). Here's an example:

var Thing = mongoose.model("Thing", mongoose.Schema({
  _id: { type: "ObjectIdAsString" },
  ...
}));

var thing = new Thing();
thing._id; //this is a string like "43345823304969c878318d12"
thing.save(function(err){
  //the database now has { _id: ObjectId("43345823304969c878318d12") }
});

//fetching works like normal
Thing.findOne({ _id: "43345823304969c878318d12" }, function(err, thing){
  thing._id; //this is a string
});

//lean
Thing.findOne({ _id: "43345823304969c878318d12" }).lean().exec(function(err, thing){
  thing._id; //this is an ObjectId
});

I suppose there's no getting around the lean issue since it's designed to be untouched.

[Spekpannenkoek]

You could also just save the id twice in your model, once as the ObjectId (primary key) and once as a string (with a (unique) index). It only adds 24~32 bytes overhead.

var thingSchema = Schema({
    _id: ObjectId, // or just leave it out
    id: {
        type: String,
        getter: function(val) { return this._id.toString(); },
        unique: true
    } // add a real "id" property that, if unset, is this._id.
}, {
    id: false // disables the creation of the virtual "id" property
});

// ...

// This might not even be needed, if the id's getter executes on-save:
Thing.pre("save", function(next) {
    this.id = this._id;
    next();
});

The only caveat is that you must use Mongoose when initially saving your document.

You can now do this:

Thing.findOne({id: idAsString}); // although findById is better

if(myThing.id === otherThing.id) {}

[vkarpov15]

I'm still somewhat confused as to why using lodash with ObjectId isn't working for you. We do have some test coverage for that case with underscore - in theory, you should be able to do _.find with ObjectIds and latest 3.8.x.

[bendytree]

For me, == doesn't work between two ObjectIds and underscore comparisons don't work if my comparator is a string. Basically I don't use much from ObjectId and plain strings would solve a lot of problems. For example:

db.User.findOne({_id:"5330606182ca660000dafd9d"}, function(err, user1){
db.User.findOne({_id:"5330606182ca660000dafd9d"}, function(err, user2){

  //this logs `false`
  console.log(user1._id == user2._id);

  //this logs `1`
  console.log(_.filter([user2], {_id:user1._id}).length);

  //this logs `0`
  console.log(_.filter([user1], {_id:user1._id.toString()}).length);
});
});

Maybe I'm on a bad version - I'm on Mongoose 3.8.11.

[vkarpov15]

Hmm there was a bug with object id comparison in 3.8.9, see #2070. Should have been fixed in 3.8.10 though. I'll investigate this later.

[vkarpov15]

Ah I see what you're getting at here. You're actually trying to compare ObjectId with its string equivalent. Unfortunately that won't quite work, but this is precisely what virtuals are for:

userSchema.virtual('id').get(function() {
  return this._id.toString();
});

You should then be able to use comparisons and lodash filters to your heart's content since user1.id and user2.id will just be strings.

[rkumar1904]

@bendytree You are correct !

[joaoreynolds]

For those looking for a solution here, I found wrapping the find function actually cleans my code up even more anyway:

const _find = require('lodash/find')

module.exports = function findById(myArray, value, key = '_id'){
  return _find(myArray, myObj => String(myObj[key]) === String(value))
}

Use:

const foundSubDocument = findById(document.subdocuments, '591e5e7a7bdf6f14accf11da')

I also have a wrapper function for _.findIndex which looks almost the same.

[bendytree]

4 years later, I'll poke my head in again and say it'd still be nice if _id could be a string.

I'm sure ObjectId's benefit some folks, but it's constant pain for me. I can count on one hand the number of times I've needed an actual ObjectId and would prefer calling the constructor in those scenarios.

Over and over I have to call String(doc._id) for comparisons or patch libs like lodash which isn't just ugly, but also very slow in tight loops. A string would work perfectly everywhere.

Totally recognize this is a mongodb thing and so maybe it's just impossible. Either way, mongoose is great and I really appreciate the library as-is.

[vkarpov15]

@bendytree I feel your pain. I'm hesitant to always convert objectids to strings because from the maintainer point of view it feels like too much magic.

We are working on a new feature for 5.4 that will give you this behavior with a one liner though: https://github.com/Automattic/mongoose/issues/6912

mongoose.ObjectId.get(v => v == null ? v : v.toString())

What are your thoughts on this syntax?

[bendytree]

Thanks @vkarpov15. That would eliminate a lot of noise. Is that defined on each schema property? Would it work on lean objects?

Recasting to a string every time would have performance issues in a few tight spots.

If I'm honest, it feels like complexity breeding complexity whereas a plain string "just works".

But again, I totally get that this goes deep into mongo's core and definitely don't expect a fix here. Mainly I was hoping a few other heads would pop up in agreement, but it doesn't seem to be a common concern.

[vkarpov15]

@bendytree it would be on every schema property of type ObjectId, equivalent to setting a custom getter on every ObjectId path. It won't work with lean(), you'll still get an ObjectId.

If you want a plain string always without the headache, you can just store _id as a string. I agree this is a pain point, I have quite a bit of code in place for stripping out objectids so I can use chai's .to.deep.equal(). But the custom getter will eliminate that pain point, and I imagine the performance impact will be insignificant for most applications, especially if you're just using this for tests.