Mongoose: Converting all ObjectIds into string by using Document.toObject() method?

[kitlee]

As I found that the _id in the document is still an ObjectId after toObject()

Is there a way to achieve this?

[Mickael-van-der-Beek]

I've got the same bug. Both toJSON() and toObject() don't stringify ObjectId instances.

[vkarpov15]

This is by design right now, because ObjectIds technically aren't strings. Off the top of my head the only way I can think of would be to JSON.parse(JSON.stringify(doc)); What's your use case for stringifying all object ids?

[tiant167]

so, function toJSON() isn't convert the object to JSON string ? The _id and date isn't a string but an object ?

[tiant167]

by the way, what is the difference between toObject() and toJSON()?
thx!

[vkarpov15]

Yeah date should still be a date, not a string, after toJSON() + toObject().

toJSON() and toObject() are essentially identical, the only difference is that when you do JSON.stringify(doc) JavaScript looks for a toJSON() function on doc and uses it to convert the object. So the reason for toJSON() is syntactic sugar for JSON.stringify().

[linusbrolin]

@vkarpov15 Sorry for commenting on a closed issue, but it seemed the most relevant place to post, since it's directly related to this issue.

I'm trying to deep-compare two mongoose docs to get a diff object of the changes between them.
ObjectID's however, cannot be compared directly it seems, because they will always end up with a diff, even if they contain the exact same id's.

The only way I've been able to do it so far, is to do this:

var objectdiff = require('object-deep-diff');

var diffresult = objectdiff(
  JSON.parse(JSON.stringify(doc1.toObject())),
  JSON.parse(JSON.stringify(doc2.toObject()))
);

That seems like an expensive way of getting a diff between objects.
Do you know of a better way to reliably compare mongoose docs?

[vkarpov15]

Not sure how deep diff works but it might be running into some issues with mongodb ObjectIds. Try converting _id and other objectid fields to strings. Or just use lodash's deep equality check, I think that one works with ObjectIds

[linusbrolin]

@vkarpov15 deep diff uses lodash for the equality check, so unfortunately that won't work:
https://github.com/rbs392/object-deep-diff/blob/master/index.js

Converting all ObjectId's to strings would work, but since the objects can contain arrays and/or other nested objects, that becomes quite an expensive task to do aswell.. :/

[vkarpov15]

So here's another question, are you using deep-diff on 2 mongoose documents or 2 documents that are the result of toObject()? The latter should work, the former not so much.

[linusbrolin]

@vkarpov15 it's usually 1 mongoose doc (converted to a javscript object with .toObject) and one regular javascript object.
The reason is that I need to compare data coming from the client, with the data already in the database.
So, the data coming from the client already have the ObjectID's converted, but the data coming from the database haven't.
But since I'm doing this deep-diff check in a mongoose plugin, I need to make absolutely sure that both docs are constructed identically, so that's why I have to run JSON.parse(JSON.stringify(doc.toObject())) on both docs.

[vkarpov15]

Yeah in that case you're going to need to do JSON.parse(JSON.stringify()), that's the easiest way to convert all the object ids to strings.

[ZacharyRSmith]

JSON.parse(JSON.stringify()) was too computationally intensive for me, so I had to create this perf-thinking lib.

var transformProps = require('transform-props');

function castToString(arg) {
    return String(arg);
}

var doc = new MongooseModel({ subDoc: { foo: 'bar' }});
var docObj = doc.toObject();

transformProps(docObj, castToString, '_id');

console.log(typeof docObj._id); // 'string'
console.log(typeof docObj.subDoc._id); // 'string'

[wi-ski]

This feature of Mongo is really regrettable

[flight9]

@ZacharyRSmith Then what's about other props, eg. author_id(ObjectId refers to User Model)/author_ids(array of ObjectIds)?

[ZacharyRSmith]

@flight9 you can tell transformProps() to look for multiple keys. see this test here: https://github.com/ZacharyRSmith/transform-props/blob/master/index.test.js#L67

i believe you can get what you want with this:

function myToString(val) {
  if (Array.isArray(val)) {
    return val.map(item => String(item));
  }
  return String(val);
}

transformProps(obj, myToString, ['author_id', 'author_ids']);

[flight9]

@ZacharyRSmith Thanks for your solution, It's great!

I think i missed one important thing: I don't konw which fields are ObjectId beforehand, because the tool I'm writing should be applied to all models in our project, which may have several ObjectId fields or have none, and its field names are indeterminate.
So after several hours' trying, I have my solution:

const ObjectID = require('mongodb').ObjectID;

/**
 * Convert ObjectId field values inside an object to String type values
 * (includes array of ObjectIds and nested ObjectId fields)
 * @param obj Object - The Object to be converted.
 * @return none
 */
function objectIdtoString(obj) {
  for(let k in obj) {
    let v = obj[k];
    if(typeof v == 'object') {
      if(k.endsWith('_id') && v instanceof ObjectID) {
        obj[k] = v.toString();
      }
      else if(
        k.endsWith('_ids') && Array.isArray(v) &&
        v.length > 0 && v[0] instanceof ObjectID
      ) {
        let vs = [];
        for(let iv of v) {
          vs.push(iv.toString());
        }
        obj[k] = vs;
      }
      else {
        objectIdtoString(v);
      }
    }
  }
}

// Call example
objectIdtoString(obj):

[Billy-]

@vkarpov15

Yeah in that case you're going to need to do JSON.parse(JSON.stringify()), that's the easiest way to convert all the object ids to strings.

https://medium.com/ft-product-technology/this-one-line-of-javascript-made-ft-com-10-times-slower-5afb02bfd93f

tldr; JSON.parse(JSON.stringify()) is a synchronous, blocking task, and can cause serious bottlenecks. This should not be the accepted/suggested solution for this issue...

[vkarpov15]

You should also be able to do mongoose.Types.ObjectId.prototype.toJSON = function() { return this.toString(); . But even that article says, JSON.parse(JSON.stringify) is one of the faster ways of doing deep clone, their problem was that they were unnecessarily deep cloning repeatedly

[gitowiec]

So, here is 2019.
Do we yet have nice and clean and obvious way to get hex string from every ObjectId instance on object?

[vkarpov15]

@gitowiec

mongoose.ObjectId.get(v => v.toString());

See: http://thecodebarbarian.com/whats-new-in-mongoose-54-global-schematype-configuration.html#schematype-getters

[nhitchins]

@vkarpov15 looks like this is not a solution when using 'lean'.
So if the desired result is to return a plain json object with string objectIds it seems the most efficient way to achieve this is to still use JSON.parse(JSON.stringify()) with the lean option.

Or possibly a lean plugin: https://www.npmjs.com/package/mongoose-lean-objectid-string

[vkarpov15]

@nhitchins correct, you'd have to use a plugin like mongoose-lean-getters.

[pwrnrd]

@vkarpov15 is this also supposed to work with a discriminator schema? I have a base schema from which I create variants. I tried setting getters on the base schema. I use lean() and used mongoose-lean-getters but I couldn't get this to work.

[vkarpov15]

@pwrnrd can you please open a new issue and follow the issue template? It should work fine with a discriminator schema.