Mongoose: beautiful mongodb native errors (unique - code 11000)

[xpepermint]

I really miss a mechanism that would create validator-like unique index error message. Setting the index of an attribute to unique: true will tell mongodb to return ugly error message like this:

{ [MongoError: insertDocument :: caused by :: 11000 E11000 duplicate key error index: mydb-api.users.$email_1  dup key: { : "[email protected]" }]
  name: 'MongoError',
  code: 11000,
  err: 'insertDocument :: caused by :: 11000 E11000 duplicate key error index: mydb-api.users.$email_1  dup key: { : "[email protected]" }' }

This message should be treated as validator error message thus we could easily display error messages on a REST service.

// koa example
index: function*() {
    try {
      var email = new User(body);
      this.body = yield email.save();
    } catch(e) {
      this.body = e; // nice json errors to display
    }
}

[mocheng]

+1

By the way, for any unique indexed fields, I have to check whether existent fields exists before creating new doc. This is duplicate work. It would be great that it is auto handled.

[xpepermint]

Something like

    if (11000 === err.code || 11001 === err.code) {
      var MongooseError = require('mongoose/lib/error')
      var valError = new MongooseError.ValidationError(err)
      valError.errors["xxx"] = new MongooseError.ValidatorError('xxx', 'Duplicate found', err.err)
      err = valError
    }

could be triggered after anything is received from a db. So every function like create, update and save should trigger this automatically before return. Let's wait to see what owners/others have to say about this.

[vkarpov15]

Good suggestion, thanks.

[andriesfilmer]

mongoose-unique-validator is a plugin which adds pre-save validation for unique fields within a Mongoose schema. https://github.com/blakehaswell/mongoose-unique-validator

[iliakan]

@vkarpov15 as of now I have a wrapper over mongoose, adding this.

Mongoose-unique-validator is doing it wrong. Pre-save validation is not a real solution here, because only unique index guarantees uniqueness.

user A: run pre-save, is the value unique? yes it is.
user B: run pre-save, is the value unique? yes it is.
user B: insert value
user A: insert value (BUG!)

[vkarpov15]

Yep mongoose-unique-validator is not at all guaranteed to be unique, at least with multiple processes. The "plugin" tag is a reference to OP's issue about beautifying MongoErrors, which would be a valid plugin use case.

[xpepermint]

any progress?

[vkarpov15]

Nope, but I think you might be able to do this as a plugin...

[iliakan]

@xpepermint I'm doing that by a wrapper around save. No idea what @vkarpov15 means about using a plugin here (?)

[vkarpov15]

I think the way to implement this would be a small separate module that does something like this:

module.exports = function(schema) {
  schema.methods.trySave = function(callback) {
    this.save(function(error, doc) {
      // Handle error logic here and callback();
    });
  };
};

And then any schema that wants to expose this trySave function would use schema.plugin(require('mongoose-beautiful-errors')); or whatever

[iliakan]

Yes, that's kind of what I'm doing.

Persist method, like this:
https://github.com/iliakan/javascript-nodejs/blob/master/modules/lib/mongoose.js#L39

[vkarpov15]

Yeah something like that as a standalone plugin would probably be a great idea. I like the general idea but I'm hesitant to put this into the mongoose core because it's a big backwards-breaking change and IMO doesn't really move the needle in terms of the experience of using mongoose.

[matteodelabre]

I made a plugin based on your recommandations.
https://www.npmjs.com/package/mongoose-beautiful-unique-validation

[iliakan]

My code is a little bit more versatile, so I stick to it for now, but thanks anyway ;)

Or does it support custom messages?

[matteodelabre]

It does indeed, you just need to set the message as the value of unique.

var userSchema = mongoose.Schema({
    name: {
        type: String,
        unique: 'Custom error message'
    }
});

userSchema.plugin(require('mongoose-beautiful-unique-validation'));

[vkarpov15]

Thanks for the plugin :) I'll mark this issue as closed.

[Jokero]

I think it's should be out of the box feature. mongoose-beautiful-unique-validation is very hacky solution. I should use trySave instead of save. So I can't use other mongoose methods like create, update, findOneAndUpdate to get same validation error.

Or maybe we can use mongoose hooks to handle duplicate error? Like this:

schema.post('error', function(err, next) { // it can be moved to plugin
    if (err.code === 11000) {
        return next(new ValidationError(...));
    }

    next(err);
});

Model.create({}).catch(function(err) {
    if (err instanceof ValidationError) { 
        ...
    }
});

[vkarpov15]

Yeah trySave() would be the best way right now. Hooks won't work because a core assumption of how hooks work is "if .save() fails, don't fire post save() hooks".

[Jokero]

trySave() is very ugly way. So I propose to add error hook where I can replace original error object

[vkarpov15]

That would be tricky, because post('error'); would imply handling more errors than just save errors. I would imagine having a catch-all for document-level errors though (validation, etc.) would be pretty useful though - would that still work?

[Jokero]

We also should be able to handle model save errors (after Model.create(), Model.update(), etc.). It should be uniformly

[ghost]

Where can I find a list of Mongodb error codes?
11000is a duplicate key error code, but where can I find a list of the others?

[simison]

@ChrisDillinger

https://github.com/mongodb/mongo/blob/2cf31060a74ac438b58c5150ba30da4144112744/src/mongo/base/error_codes.err

[iliakan]

@vkarpov15 anything changed here?

[VtoCorleone]

I upgraded to Mongoose 4.5.0 and mongo 3.2.7. I tried implementing the following post save hook but it is not working. The err object is not an error object but still the doc object.

Person.post('save', function (err, doc, next) {
  if (err) {
    next(err);
  } 
  else {
    next();
  }
});

It looks like the post save is getting unit tested against here.

I am triggering an 11000 mongo error but violating a unique index.

[vkarpov15]

@VtoCorleone you need to define the post hook on a schema, not a model. Also, can you show your schema please?

[iliakan]

@vkarpov15 as I recall this issue was about better handling of unique errors, to put them in line with regular validation errors.

May I ask, was anything changed here?

[Jokero]

After I learned about these changes I tried to figure out how to use this feature to handle 11000 errors. If I understood correctly I should implement post hooks for all write methods: save, update, create, findOneAndUpdate (if I use all of them of course). Is it really good solution?

Additionally I had a question about Model.create(). What should I do to handle 11000 error for this method? Will some hook be called? I figured out that save hook works for Model.create() too. That's why I also created https://github.com/Automattic/mongoose/issues/4233

[vkarpov15]

@Jokero yeah that's the general idea, youd also need one for insertMany. IMO it's the solution that adds the least surface area to the api - if you use mongoose, you know how to use hooks, so here's a slight modification to hooks to let you process errors

[markwain]

@vkarpov15 Does mongodb native errors have beautiful handling now?

[vkarpov15]

Nope but you can write a plugin to do so pretty easily: http://thecodebarbarian.com/mongoose-error-handling.html

[jhenriquez]

I created one that I expect to grow to handle most common MongoErrors, this one is using the 4.5 middlewares.

https://github.com/jhenriquez/mongoose-mongodb-errors

[niftylettuce]

I strongly recommend everyone to use the well-documented package by @matteodelabre at https://github.com/matteodelabre/mongoose-beautiful-unique-validation per my PR here https://github.com/matteodelabre/mongoose-beautiful-unique-validation/pull/40. It is well-tested too and also supports a custom message formatted which builds on top of Mongoose standard error messages, e.g. mongoose.Error.messages.general.unique has a set default value of "Path `{PATH}` ({VALUE}) is not unique." if not already defined.

Furthermore, you can also use my new package mongoose-validation-error-transform at https://github.com/niftylettuce/mongoose-validation-error-transform, which will automatically transform the validation error message to a humanized and readable format.

Here's how to get started with both of these awesome packages:

npm install --save mongoose-beautiful-unique-validation mongoose-validation-error-transform

const mongooseBeautifulUniqueValidation = require('mongoose-beautiful-unique-validation');
const mongooseValidationErrorTransform = require('mongoose-validation-error-transform');

mongoose.plugin(mongooseBeautifulUniqueValidation);
mongoose.plugin(mongooseValidationErrorTransform);

For example, it will take a message of "full_name" is required and rewrite it to "Full name is required" automatically. It is also excellent for error handling, as it gives you a joined validation error message by comma (e.g. multiple props have validation errors in your schema).

If you wish to customize how the messages are joined, you can provide your own transformation (e.g. you may want to return an err.message that is a <ul> HTML tag of all the errors instead of a plain String joined by a comma. To do so simply pass an options object and override the default transform function.

mongoose.plugin(mongooseValidationErrorTransform, {
  // change this however you like, this is the default:
  transform: function(messages) {
    return messages.join(', '); 
  }
});

To output a <ul> list of error messages, of course you'd only want a <ul> if there's more than one:

mongoose.plugin(mongooseValidationErrorTransform, {
  // output a <ul> with bootstrap alpha 4 css classes
  transform: function(messages) {
    if (messages.length === 1) return messages[0];
    return `<ul class="text-xs-left mb-0"><li>${messages.join('</li><li>')}</li></ul>`;
  }
});

Complete default options for mongooseValidationErrorTransform are:

{
  capitalize: true,
  humanize: true,
  transform: function(messages) {
    return messages.join(', ');
  }
}

[iliakan]

I also use https://github.com/matteodelabre/mongoose-beautiful-unique-validation atm.