Mongoose: insertMany multiple validation failures

[Mattchewone]

Issue/Comment
Not sure if this is a bug or expected behaviour.

I am using insertMany and when I have a validation error(s) only the first error is returned and this throws an error.

For example:

User.insertMany(users)
.then(users => {
    console.log(`Results: ${users.length}`);
})
.catch(err => {
    console.log(`Error: ${err}`);
});

If I have 5 users, and 2 of them have failed due to validations, then the catch is hit, but it will only show the first validation error, rather than a list of validations.

Expected behavior?
A list of validations to be returned if there are more than one error.

I would also think that it would be better to return the results and the errors much like Mongodb does natively, where it lists the inserted ids and any failures. Not sure if that would be a breaking change to the current API, but it seems more of how the mongodb driver works natively.

Versions
Node = v8.0.0
Mongoose = 4.10.5

[varunjayaraman]

ill label this an enhancement for now.

const mongoose = require('mongoose');
const co = require('co');
mongoose.Promise = global.Promise;
const GITHUB_ISSUE = `gh-5337`;


exec()
  .then(() => {
    console.log('successfully ran program');
    process.exit(0);
  })
  .catch(error => {
    console.error(`Error: ${error}\n${error.stack}`);
  });


function exec() {
  return co(function* () {
    const db = mongoose.createConnection(`mongodb://localhost:27017/${GITHUB_ISSUE}`)
    const schema = new mongoose.Schema({
      name: { type: String, required: true },
      age: Number
    });


    const Model = db.model('Model', schema);

    yield Model.remove({});

    try {
      const op = yield Model.insertMany([
        { name: 'test1', age: 1 },
        { age: 2 },
        { name: 'test3', age: 3 },
        { age: 4 }
      ]);

    } catch (error) {
      console.log(error);
    }

  });
}

[NineToeNerd]

I am having a related issue. I expect to get the inserted objects back in the result even when there are validation errors, but I only get the first validation error - _no result array_. In my case, my app may send duplicate objects to the server and there will be a validation error with "code":11000. When this happens I'm not getting an array with the objects that were successfully saved (I get undefined).

It appears commit 89e1271e addresses the issue but I wanted to make sure it was known that was part of the issue and follow the thread.

[vkarpov15]

Added an issue to return all docs that we inserted in #5783 . We added a basic fix in https://github.com/Automattic/mongoose/pull/5748 but we'll need a little extra work to support your case.

[PaulBrimley]

I have searched everywhere and can't find how to get the documents that were NOT inserted. Is this possible as well? If it is possible I can then handle the errors for each document as needed. Thank you!

[lineus]

@PaulBrimley with Model.insertMany options set to { ordered: false, rawResult: true } the result returned from mongoose contains a property called mongoose that contains an array of the validation errors. Here is an example:

5337.js

#!/usr/bin/env node
'use strict';

const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost:27017/test', { useNewUrlParser: true });
const conn = mongoose.connection;
const Schema = mongoose.Schema;

const schema = new Schema({
  name: {
    type: String,
    minlength: 6
  }
});

const Test = mongoose.model('test', schema);

const tests = [];

for (let i = 0; i < 10; i++) {
  tests.push(new Test({ name: 'x'.repeat(i) }));
}

async function run() {
  await conn.dropDatabase();
  let res = await Test.insertMany(tests, { ordered: false, rawResult: true });
  let inserted = res.ops;
  let notInserted = res.mongoose.validationErrors.map((e) => {
    return tests.find((t) => {
      return t.name === e.errors.name.properties.value;
    });
  });

  console.log('Inserted:\n', inserted);
  console.log('Not Inserted:\n', notInserted);
  return conn.close();
}

run();

Output:

issues: ./5337.js
Inserted:
 [ { _id: 5b49eebb7029ac17d5ae8e05, name: 'xxxxxx', __v: 0 },
  { _id: 5b49eebb7029ac17d5ae8e06, name: 'xxxxxxx', __v: 0 },
  { _id: 5b49eebb7029ac17d5ae8e07, name: 'xxxxxxxx', __v: 0 },
  { _id: 5b49eebb7029ac17d5ae8e08, name: 'xxxxxxxxx', __v: 0 } ]
Not Inserted:
 [ { _id: 5b49eebb7029ac17d5ae8dff, name: '' },
  { _id: 5b49eebb7029ac17d5ae8e00, name: 'x' },
  { _id: 5b49eebb7029ac17d5ae8e01, name: 'xx' },
  { _id: 5b49eebb7029ac17d5ae8e02, name: 'xxx' },
  { _id: 5b49eebb7029ac17d5ae8e03, name: 'xxxx' },
  { _id: 5b49eebb7029ac17d5ae8e04, name: 'xxxxx' } ]
issues:

[PaulBrimley]

Thank you for your quick response and the example. I assume, then, that it is not currently possible for the validation error to return the document that failed rather than just matching the error to the documents attempting to be inserted? ie. What if (in your example) two documents have the same 'name'. The find will just return the first one that matches rather than the actual document that failed. Another problem I have is that in our app the documents are inserted after being created dynamically after an external api pull and won't know what specific validation error(s) we might encounter. We are also bringing in tens of thousands of documents and to have to map over each error with a find will be quite expensive on the app. If the document that failed validation is returned with the error we can then have access to it (by placing it in a different collection) to see why it is failing validation. Anyway, just a thought and thank you again for your help!

[vkarpov15]

@PaulBrimley you're better off matching which documents failed by their _id, if you have duplicate _ids in insertMany then figuring out which document actually got saved is very tricky

[notflip]

I don't seem to have access to the res.ops using that same code. It's returning an empty array for me ..

[lineus]

@notflip what version of mongoose/node/mongodb are you using?

[notflip]

Latest, I suppose. Installed last week

On Sat, 27 Oct 2018, 22:43 Kev, [e-mail] wrote:

@notflip https://github.com/notflip what version of
mongoose/node/mongodb are you using?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Automattic/mongoose/issues/5337#issuecomment-433606354,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD9E0PywAuOZ3sl-7C0DItCruc_XHZfCks5upCq2gaJpZM4N0Ruk
.

[vkarpov15]

@notflip that isn't very helpful unfortunately. Can you just print out require('mongoose').version from your node shell? See docs

[carlosvargas]

@notflip, getting the same result as you, however I tracked it down due to the fact that all of my objects failed validation (I'm using {ordered: false}).

Looking at https://github.com/Automattic/mongoose/blob/5.4/lib/model.js#L2832, it returns an empty array when there's no valid objects. I would have preferred if it returned the validation errors though since I'm re-running the validations when I get an empty array as a result.