Mongoose: Array Filters broken 5.4.4

Created on 13 Mar 2019  路  18Comments  路  Source: Automattic/mongoose

Mongoose: 5.4.4
NodeJS: LTS
MongoDB: 4

UpdateMany fails when filter is in nested array

const m = {
  arr: [
    {
      id: String,
      nestedArr: [
        {
          nestedId: String,
          code: Boolean
        }
      ]
    }
  ]
};

const o = {
  _id: '0'
  arr: [
    {
      id: '1',
      nestedArr: [
        {
          nestedId: '2',
          code: false
        }
      ]
    }
  ]
};

const q = {
  _id: '0'
}

const p = {
  'arr.$[arr].nestedArr.$[nArr].code': true
}

const opts = {
  arrayFilters: [
    {
      'arr.nestedArr.nestedId': '2'
    }, {
      'nArr.nestedId': '2'
    }
  ]
}

Model.updateMany(q, p, opts)

TypeError: Cannot read property 'castForQuery' of undefined
    at castArrayFilters (/Users/dbayo/data-model/node_modules/mongoose/lib/helpers/update/castArrayFilters.js:60:37)
    at _castArrayFilters (/Users/dbayo/data-model/node_modules/mongoose/lib/query.js:1736:5)
    at model.Query._updateThunk (/Users/dbayo/data-model/node_modules/mongoose/lib/query.js:3449:3)
    at model.Query.<anonymous> (/Users/dbayo/data-model/node_modules/mongoose/lib/query.js:3546:23)
    at model.Query._wrappedThunk [as _updateMany] (/Users/dbayo/data-model/node_modules/mongoose/lib/helpers/query/wrapThunk.js:16:8)
    at /Users/dbayo/data-model/node_modules/kareem/index.js:278:20
    at _next (/Users/dbayo/data-model/node_modules/kareem/index.js:102:16)
    at process.nextTick (/Users/dbayo/data-model/node_modules/kareem/index.js:499:38)

On the other hand, could you explain me what is the purpose of that file?
lib/helpers/update/castArrayFilters.js

I don't understand why filter path becomes to arr.0.nestedArr.code, because filter is only applying to first element.

Thank you all in advance,
Best regards

can't reproduce
Source
picture davidbayo10
👍1

All 18 comments

Yes, i have similar error when i use arrayFilters in 5.4.6.

dantehemerson picture dantehemerson on 15 Mar 2019
👀1

Here is another report of failures in version 5.4.9.

dantehemerson picture dantehemerson on 15 Mar 2019
👀1

Thanks for your patience. Fix will be in v5.4.21

vkarpov15 picture vkarpov15 on 27 Mar 2019
👍1

Thank you a lot @vkarpov15

davidbayo10 picture davidbayo10 on 27 Mar 2019

Hi @vkarpov15

About this error. The last release v5.4.21, fix the problem (TypeError: Cannot read property 'castForQuery' of undefined) but now I have the next error:

Error: Could not find path "routes.0.tracks.0.lugar_origen._id" in schema at castArrayFilters (D:\ProjectsWeb\ext\bus\back\back\node_modules\mongoose\lib\helpers\update\castArrayFilters.js:61:13) at _castArrayFilters (D:\ProjectsWeb\ext\bus\back\back\node_modules\mongoose\lib\query.js:1742:5) at model.Query._updateThunk (D:\ProjectsWeb\ext\bus\back\back\node_modules\mongoose\lib\query.js:3455:3) at model.Query.<anonymous> (D:\ProjectsWeb\ext\bus\back\back\node_modules\mongoose\lib\query.js:3552:23) at model.Query._wrappedThunk [as _updateMany] (D:\ProjectsWeb\ext\bus\back\back\node_modules\mongoose\lib\helpers\query\wrapThunk.js:16:8)

In befores versions the code was working well. Maybe have i to change the code for use array filters???

this is my code

CompanyModel.updateMany( { _id: companyId }, { $set: { "routes.$[].tracks.$[track].lugar_origen": oriPlace } }, { arrayFilters: [{ "track.lugar_origen._id": idPlaceToEdit }] }, function(err, company) { if (err) console.log(err); else console.log(company); } );

fastuller88 picture fastuller88 on 2 Apr 2019
👍1

same problem :(

enginkartal picture enginkartal on 2 Apr 2019

5.4.21 did not resolve the problem for me either

fracmak picture fracmak on 4 Apr 2019

@fastuller88 @enginkartal @fracmak can you please provide more detailed code samples, including your schema? Stack trace isn't sufficient for me to figure out the issue here.

vkarpov15 picture vkarpov15 on 8 Apr 2019

@vkarpov15

Basically is:

Schema:

CompanySchema = new Schema({
_id: Schema.Types.ObjectId,
routes: [RouteSchema],
});

RouteSchema = new Schema({
_id: {
type: Schema.Types.ObjectId
},
tracks: [ScheduleSchema]
});

PlaceSchema = new Schema({
_id: {
type: String
},
administrative: {
type: String
},
city: {
type: String
},
});

ScheduleSchema = new Schema({
_id: String,
lugar_origen: [PlaceSchema],
lugar_destino: [PlaceSchema],
});

And the code:

CompanySchema.updateMany( { _id: companyId }, { $set: { "routes.$[].tracks.$[track].lugar_origen": oriPlace } }, { arrayFilters: [{ "track.lugar_origen._id": idPlaceToEdit }] }, function(err, company) { if (err) console.log(err); else console.log(company); } );

It worked well in before versions to 5.4.20

I hope you can help us.

Maybe @enginkartal @fracmak do you have more examples???

Cheers

fastuller88 picture fastuller88 on 8 Apr 2019

This is my setup (schema truncated down to only the relevant fields)
Schema:

const ContactSchema = new mongoose.Schema({
  id: { type: String, required: true },
  institutionId: { type: String },
}, { _id: false });

const CollegeRSVPSchema = new mongoose.Schema({
  contact: { type: ContactSchema, required: true },
}, { _id: false });

const AppointmentsSchema = new mongoose.Schema({
  id: { type: String, required: true },
  attendees: [CollegeRSVPSchema],
}, { _id: false });

const Schema = new mongoose.Schema({
  id: { type: String, required: true },
  appointments: [AppointmentsSchema],
});

Update:

Model.bulkWrite([
  {
    updateMany: {
      filter: { 'appointments.attendees.contact.id': heUser.uuid },
      update: { $set: { 'appointments.$[].attendees.$[attendee].contact.institutionId': destination.id } },
      arrayFilters: [{ 'attendee.contact.id': heUser.uuid }],
    },
  },
]);
fracmak picture fracmak on 8 Apr 2019

@fracmak the below script executes without error for me in 5.4.4, 5.4.5, 5.4.20, 5.4.21, and 5.5.2. Please modify the below script to demonstrate your issue:

const assert = require('assert');
const mongoose = require('mongoose');
mongoose.set('debug', true);

const GITHUB_ISSUE = `gh7603`;
const connectionString = `mongodb://localhost:27017/${ GITHUB_ISSUE }`;
const { Schema } = mongoose;

run().then(() => console.log('done')).catch(error => console.error(error.stack));

async function run() {
  await mongoose.connect(connectionString);
  await mongoose.connection.dropDatabase();

  const ContactSchema = new mongoose.Schema({
    id: { type: String, required: true },
    institutionId: { type: String },
  }, { _id: false });

  const CollegeRSVPSchema = new mongoose.Schema({
    contact: { type: ContactSchema, required: true },
  }, { _id: false });

  const AppointmentsSchema = new mongoose.Schema({
    id: { type: String, required: true },
    attendees: [CollegeRSVPSchema],
  }, { _id: false });

  const schema = new mongoose.Schema({
    id: { type: String, required: true },
    appointments: [AppointmentsSchema],
  });

  const Model = mongoose.model('Test', schema);

  await Model.create({
    id: '1',
    appointments: [{
      id: '2',
      attendees: [
        { contact: { id: 'foo', institutionId: 'test' } },
        { contact: { id: 'baz', institutionId: 'test' } }
      ]
    }]
  });

  await Model.bulkWrite([{
    updateMany: {
      filter: { 'appointments.attendees.contact.id': 'foo' },
      update: { $set: { 'appointments.$[].attendees.$[attendee].contact.institutionId': 'bar' } },
      arrayFilters: [{ 'attendee.contact.id': 'baz' }],
    },
  }]);

  console.log(await Model.findOne().then(doc => doc.appointments[0].attendees));
}
vkarpov15 picture vkarpov15 on 17 Apr 2019

@vkarpov15 the update: { $set: { 'appointments.$[].attendees.$[attendee].contact.institutionId': 'bar' } }, element is not correct because you are updating all appointments which in attendees exists a contact id as 'foo'. This is wrong because is not the same case as I wrote at top

Try to add other element in appointments with different id and at least one attendee with contact id as 'foo'. You are going to update this element and we don't want to do that.

It is necessary to add other array filter with 'appintments.id': '2'

I haven't tested yet, but I think this will happen. You have to look up.

If some of my explanation is misunderstood I can try to explain with a full script

Thank you a lot all for helping and collaborating!

davidbayo10 picture davidbayo10 on 17 Apr 2019

@fastuller88 as far as I can tell, your issue is fixed in 5.4.21. The below script errors out with v5.4.20, and succeeds with v5.4.21:

const assert = require('assert');
const mongoose = require('mongoose');
mongoose.set('debug', true);

const GITHUB_ISSUE = `gh7603`;
const connectionString = `mongodb://localhost:27017/${ GITHUB_ISSUE }`;
const { Schema } = mongoose;

run().then(() => console.log('done')).catch(error => console.error(error.stack));

async function run() {
  await mongoose.connect(connectionString);
  await mongoose.connection.dropDatabase();

  const PlaceSchema = new Schema({
    _id: {
      type: String
    },
    administrative: {
      type: String
    },
    city: { 
      type: String
    },
  });

  const ScheduleSchema = new Schema({
    _id: String,
    lugar_origen: [PlaceSchema],
    lugar_destino: [PlaceSchema],
  });

    const RouteSchema = new Schema({
    _id: {
      type: Schema.Types.ObjectId
    },
    tracks: [ScheduleSchema]
  });

  const CompanySchema = new Schema({
    _id: Schema.Types.ObjectId,
    routes: [RouteSchema],
  });

  const Company = mongoose.model('Company', CompanySchema);
  await Company.updateMany(
    {},
    { $set: { "routes.$[].tracks.$[track].lugar_origen": { city: 'testcity' } } },
    { arrayFilters: [{ "track.lugar_origen._id": 'testid' }] }
  );
}
vkarpov15 picture vkarpov15 on 17 Apr 2019

@fracmak thanks for your feedback. I'm not sure how to interpret your comment, there's a lot of prose for me to try to convert to code. I copied the bulkWrite() from this comment, if that isn't sufficient to repro your issue then please modify the script from my previous comment to demonstrate the issue. Thanks!

vkarpov15 picture vkarpov15 on 17 Apr 2019

Sorry for the trouble, I had 4 different arrayFilter queries all in a row and I mistakenly grabbed the wrong one. I've taken your test script and updated it with the correct schema and query. This script does error out on me with the latest [email protected] with the error Error: Could not find path "appointments.0.contact.institutionId" in schema

const assert = require('assert');
const mongoose = require('mongoose');
mongoose.set('debug', true);

const GITHUB_ISSUE = `gh7603`;
const connectionString = `mongodb://localhost:27017/${ GITHUB_ISSUE }`;
const { Schema } = mongoose;

run().then(() => console.log('done')).catch(error => console.error(error.stack));

async function run() {
  await mongoose.connect(connectionString);
  await mongoose.connection.dropDatabase();

  const ContactSchema = new mongoose.Schema({
    id: { type: String, required: true },
    institutionId: { type: String },
  }, { _id: false });

  const CollegeRSVPSchema = new mongoose.Schema({
    contact: { type: ContactSchema, required: true },
  }, { _id: false });

  const AppointmentsSchema = new mongoose.Schema({
    id: { type: String, required: true },
    attendees: [CollegeRSVPSchema],
    institutionIds: [String],
  }, { _id: false });

  const schema = new mongoose.Schema({
    id: { type: String, required: true },
    appointments: [AppointmentsSchema],
  });

  const Model = mongoose.model('Test', schema);

  await Model.create({
    id: '1',
    appointments: [{
      id: '2',
      institutionIds: ['test']
      attendees: [
        { contact: { id: 'foo', institutionId: 'test' } },
        { contact: { id: 'baz', institutionId: 'test' } }
      ]
    }]
  });

  await Model.updateMany(
    {'appointments.attendees.contact.id': 'foo'},
    { $addToSet: { 'appointments.$[appointment].institutionIds': 'bar' } },
    { arrayFilters: [{ 'appointment.contact.institutionId': 'test' }] },
  );

  console.log(await Model.findOne().then(doc => doc.appointments[0].attendees));
}
fracmak picture fracmak on 17 Apr 2019

actually, I'm realizing that there's actually a bug in that query. So the error it's generating is legitimate.

fracmak picture fracmak on 17 Apr 2019

This script works for me in last mongoose version but fails in 5.4.4:

const assert = require('assert');
const mongoose = require('mongoose');
mongoose.set('debug', true);

const GITHUB_ISSUE = `gh7603`;
const connectionString = `mongodb://localhost:27017/${GITHUB_ISSUE}`;
const { Schema } = mongoose;

run().then(() => console.log('done')).catch(error => console.error(error.stack));

async function run() {
  await mongoose.connect(connectionString);
  await mongoose.connection.dropDatabase();

  const ContactSchema = new mongoose.Schema({
    id: { type: String, required: true },
    institutionId: { type: String },
  }, { _id: false });

  const CollegeRSVPSchema = new mongoose.Schema({
    contact: { type: ContactSchema, required: true },
  }, { _id: false });

  const AppointmentsSchema = new mongoose.Schema({
    id: { type: String, required: true },
    attendees: [CollegeRSVPSchema],
    institutionIds: [String],
  }, { _id: false });

  const schema = new mongoose.Schema({
    id: { type: String, required: true },
    appointments: [AppointmentsSchema],
  });

  const Model = mongoose.model('Test', schema);

  await Model.create({
    id: '1',
    appointments: [
      {
        id: '2',
        institutionIds: ['test'],
        attendees: [
          { contact: { id: 'foo', institutionId: 'test' } },
          { contact: { id: 'baz', institutionId: 'test' } }
        ]
      }, {
        id: '3',
        institutionIds: ['test'],
        attendees: [
          { contact: { id: 'foo', institutionId: 'test2' } },
          { contact: { id: 'baz', institutionId: 'test2' } }
        ]
      }
    ]
  });

  await Model.updateOne(
    { 'id': '1' },
    { $set: { 'appointments.$[appointment].attendees.$[attendee].contact.institutionId': 'updated' } },
    { arrayFilters: [
      {
        'appointment.attendees.contact.id': 'foo'
      }, {
        'attendee.contact.id': 'foo'
      }
    ] },
  );

  console.dir(await Model.findOne().lean().exec().then(doc => doc), { depth: 10 });
}

Mongoose: v5.5.2
MongoDB: v4.0.3
NodeJS: v10.15..0

For Mongoose v5.4.4 the output is like:

TypeError: Cannot read property 'castForQuery' of undefined
    at castArrayFilters (/Users/dbayo/Projects/mongoose-test/node_modules/mongoose/lib/helpers/update/castArrayFilters.js:59:37)
    at _castArrayFilters (/Users/dbayo/Projects/mongoose-test/node_modules/mongoose/lib/query.js:1738:5)
    at model.Query._updateThunk (/Users/dbayo/Projects/mongoose-test/node_modules/mongoose/lib/query.js:3403:3)
    at model.Query.<anonymous> (/Users/dbayo/Projects/mongoose-test/node_modules/mongoose/lib/query.js:3506:23)
    at model.Query._wrappedThunk [as _updateOne] (/Users/dbayo/Projects/mongoose-test/node_modules/mongoose/lib/helpers/query/wrapThunk.js:16:8)
    at process.nextTick (/Users/dbayo/Projects/mongoose-test/node_modules/kareem/index.js:369:33)
    at process._tickCallback (internal/process/next_tick.js:61:11)
davidbayo10 picture davidbayo10 on 18 Apr 2019

@davidbayo10 please upgrade to Mongoose 5.4.21.

vkarpov15 picture vkarpov15 on 22 Apr 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

efkan picture efkan  路  3Comments
simonxca picture simonxca  路  3Comments
wlingke picture wlingke  路  3Comments
jeneser picture jeneser  路  3Comments
p3x-robot picture p3x-robot  路  3Comments