Mongoose: finding document with array property

[hwkd]

Do you want to request a feature or report a bug?
Bug

What is the current behavior and if the current behavior is a bug, please provide the steps to reproduce.**
Given that we have a document in mongodb as shown below,

{
  _id: ObjectID("5d8d3fdb563f2001b5559242"),
  prop: ["va1", "val2", "val3"]
}

Executing the query mymodel.find({ prop: ["val1", "val3"] returns the above document (but it shouldn't).

What is the expected behavior?
According to the mongodb official page, this should not return the above document. Unless the array of the query exactly matches the array of the document, it should not return as a matched document.

I've run the exact same query on the mongodb shell and it worked as expected, so I can only conclude that mongoose is the culprit.

What are the versions of Node.js, Mongoose and MongoDB you are using? Note that "latest" is not a version.
Mongoose: 5.6.11 (latest)
Node.js: 12.10.0
MongoDB: 3.4.20

[BenSower]

Hey hwkd, I believe I stumbled upon the same problem. What does your Schema look like? Is prop defined as Schema.Types.Mixed by any chance?

Because in this case, mongoose does not directly query for the array, but puts the values in an $in-clause. So your query actually becomes this:
mymodel.find({ prop: {$in: ["val1", "val3"]}}), which returns true, because one of the values in the array is in the prop array.

Here's a working example for the problem (mongo-server 4.0.12, [email protected], node v10.12.0)

const mongoose = require('mongoose');
mongoose.set('debug', true);

const GITHUB_ISSUE = `8202`;
const connectionString = `mongodb://localhost:27017/${GITHUB_ISSUE}`;
const { Schema, SchemaTypes } = mongoose;

run()
  .then(() => console.log('done'))
  .catch(error => console.error(error.stack));

async function run() {
  await mongoose.connect(connectionString);
  await mongoose.connection.dropDatabase();
  // if we manually define the schema like this it'll work as expected:
  // validations: [{a:String}]
  const TestSchema = new Schema({ validations: Schema.Types.Mixed });
  const TestModel = mongoose.model(GITHUB_ISSUE, TestSchema);

  // creating testObject
  let test1 = new TestModel({ validations: [{ a: 'b' }, { a: 'a' }] });
  await test1.save();

  /*
    The output of mongoose.debug will look like this:
    Mongoose: indextests.count({
    validations:
      {
        '$in': [
          { a: 'b' },
          { a: 'THIS SHOULD NOT BE FOUND' }
        ]
      }
    }, {})
  */
  const count = await TestModel.count({ validations: [{ a: 'b' }, { a: 'THIS SHOULD NOT BE FOUND' }] })
  if (count === 1) {
    // This will be the output
    console.log('----> Error, count should be 0, not 1!');
  } else {
    console.log('All is fine!')
  }

  process.exit();
}

[BenSower]

Ok, by digging a little deeper I found this part of the code that seems to be the culprit in my case:
https://github.com/Automattic/mongoose/blob/9a1d494141a482d2923ac7f2224b80886bc2b6e8/lib/cast.js#L296
[...]
https://github.com/Automattic/mongoose/blob/9a1d494141a482d2923ac7f2224b80886bc2b6e8/lib/cast.js#L305
In this, the default behaviour of mongoose is to query with $in if the query contains an array and the target field is not of type array or buffer.

@vkarpov15, @QuotableWater7 : Why was this implementation chosen and is there a way around this, without having to change my schema?

[hwkd]

Hi @BenSower, to answer your earlier question, the schema type for the property I declared is an array of string.

In my case, although the target field is an array of string and the query was an array, it still didn't work as expected.

There's a minute detail here though: I am using the Typegoose library, and although I highly doubt that it's an issue with Typegoose (since it's just a wrapper around Mongoose to work smoothly with TypeScript without changing anything), I'll also look into that.

[hwkd]

My apologies. I just looked into Typegoose docs and I found out that I was misusing the prop decorator when, in fact, I was supposed to use arrayProp.

I will leave the issue opened in case @BenSower needs to clarify a few things with the collaborators before it is closed.

@BenSower I appreciate the response since I could drill down what was causing this and also learned about the casting behaviour in Mongoose with arrays.

[hwkd]

I'll close the issue since the main issue has been resolved and there hasn't been any activity from the collaborators.

[BenSower]

@vkarpov15 I saw that you reopened the ticket, is there anything I can do to support this ticket?

[vkarpov15]

@BenSower I just wanted to make a note to read through this issue and suggest workarounds. The original justification for this feature is in #4912 and #4873: generally, if you're querying on a non-array with an array, you want to use $in, so we decided to add this as syntactic sugar.

As a workaround, you can explicitly use $eq:

'use strict';

const mongoose = require('mongoose');
const Schema = mongoose.Schema;

mongoose.set('debug', true);

run().catch(error => console.log(error));

async function run() {
  await mongoose.connect('mongodb://localhost:27017/test', { useNewUrlParser: true, useUnifiedTopology: true });

  const Model = mongoose.model('Test', Schema({
    prop: {}
  }));

  await Model.deleteMany({});
  await Model.create({ prop: ['val1', 'val2', 'val3'] });

  // Using `$eq` tells Mongoose to not use `$in` syntactic sugar
  const doc = await Model.find({ prop: { $eq: ['val1', 'val3'] } });
  console.log(doc);
}

[BenSower]

Thank you @vkarpov15 for the detailed answer! :-)