Mongoose: aggregate(...).allowDiskUse(...).cursor(...).exec(...).toArray is not a function

[alperdemirci]

I recently updated my mongoose to version ^5.0 and this is the issue I am having now.
TypeError: model.collection.aggregate(...).allowDiskUse(...).cursor(...).exec(...).toArray is not a function

This query used to work just fine but now it doesn't work at all: model.collection.aggregate(query).allowDiskUse(true).cursor({batchSize:1000}).exec().toArray()

I ran the process without the .allowDiskUse(true).cursor({batchSize:1000}) and it worked fine.

Any idea?

[lineus]

@alperdemirci

mongoose 5 returns a mongoose AggregationCursor now all of the time. Mentioned Here in the migration guide.

Here is a quick example based on the docs:

#!/usr/bin/env node
'use strict';

const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/test');
const conn = mongoose.connection;
const Schema = mongoose.Schema;

const schema = new Schema({
  name: String,
  age: Number
});

const Test = mongoose.model('test', schema);

const tests = [];

for (let i = 0; i < 100; i++) {
  tests.push(new Test({ name: `test${i}`, age: i }));
}

async function run() {
  await conn.dropDatabase();
  await Test.create(tests);

  const agg = [
    { $match: { age: { $gte: 50 } } },
    { $sort: { age: 1 } }
  ];

  const cursor = Test.aggregate(agg)
    .allowDiskUse(true)
    .cursor({batchSize: 10})
    .exec();
  await cursor.eachAsync(console.log);
  return conn.close();
}

run().catch(console.error);

output:

issues: ./6344.js
{ _id: 5ad1fe1acee0c04727670c1a, name: 'test50', age: 50, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c1b, name: 'test51', age: 51, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c1c, name: 'test52', age: 52, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c1d, name: 'test53', age: 53, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c1e, name: 'test54', age: 54, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c1f, name: 'test55', age: 55, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c20, name: 'test56', age: 56, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c21, name: 'test57', age: 57, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c22, name: 'test58', age: 58, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c23, name: 'test59', age: 59, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c24, name: 'test60', age: 60, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c25, name: 'test61', age: 61, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c26, name: 'test62', age: 62, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c27, name: 'test63', age: 63, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c28, name: 'test64', age: 64, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c29, name: 'test65', age: 65, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c2a, name: 'test66', age: 66, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c2b, name: 'test67', age: 67, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c2c, name: 'test68', age: 68, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c2d, name: 'test69', age: 69, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c2e, name: 'test70', age: 70, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c2f, name: 'test71', age: 71, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c30, name: 'test72', age: 72, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c31, name: 'test73', age: 73, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c32, name: 'test74', age: 74, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c33, name: 'test75', age: 75, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c34, name: 'test76', age: 76, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c35, name: 'test77', age: 77, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c36, name: 'test78', age: 78, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c37, name: 'test79', age: 79, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c38, name: 'test80', age: 80, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c39, name: 'test81', age: 81, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c3a, name: 'test82', age: 82, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c3b, name: 'test83', age: 83, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c3c, name: 'test84', age: 84, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c3d, name: 'test85', age: 85, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c3e, name: 'test86', age: 86, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c3f, name: 'test87', age: 87, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c40, name: 'test88', age: 88, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c41, name: 'test89', age: 89, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c42, name: 'test90', age: 90, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c43, name: 'test91', age: 91, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c44, name: 'test92', age: 92, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c45, name: 'test93', age: 93, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c46, name: 'test94', age: 94, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c47, name: 'test95', age: 95, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c48, name: 'test96', age: 96, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c49, name: 'test97', age: 97, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c4a, name: 'test98', age: 98, __v: 0 }
{ _id: 5ad1fe1acee0c04727670c4b, name: 'test99', age: 99, __v: 0 }
issues:

[alperdemirci]

@lineus
Right! but what is the reason I am getting such an error when I try to pass an option to cursor. Are you saying I dont need cursor() at all to determine the batchSize? or is it because of the promise I am waiting for?

following example from the document.

// When you call `.cursor()`, `.exec()` will now return a mongoose aggregation
// cursor.
const cursor = MyModel.aggregate([{ $match: { name: 'Val' } }]).cursor().exec();
// No need to `await` on the cursor or wait for a promise to resolve
cursor.eachAsync(doc => console.log(doc));

// Can also pass options to `cursor()`
const cursorWithOptions = MyModel.
  aggregate([{ $match: { name: 'Val' } }]).
  cursor({ batchSize: 10 }).
  exec();

[lineus]

It's not the options that you are passing to .cursor() that are the problem. It's calling .toArray() on the return value of

Model.aggregate(..).cursor(..).exec().

Starting in mongoose 5, this returns a special mongoose aggregation cursor, rather than a cursor from the native mongodb driver. The mongoose AggregationCursor doesn't have a .toArray() method like the native cursor does.

[lineus]

as a related side note, I noticed that the api docs don't include the aggregation cursor docs. I'll submit a PR to add those in.

[alperdemirci]

@lineus

I tried my query without the toArray() as follow

return model.collection.aggregate(qobj)
                .allowDiskUse(true)
                .cursor({batchSize: 1000})
                .exec().then((obj) => {
                return obj;
            });

returned TypeError: model.collection.aggregate(...).allowDiskUse(...).cursor(...).exec(...).then is not a function

This is also problematic unless I remove the .cursor({batchSize: 1000}).
Not sure what is going on.
FYI: migrated from mongoose version: 4.1.13

[lineus]

there are several methods on AggregationCursor, .eachAsync(fn, opts, callback), the one used in the example, will run a function on each doc ascynhronously as they are returned from the db.

here is the link to AggregationCursor until the API docs are updated.

The comments before each of the available methods on AggregationCursor contain information about how they work.

[lineus]

added PR #6353 to include Aggregation and Query Cursor docs in the API docs.

[lineus]

The api docs now include AggregationCursor here.

@alperdemirci were you able to get the aggregationCursor methods working?

[hellosandeep]

Thanks Alper

[Lwdthe1]

For anyone who needs to make a quick migration window and needs the toArray() method:

// Put this before you connect
const AggregationCursor = require('mongoose/lib/cursor/AggregationCursor')
if (!AggregationCursor.prototype.toArray) {
  AggregationCursor.prototype.toArray = function() {
    const arr = []
    return this.eachAsync(Array.prototype.push.bind(arr)).then(() => arr)
  }
}