Mongoose: Docs: add note about `execPopulate()` to populate docs

[soroushm]

mongoose Not able to return populated
see blow

Model

import { Schema, model } from 'mongoose'
import { transformVirtuals } from 'src/utils/mongoose'

const GameSchema = new Schema(
  {
    _t: {
      type: String,
      alias: 'title',
      required: true,
    },
    _r: {
      type: Number,
      alias: 'round',
      min: 1,
      max: 7,
      default: 7,
    },
    _pr: {
      type: Boolean,
      default: false,
      alias: 'private',
    },
    _o: {
      alias: 'owner',
      required: true,
      type: Schema.Types.ObjectId,
      ref: 'User',
    },
    _pl: {
      type: [{ type: Schema.Types.ObjectId, ref: 'User' }],
      alias: 'players',
      validate: [
        {
          validator: playersLimit,
          message: 'board is full',
        },
        {
          validator: playersUnique,
          message: 'player already exist',
        },
      ],
    },
    _p: {
      type: Number,
      alias: 'playersCount',
      default: 4,
      min: 2,
      max: 4,
    },
    _s: {
      type: Number,
      alias: 'status',
      default: 0,
      enum: [
        0, //waiting for player
        1, // waiting for select hakem
        2, // playing
        3, //game over
      ],
    },
  }
)
export default model('game', GameSchema)

usage

 async create(data, user) {
    const players = new Array(data.playersCount || 4).fill(null)
    players[0] = user.id
    const newGame = new Game({
      ...data,
      owner: user.id,
      players: players,
    })
    const board = await newGame.save()
   const pop =    await board.populate({ path: '_pl', select: '_n', options: { retainNullValues: true } })
// this pop is not populated
    const res = await pop.toJSON({ virtuals: true })
    return {
      board: res,
      channel: createGameChannel(res.id),
    }
  }

expectation as return

in case using const pop = await board.populate({ path: '_pl', select: '_n', options: { retainNullValues: true } })
it return only refrence id like below

 {
  _r: 7,
  _pr: false,
  _pl: [
    5e62c3ebb304b991aa92bedf,
    5e66d697636392d78c267c80,
    5e6bc110e5ae9b25315c03a1,
    5e62dbe88af6a6986dd12d04
  ],
  _p: 4,
  _s: 1,
  _id: 5e6bd5ce63a31d33985f909d,
  _t: 'a',
  _o: 5e62c3ebb304b991aa92bedf,
  __v: 3
}

working fin on call back

but on call back await x.populate({ path: '_pl', select: '_n' }, console.log) working fine

{
  _r: 7,
  _pr: false,
  _pl: [
    { _id: 5e62c3ebb304b991aa92bedf, _n: 'masoud' },
    { _id: 5e66d697636392d78c267c80, _n: 'mina' },
    { _id: 5e6bc110e5ae9b25315c03a1, _n: 'mitra' },
    { _id: 5e62dbe88af6a6986dd12d04, _n: 'soroush' }
  ],
  _p: 4,
  _s: 1,
  _id: 5e6bd5ce63a31d33985f909d,
  _t: 'a',
  _o: 5e62c3ebb304b991aa92bedf,
  __v: 3
}

Do you want to request a feature or report a bug?
Its a bug i think
What is the current behavior?

If the current behavior is a bug, please provide the steps to reproduce.

What is the expected behavior?

What are the versions of Node.js, Mongoose and MongoDB you are using? Note that "latest" is not a version.

nodejs: 10
"mongoose": "^5.9.2",
db version v4.2.3

[AbdelrahmanHafez]

EDIT: Look at the next comment.

~Can confirm this is an issue, I simplified the script a little bit.~

const mongoose = require('mongoose');
const { Schema } = mongoose;
const assert = require('assert');

mongoose.connect('mongodb://localhost:27017/8671', { useNewUrlParser: true, useUnifiedTopology: true });

const commentSchema = new Schema({ content: String });
const Comment = mongoose.model('Comment', commentSchema);

const postSchema = new Schema({ commentsIds: [{ type: Schema.ObjectId, ref: 'Comment' }] });
const Post = mongoose.model('Post', postSchema);

async function run () {
  await Promise.all([
    Post.deleteMany(),
    Comment.deleteMany()
  ]);

  const [comment1, comment2] = await Comment.create([{ content: 'first comment' }, { content: 'second comment' }]);

  const post = await Post.create({ commentsIds: [comment1._id, comment2._id] });
  await post.populate({ path: 'commentsIds' });

  assert.equal(post.commentsIds[0].content, 'first comment');
}

run().catch(console.error);

The reason this is happening is because Document.populate() does not return a promise, nor a thenable. We're not getting an error because await wraps the value that populate returns into a promise that immediately resolves to the same value.

If we were to use post.populate().then() we would receive an error populate().then is not function.

I'll be looking into it.

[AbdelrahmanHafez]

Oh, this seems to be by design.

For that, we'll need to use .execPopulate() like that

await post.populate({ path: 'commentsIds' }).execPopulate();

@vkarpov15 Wouldn't it be neat if we made Document.prototype.populate(...) thenable, and chainable? Just like Model.find().populate().select()

[soroushm]

@AbdelrahmanHafez I already handle that like you say
await board .populate({ path: '_pl', select: '_n', options: { retainNullValues: true }, }) .execPopulate()
but its gonna be extra excute and triky!
this populate not working as the documentation says so its look like a bug or maybe should update docs
populating-multiple-paths

[AbdelrahmanHafez]

The example in the referred-to-documentation uses Model.populate(), which is different than Document.prototype.populate()

The use case in the first comment (Document.prototype.populate) is that we have found a document, and later we found that we needed to populate some fields for that document.

const board = await Game.findOne({ _id: someGame._id });
// here the API only supports execPopulate();
await board.populate({ path: '_p1' }).execPopulate();

Using Model.populate() however, supports chaining, and is a thenable.

// notice Story is a model, not a document
Story.
  find(...).
  populate('fans').
  populate('author')

Please notice that mongoose queries are thenables, but are _not_ real promises. Read more here.

Also, now that I have given it some though, I don't think we can make Document.prototype.populate thenable without introducing a breaking change.

[soroushm]

@AbdelrahmanHafez Its good to have populated as the prototype in future but for now, I prefer this line of code to docs

const board = await Game.findOne({ _id: someGame._id });
// here the API only supports execPopulate();
await board.populate({ path: '_p1' }).execPopulate();

[AbdelrahmanHafez]

I believe it's already present in the documentation

NOTE:
Population does not occur unless a callback is passed or you explicitly call execPopulate(). Passing the same path a second time will overwrite the previous path options. See Model.populate() for explaination of options.

[vkarpov15]

We have an issue to track this change: #3834 . We'll make it so that Document#populate() returns a thenable for 6.0.