Mongoose: Is it possible to populate without `_id` ?
I checked the API http://mongoosejs.com/docs/populate.html and tried out. It seems that populate can only works on foreign _id. Even there is a match option, it just add more filtering condition on _id query condition. There is no way to just populate with fields not involved with _id.
In my project, we keep the _id as ObjectId and has another field as auto-increment Number. For example, customers model has _id as Object and cuid as incremental number. Other collections just reference to cuid not to _id.
Is would be great to be able to customize this?
Is there any design limitation causing this feature missing?
mocheng
All 82 comments
It would be great to be able to customize this, but right now we enforce _id. I can't think of a good reason off the top of my head why not though. I'll have to investigate further.
vkarpov15
on 1 Jan 2015
The ability to populate not just on a single other field but even on a set of fields is almost required to properly support sharding, which is a huge use case for us.
For example, we have a collection "students" which may have a shard key for "subscription"; when we populate student data from our sections model (which contains an array of students) we have the subscription on the document but it will only look up based on the _id stored in "student" rather than allowing us to specify that it should also map the documents "assignment" field to "assignment" on the student.
Since we'd then be doing a lookup without the shard key when doing populate it renders sharding useless, requiring all shards to be hit to retrieve the data instead of just one.
This is something we would be willing to fund development for or even do ourselves if we could get some guidance of where to do the work. It is a major need for our company (GradeCam).
taxilian
on 20 Feb 2015
@taxilian are you setting the shardKey schema option? AFAIK that's what that option is for.
vkarpov15
on 20 Feb 2015
I haven't actually moved to start doing it yet because I've been looking for a solution to this issue; does populate take into account shardKey? My understanding was that shardKey only affected updates and inserts
taxilian
on 20 Feb 2015
It would be great to be able to specify the field which is to be "joined by", for example I'd like to be able to populate not only by userId (i.e. _id) field but also by userType, userGroup etc.
Thanks!
arliber
on 19 Apr 2015
+1
podoler
on 19 Apr 2015
+1 would be very useful
nadavsinai
on 30 Apr 2015
+1 same here!
xr
on 9 May 2015
+1 love to have this
prakash122
on 21 May 2015
+1
ricolo
on 24 May 2015
+1
vlapo
on 4 Jun 2015
+1
It would be especially nice to be able to select the variable you would like to populate with by adding a parameter such as key:
For example if had posts that I wanted to populate the author and comments parameters of, but the parameters are holding non ObjectId parameters (username of user and title of comments) before population:
{
author: "user1",
comments:["comment5", "comment6"],
title:"Post1"
}
It would be nice to be able to add a parameter like "key" to let mongo know which parameter to lookup by:
Posts.find().populate([{path:"author", model:"Users", key:"username"}, {path:"comments", model:"Comments", key:"title"}])
This would fill the author parameter with a User found by username as well as fill the comments parameter with comments found by their title (I realize the comments by title is a little unrealistic, but you get the point).
prescottprue
on 8 Jun 2015
+1
apankov
on 10 Jun 2015
+1
marklar21
on 14 Jul 2015
+1
john-bower
on 31 Jul 2015
+1
jack-guy
on 10 Aug 2015
+1
asafam
on 23 Aug 2015
+1
whyhankee
on 25 Aug 2015
+1
ArcoMul
on 25 Aug 2015
Big agree. I'd love the ability to define custom population schemes. It'd be great for any schema with one or more fields that are file paths, allowing populate to read from storage.
cannonpalms
on 3 Sep 2015
+1
raphaelokon
on 11 Sep 2015
+1
naoxink
on 14 Sep 2015
+1
manasb-uoe
on 7 Oct 2015
+1
Poplava
on 9 Oct 2015
+1
lfk
on 20 Oct 2015
+1
RithwikAnand
on 31 Oct 2015
+1
justvisiting
on 6 Nov 2015
+1
kumasento
on 13 Nov 2015
+1
BentoumiTech
on 14 Nov 2015
+1
loris
on 17 Nov 2015
+1
tomerb15
on 18 Nov 2015
+1
keithl8041
on 20 Nov 2015
+1
danielflippance
on 22 Nov 2015
+1
prabhupadas
on 24 Nov 2015
+1
leonardokl
on 17 Dec 2015
+1
robjens
on 18 Dec 2015
+1
beinnova
on 21 Dec 2015
+1
MangeshPimpalkar
on 24 Dec 2015
+1
roygwsidev
on 2 Jan 2016
+1
docksteaderluke
on 12 Jan 2016
+1
wohugb
on 15 Jan 2016
+1
joshbeam
on 25 Jan 2016
+1
pavan612
on 28 Jan 2016
+1
kennethkoontz
on 30 Jan 2016
+1
R0R0N0A
on 4 Feb 2016
+1
liraniom
on 9 Feb 2016
+1
Pyo25
on 11 Feb 2016
+1
danilvalov
on 17 Feb 2016
Try this code: https://github.com/alexmingoia/mongoose-populate-virtuals.
Actually, any reason for not merging Alex's code into main repo and make this a core feature?
rturk
on 18 Feb 2016
@rturk I really really like that plugin, thanks for pointing it out. @alexmingoia what are your thoughts re: merging your module into mongoose core? The idea of using virtuals to handle more generic populate (without having an actual field persisted to the db) is really powerful.
vkarpov15
on 18 Feb 2016
That would be ideal. It'd be much better in core because it overloads Query.prototype.populate and other core functionality.
The only thing blocking me from creating a pull request is that there's little to no test coverage, and documentation is lacking. I am using it in production in multiple scenarios and it's stable, but I think if someone were to write a test suite it might expose some minor issues or idiosyncrasies that need to be documented.
Is someone willing to tackle the test suite? If not - I can do it but don't expect that to land before next month.
alexmingoia
on 18 Feb 2016
I'll take a look. I've been meaning to rewrite populate anyway :)
vkarpov15
on 19 Feb 2016
Ping me on IRC (freenode, amingoia) or gitter if you have any questions about the code.
alexmingoia
on 19 Feb 2016
https://github.com/whitecolor/mongoose-fill also does a great job to populate using data from other models
sibelius
on 28 Feb 2016
+1
BenSower
on 2 Mar 2016
+1
juanpujol
on 3 Mar 2016
+1
obengwilliam
on 9 Mar 2016
this is a must :)
gitawego
on 9 Mar 2016
+1
amit3vr
on 11 Mar 2016
+1
teja-paketts
on 18 Mar 2016
+1
stewwan
on 22 Mar 2016
+1
nguyentrongbang
on 26 Mar 2016
+1
ingoncalves
on 5 Apr 2016
+1
markotom
on 7 Apr 2016
+1
forksofpower
on 12 Apr 2016
+1
ashe540
on 17 Apr 2016
+1
BenBestmann
on 26 Apr 2016
+1
realbisoye
on 11 May 2016
+1
rychkog
on 13 May 2016
+1
atamas101
on 18 May 2016
+1
mogita
on 19 May 2016
+1
samidarko
on 20 May 2016
+1
zing777
on 7 Jun 2016
+1
oguzhanaslan
on 28 Jun 2016
thatisuday
on 20 Jul 2016
+1
atakangktepe
on 6 Oct 2016
+1
anaibol
on 26 Oct 2016
+1
teemuniiranen
on 17 Feb 2017
+1
coderafei
on 2 Mar 2017
+1
fbegue
on 26 Mar 2017
+3
anaibol
on 31 Mar 2017
+1
omarps
on 16 Jun 2017
Related issues
Soviut
路
3Comments
jeneser
路
3Comments
ArThoX
路
3Comments
Igorpollo
路
3Comments
adamreisnz
路
3Comments
Most helpful comment
@rturk I really really like that plugin, thanks for pointing it out. @alexmingoia what are your thoughts re: merging your module into mongoose core? The idea of using virtuals to handle more generic populate (without having an actual field persisted to the db) is really powerful.