Parse-server: Randomly get objects from Mongodb

[lastMove]

I would like to add to ParseServer the possibility to get objects from a collection Randomly.
Similar to mongoDb random aggregation : https://docs.mongodb.org/manual/reference/operator/aggregation/sample/

Is this possible ?

[lacker]

Hmm in general I think it would be cool to make it easier to drop down to the database to do things directly there. I don't think it is ideal to make this an operator in the Parse query lagnauge though.

[flovilmart]

it would require an update on all SDK's... that's the most problematic part of it...

[lastMove]

@flovilmart
True, but not really a problem since it will not be a breaking change. If it is not implemented immediately in the differents SDK they will just work as before. I think we should add it to the Rest Client SDK.

[lacker]

True. IMO adding it to the rest API seems reasonable, just need to figure out the naming + docs stuff. Want to send a pull request?

[flovilmart]

I would name it sample=[n], same citizen as size as limit. where n is the size of the sample. if sample is set, ignore limit, size and skip

[lacker]

Oh hmm I didn't realize this was part of the "aggregation pipeline". Is there a normal query operator that does a random selection? AFAIU the aggregation stuff is not ideal ops-wise to run lots of.

[flovilmart]

it doesn't seems like it. I understand the need for the feature, but that can be mocked with multiple queries with a random skip/limit + concatenating the results.

Opening to aggregation pipelines, I'd go for 'group' more than 'random', or enabling map/reduce in cloud code, which could be useful for the abstraction.

Otherwise it's a matter of exposing the DBController that can perform the queries for more advanced users.

[lastMove]

Ok Now 2 mains solutions IMO :

• implementing this as .Find option in DatabaseController (like limit, skip and count), named sample as @flovilmart said.
• Adding the ability to pass in custom "mongoWhere" object to find though SDKS (like @lacker said)

I personnaly prefer the first solution since it a frequent Requested features and it's easier to use.
The second solutions adds one more dependancy to mongoDB #20

[tobernguyen]

+1 for randomly get object and it should be done in database stack as MongoDB already supported it

[drew-gross]

It's also possible to do this using a query on objectId, since objectIDs are random, eg. for 1000 random objects to a query with limit 1000 ordered by objectId. You can't use this technique to get a different sample with every query though, but it could work for some use cases.

[raubas]

Would be a great feature, since a lot of people have been requesting it on parse.com (myself included, ended up writing cloudcode to achieve it).

[flovilmart]

PR are welcome!

[lastMove]

I am working on it ;)

[flovilmart]

Awesome!!

[lastMove]

Hi,

I had almost finish the pull request.
But I faced a little (actually a big one) problem…

Parse’s mongodb version don’t support sample. Mongo supports ‘sample’ from mongo 3.2 (https://docs.mongodb.org/manual/reference/operator/aggregation/sample/)
and parse uses mongo 2.1.*
Yes this is a beginner error of only seen the end

So, I have some questions :

1. Did you plan to upgrade Parse-server mongo version ? ( I know, there is some breaking changes, but i don’t know if it affects Parse-server)
2. I can still implement the sample by using some workarounds so that when parse upgrades to mongo 3.2+ it will be easy to use the true path. ( I don’t think it will be useful since the best workarounds depends of user’s needs.)
3. Is there a way to do the sample efficiently using mongo 2.1.*

[drew-gross]

1. Upgrading the mongo version isn't near the top of our roadmap although we do plan to generally keep our dependencies up to date.
2. This is going to depend on the workaround. If it's a workaround that will be easy to remove when we upgrade the mongo version, it's probably fine.
3. A workaround we use at Parse is to add an objectID < "1" constraint or something like that. It's a pretty limited technique compared to the built in sample aggregation, but it worked for us.

[montymxb]

Going to go ahead and close this out, been quite some time and a functional random implementation can be done client side on any of the SDKs without having to make further modifications.

If any other core developers have a moment, or other contributors are willing to give this a shot go ahead and setup a PR so we can have this actively moving along. Until then we'll put this aside.