Elasticsearch: Seeded random ordering (Feature request)

[pawpro]

Implementation of random result ordering with optional seed allowing for recreation of random order i.e. across "pages" when using pagination (from, size).

[mattiassvedhem]

I'm really interested in this, did someone try to implement this?

[pawpro]

I discussed this briefly with kimchy at the time and his feedback was that it wouldn't be hard to do. Definitely a great feature candidate.
I have implemented this by storing set of "random" numbers e.g. each object has say 20 additional integers (each one of these is a sorting field named as rand1, rand2, rand3 etc). For each user I select (calculate based on date and UA) number 1...20 which is used to select which "random" field will be used. When new objects are added each simply receives set of random. Umbers. You decide how many random states you need. Remember that reverse order in this case doubles the number of random states. This solution is all about perception.

[mattiassvedhem]

Cool! I'll try that.

[Evan-R]

Has this been implemented into the current version of ES yet?

[s1monw]

@uboness can we pull your latest commit in for this?

[emarthinsen]

This would be great if it were added.

[missinglink]

BUMP++

[uboness]

random order is now supported as part of the new function score queries. request example:

curl -XGET 'localhost:9200/_search' -d '{
  "query": {
    "function_score" : {
      "query" : { "match_all": {} },
      "random_score" : {}
    }
  }
}';

or with a seed (for near-consistent pagination):

curl -XGET 'localhost:9200/_search' -d '{
  "query": {
    "function_score" : {
      "query" : { "match_all": {} },
      "random_score" : { "seed" : 1376773391128418000 }
    }
  }
}';

Check out more on function score queries here: #3423

[emptyflask]

This is immensely helpful. I hope we'll see this in v0.90.4 in a matter of days...

[s1monw]

@emptyflask can you tell us a bit about your actual usecase for this? I am very curious and it would be good to have a couple of usecases on the reference documenation

[emptyflask]

@s1monw I'm currently doing this with Solr, and it's the one thing that's tying up my migration to ES on that app. There are a few thousand user-submitted ideas that receive votes from other users, so to give every idea an equal chance of showing up on the first page, we want the default view to be randomly sorted, yet paginated and consistent for each user. Using the current user's id as a random seed solves this problem nicely.

[missinglink]

:+1: 10 points for @uboness

[brupm]

:+1: Will this make it into 0.90.4?

[s1monw]

@brupm it's already pushed to 0.90 branch so yes it will be in 0.90.4

[damienalexandre]

Awesome feature, thx!

Another use-case is a Wikipedia-like random page - coupled to some search criteria (to avoid low interest pages) it's much faster than querying the DB for a random ID.

[brupm]

@s1monw Any idea when 0.90.4 will be released? Thanks!

[kimchy]

we plan to release 0.90.4 early next week.

[dctdct]

"sort": [
{
"name": {
"order": "random"
}
}
],

Is not working :-(

[emptyflask]

@dctdct This is the way you should do it: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html

[dctdct]

Maybe this is a bug with the php client but passing this query as array doesn't work.

$json = '{
"query": {
"function_score": {
"functions": [
{
"random_score": {

                   }
                }
             ],
             "boost_mode": "replace",
             "query": {
                "match_all": {}
             }
          }
       }
    }';

    $qry = array(
        'query' => array(
            'function_score' => array(
                'functions' => array(
                    array('random_score' => array())
                ),
                'query' => array(
                    array('match_all' => array())
                )
            )
        )
    );

    $searchParams['body'] = $qry;

    $retDoc = $elastic->search($searchParams);

[yao23]

How to iterate every items in each page with the random score function?

[emptyflask]

Same way you would iterate over any other set of results. The function
score only affects the score attribute of each document, which reorders the
results. Everything else remains the same.
On May 16, 2014 4:28 PM, "Yao Li" [email protected] wrote:

How to iterate every items in each page with the random score function?

—
Reply to this email directly or view it on GitHubhttps://github.com/elasticsearch/elasticsearch/issues/1170#issuecomment-43381486
.

[bato3]

@dctdct for PHP use new stdClass() instead Array() This is json_encode "feature"

[johnfrades]

Sweet! thanks @uboness