Searchkick: Support for boosting results by recency?

[cdimartino]

Hi,

I've been trying to get this to work in searchkick for some time now, and continue to fail :(

I have a corpus of about 600K news articles indexed that I need to search and boost the results based on recency. I've attempted numerous combinations of boosting (using the various boost_ methods) on date_added etc but to no avail. The only solution that worked was to use the sort methods, but that ended with very un-relevant search results at the top because they were more recent.

Any hints? I dont doubt that ES can handle it... just not sure how to communicate it to it!

[ankane]

You'll need to modified the generated query to accomplish this.

query = Article.search "news", execute: false
# make changes to query.body
# then
articles = query.execute

https://github.com/ankane/searchkick#advanced

[cdimartino]

Understood thank you. It looks like I'll probably have to do a function
score query with a decay function on the date.

Thanks again for the quick reply

Chris

On Wednesday, July 23, 2014, Andrew Kane [email protected] wrote:

You'll need to modified the generated query to accomplish this.

query = Article.search "news", execute: false# make changes to query.body# thenarticles = query.execute

https://github.com/ankane/searchkick#advanced

—
Reply to this email directly or view it on GitHub
https://github.com/ankane/searchkick/issues/241#issuecomment-49832737.

• Chris

ps - I am mobile, so please excuse any typos or Siri translation errors!

[ankane]

No problem, feel free to update this issue with your solution. I'm sure others will find it useful. I'd like to compile common tasks like this into the readme (or even the gem).

[robbi5]

Hi,

I had the same problem, after reading the function score query documentation and the release blogpost for function scores I came up with following solution:

query = Article.search "news", execute: false

# boost newer articles
unboosted_query = query.body[:query]
query.body[:query] = {
  function_score: {
    query: unboosted_query,
    functions: [
      { boost_factor: 1 },
      {
        gauss: {
          created_at: {  # field you want to boost
            scale: "4w"  # 4 weeks
          }
        }
      }
    ],
    score_mode: "sum"
  }
}

articles = query.execute

Thanks for the great gem :)

[cdimartino]

This is what I did to boost by 'created' date:

q.body[:query][:function_score] = { query: nil }
q.body[:query][:function_score][:query] =
  if q.body[:query].has_key? :dis_max
    {
      dis_max: q.body[:query].delete(:dis_max)
    }
  else
    {
      more_like_this: q.body[:query].delete(:more_like_this)
    }
  end

q.body[:query][:function_score].merge!({
  functions: [
    exp: {
      created_at: {
        scale:  '1m',
        offset: '7d',
        decay:  0.5
      }
    }
  ]
})
q.execute

[peddinti]

I tried using this but i am getting the following error:
Unknown field [create_time]

Is this because my field is a DateTime instead of Date? does this only work for DateFields?

[mahemoff]

Via #300, I found this can be done directly using boost_by_distance. It works for any range related data, including durations, not just physical distance.

Order.search 'coffee', boost_by_distance: {
  field: :ordered_at,
  function: :gauss,
  origin: Time.now,
  scale: '1w',
  offset: '0',
  decay: 0.5
}

(Note I have date fields indexed as Time type, not sure if it will work w/ standard Rails timestamps.)

[zerokol]

@mahemoff , thanks, your discovery was amazing!

[mahemoff]

Thanks @zerokol.

Note that boost_by_distance doesn't support a list arg (unlike boost_by, which can take a list or a hash), so you can only boost by one variable this way. If you want to boost by more than one distance factor (e.g. boost by both age and date of last publication), you would still need the workarounds posted earlier in this thread.

Hopefully boost_by_distance will accept a list in the future.

[ankane]

Happy to accept a boost_by_recency option to make this easier for people.

[Tybot204]

:+1: to adding a custom Searchkick option for this. I have a situation where I need to boost by both location and time in the same query. My application has a list of events with locations and times which need to be "sorted" by both attributes simultaneously.

No one cares about events starting in 5 minutes on the other side of the world or events starting in 5 years next door!

[pplant]

Is there a way to add a scale factor to this

boost_by_distance: {
field: :ordered_at,
function: :gauss,
origin: Time.now,
scale: '1w',
offset: '0',
decay: 0.5
}

Since it returns scores from 0-1 only and I would like to multiply it by a factor of 1000.

[pplant]

A brief review of the library code revealed that this is not possible yet. Therefore I implemented an easy way to achieve this and created a pull request https://github.com/ankane/searchkick/pull/1028

[TechCanuck]

https://github.com/ankane/searchkick/pull/1028 - Correction of the previous link

[bbugh]

To confirm for anyone else reading this, you can use Rails timestamps just fine. The example above scale unit ("w") raises an error, so we just use "7d" instead. This works great for us:

boost_by_distance: {
  field: :created_at,
  origin: Time.current,
  scale: '7d',
  decay: 0.5,
  factor: 1500
}

[ankane]

https://github.com/ankane/searchkick/commit/2fd44addd3ba4d72468469b7782158323a6e666f