Searchkick: Kaminari max_pages to limit result window too large

[FranciscoCG]

Hi @ankane You mentioned in #642 that an option to limit results is 'set max_pages in Kaminari'. Are you sure it works?

I've beed 2 month stucked with large result windows issue, I've tried many things, and my conclution is that the only way to do that is doing with some max_pages in kaminari or total_entries in will_paginate.

Here is a options summary I've done:

1) Increase ES max_result_window:
Not an option for million records. (kill the ES server)

2) Convert to Active Record relation:

Perfomance prohibitive.

Post.search("*", limit: 10000)
posts_ids = @searchkick_items.map{|p| p.id } #This is a HUGE array
@posts = Post.where(id: posts_ids).paginate(page: params[:page], per_page: 100)

3) Limit before passing to Searchkick:

Results has no sense, because we are searching over a subset of 10000
We can do a SQL query before to ensure the resultset has some sence and then query ES, but this is very fancy solution

Post.limit(10000).search("*")

[ankane]

Hey @FranciscoCG, don't think I understand what you're asking for help on.

[FranciscoCG]

Hi @ankane summarizing: You said in #642 that 'set max_pages in Kaminari' is an option for limiting result window. But it doesn't works for me. Did you test it works?

The rest of the post is a summary for future readers that will face the issue. I pointed out the 3 options I've tried for limiting result window for paginating. I

[licatajustin]

@ankane, he has a valid use-case in reference to trying to limit the total number of pages using Kaminari's max_pages configuration.

Seems to only work when using Kaminari without Searchkick, which would force the use of updating the Elasticsearch setting of max_result_window. Not ideal, but still a viable solution.

Let me know if I can help out or provide some more details 👊

[FranciscoCG]

@licatajustin you are right. Kaminari and will_paginate works well without Searchkick.

Updating Elasticsearch setting of max_result_window is not an option when we have millions results or the requirements is to narrow the resultset. In this cases, the only option is use Scrollapi which is also not supported Searchkick

[ankane]

Cleaning up stale issues. Unfortunately, I don't have the bandwidth to explain how to use max_pages, but maybe Stack Overflow could help.

[FranciscoCG]

Hi @ankane Why you closed this issue since is not resolved?

I'm afraid you don't understand. max_pages does not works with Searchkick. It's not a thing to search in StackOverflow.

The issue is add support for limit pagination results in: will_paginate and kaminari

[ankane]

Hey @FranciscoCG, it's closed because I don't plan to spend any more time on it. Feel free to submit a PR if you have a working solution to this feature request.

[FranciscoCG]

I understand, thanks.