Describe the feature:
The new search_after
parameter is awesome for paging through huge result sets! I'm trying to connect ElasticSearch to a GraphQL API using the Relay connection spec, and I'm hitting a problem where I'm unable to determine whether there are additional results before or after the current set. I can work around it for the moment by retrieving one more result than I need on the back end of the set, then executing another query for a single result before the set (reversing the order). This seems clunky though. Is it possible to get a way to either determine the presence of forward and backward records, or to determine the current set's position within the overall result? I can see this being a requirement for a lot of use cases.
Given the way things are implemented, we could theoretically count how many documents we ignore due to the fact that the compare less than the sort values that are provided in search_after
. However, this means updating responses to the search API to include thiss information, modifying the Lucene collectors to expose this offset and more generally propagating this information from the collector up to the search response, so that would not be a trivial change. I suggest that we keep this issue open for some time in order to gauge how much interest it gathers.
cc @elastic/es-search-aggs
Since we did not see much interest in this issue, and because it will require a lot of modifications, and there is a workaround from the client size, I am closing this issue.
Most helpful comment
Given the way things are implemented, we could theoretically count how many documents we ignore due to the fact that the compare less than the sort values that are provided in
search_after
. However, this means updating responses to the search API to include thiss information, modifying the Lucene collectors to expose this offset and more generally propagating this information from the collector up to the search response, so that would not be a trivial change. I suggest that we keep this issue open for some time in order to gauge how much interest it gathers.