Elasticsearch: search_after -> Failed to parse search_after value for field [__anonymous_date].

[GuillaumeCisco]

Elasticsearch version: 5.2.2

Plugins installed: []

JVM version:

OS version: Linux

Description of the problem including expected versus actual behavior:

I was trying the search_after option for paginate large sorted dataset (>10000) but came across an error I can't explain.

I've got logs object in my elasticsearch storage with fiedls like @timestamp, abs_timestamp, device : {id}, category, message and more.

Context : page_size = 500
I first make a query like that (page 1 example):

{'from': 0,
 'query': {'bool': {'filter': [{'term': {'tag': u'headband.logs'}}],
                    'must': [{'match': {u'device.id': UUID('ce510315-25ec-4455-8b90-7b7761e0ce15')}},
                             {'range': {u'abs_timestamp': {u'format': u"yyyy-MM-dd'T'HH:mm:ssZ",
                                                           u'gte': '2017-03-09T08:59:55+0000',
                                                           u'lte': '2017-03-10T08:59:55+0000'}}}]}},
 'size': 500,
 'sort': [{u'_uid': {u'order': u'desc'},
           u'abs_timestamp': {u'order': u'asc', u'unmapped_type': u'date'}}]}

I get my objects correctly.
Due to restriction with from/size, a query with a top + page_size > 10 000 will fail :
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-from-size.html

So I decided to use the new search_after option like that:

{'from': -1,
 'query': {'bool': {'filter': [{'term': {'tag': u'headband.logs'}}],
                    'must': [{'match': {u'device.id': UUID('ce510315-25ec-4455-8b90-7b7761e0ce15')}},
                             {'range': {u'abs_timestamp': {u'format': u"yyyy-MM-dd'T'HH:mm:ssZ",
                                                           u'gte': '2017-03-09T08:59:55+0000',
                                                           u'lte': '2017-03-10T08:59:55+0000'}}}]}},
 u'search_after': [u'2017-03-09T14:34:16.026000Z', u'AVqzvRDe_-VptwkQaduX'],
 'size': 500,
 'sort': [{u'_uid': {u'order': u'desc'},
           u'abs_timestamp': {u'order': u'asc', u'unmapped_type': u'date'}}]}

I get the search_after values from my precedent query (last object returned)
When I execute this query I get an error:
RequestError: TransportError(400, u'search_phase_execution_exception', u'Failed to parse search_after value for field [__anonymous_date].')

I've looked into the code here https://github.com/elastic/elasticsearch/blob/master/core/src/main/java/org/elasticsearch/search/searchafter/SearchAfterBuilder.java and can't find a way to correct my query.

I don't know if it's a bug or if I didn't write my query correctly.

Thank you

[GuillaumeCisco]

Ok just found my error.
search_after is waiting for the sorted values from a precedent query (last object of the query)...
I was computing them myself from the last object. But it is already computed for me and I've been able to access it with my_last_object.meta.sort. I've been able to understand that the sorted value for the date was a timestamp (number) and not a representation string of the date like my_last_object.abs_timestamp gave me.

Thank you,

Hope it will help others ;)