Elasticsearch-dsl-py: Iteration gives me 10 items only

[rokcarl]

If I search and iterate over the results, I get 10 items only even though the total hits is much higher.

>>> result = Link.search().filter("term", run_id=3).execute()
>>> print 'Len: {}, Slice len: {}, Hits: {}'.format(len(list(result)), len(result[10:20]), result.hits.total)
Len: 10, Slice len: 0, Hits: 400

I would expect the numbers to be 400, 10 and 400.

Am I doing something wrong?

[amitpandita]

@rokcarl there is nothing wrong that you're doing.
Elasticsearch Search api by default returns 10 documents. Check this link

http://elasticsearch-py.readthedocs.org/en/latest/api.html

and look for search size. You can further read on how to override the default count.

Hope that helps!

[njoannin]

@rokcarl You can use the slice operator to retrieve the results you want.
The way I would do it is:

s = Link.search().filter("term", run_id=3)
count = s.count()
result = s[0:count].execute()

Hope this helps.

[rokcarl]

Ah, thanks @njoannin, I should have sliced _before_ the execute().

[ellemac123]

To access all documents you can use the scan method s.scan():

for hit in s.scan(): print(hit.title)

located in docs: http://elasticsearch-dsl.readthedocs.io/en/latest/search_dsl.html#pagination