Elasticsearch-dsl-py: ElasticSearch 6 support?

[braunsonm]

Does this repository support ES6?
Or is that still on the way?

Thanks

[chouseknecht]

Nope See #767

[braunsonm]

Is the feature not planned then? Why is it closed?

[HonzaKral]

It is planned, hopefully soon. Unfortunately i haven't had much time to devote to it but it is high on my priority list

[braunsonm]

Thanks @HonzaKral I would help if I knew more about ES but I feel like I'd get in the way.

All the best!

[HonzaKral]

master should now fully support elasticsearch 6.0.0. I will hold of on doing an actual release though since there are still some details to be worked out around the conventions we promote in the library that need to change because of the type removal...

[nrsimha]

@HonzaKral It would be nice if #454 will be committed before new release.

[okke-formsma]

@HonzaKral, can we do anything to help you preparing the 6.0 release?

[HonzaKral]

I would love to get a better Nested solution if I have time (see [0] for details) and change DocType/Search connection to be able to work based on _index name instead of just _type as it is now ([1]). Finally I would like to move to having DocType's name default to just doc for fw compatibility with elasticsearch 7.

Also introduce better/more (de)serialization as per #785

Since those would be breaking changes I would like to have them in before 6.0 is released to public.

If you would like to help with any of those I would be super happy, for any stage of the process - design review, implementation, testing, docs, ... I will try to tackle it later this week if I find some time. Thanks!

0 - https://github.com/elastic/elasticsearch-dsl-py/issues/419#issuecomment-275457376
1 - https://github.com/elastic/elasticsearch-dsl-py/blob/master/elasticsearch_dsl/response/__init__.py#L43

[3lnc]

+1 for refactoring Nested/Objects. Tbh, I have no idea how to implement this in good way but for me, it would be awesome to have "class-oriented" interface (think DocType without Meta options). I.e.

class Nested(Something): ...
class Object(Something): ...

class Fragment(Nested):  # maybe even DocType here
    num = Integer()
    text = Text()

class Doc(DocType):
    nested = Fragment(dynamic=True, enabled=False)

My vision on this – make obj/nested more "high level" classes, to be able to describe subfields with same Field descriptors as we do for DocType. properties should be collected from descriptors, multi/dynamic/enabled should be set in __init__. For now, this looks as good reusable and more intuitive approach, for me at least.

[3lnc]

Implementation for serialization (for #785 ) should be low-effort (god bless python), as almost everything will be covered with int/float. The question is to find common ground in approach to this.

[De]serialization per-field lgtm, and this kinda follows old django approach to keeping data clean, but at the same time violated DRY corresponding to ES itself (personally, I don't think so, as ES-side coercion is not something that most ppl expect).

Overall – I think we can try to make this change to dsl, keeping elasticsearch-py pure.

[HonzaKral]

See https://github.com/elastic/elasticsearch-dsl-py/pull/790 for Nested/Object refactoring. Need to make sure it works properly with inner_hits queries and clean it up a lot, replacing the current syntax for the new one.

[bpiselli]

+1

[azhard4int]

+1

[HonzaKral]

So all the features mentioned here have been implemented in master. Any testing is more than welcome, I hope to get a release out later this week. Thank you all for participation, it was a great help!