Elasticsearch-dsl-py: Multiple levels of nesting

[mh00h]

Is it possible to have multiple levels of nesting defined in persistence using elasticsearch-dsl-py?

[HonzaKral]

Absolutely, I assume you mean nested fields?

class Company(DocType):
    departments = Nested(properties={"employees": Nested()})

[mh00h]

Yes exactly. Following the documentation, would it be something like this instead?

class Department(InnerObjectWrapper):
    pass

class Employee(InnerObjectWrapper):
    pass

class Company(DocType):
    departments = Nested(
        doc_class = Department,
        properties = {
            'name' : String(),
            'employees' : Nested(
                doc_class = Employee,
                properties = {
                    'lastName' : String(),
                    }
                ),
            }
        )

[HonzaKral]

Yes, that is exactly what it would look like.

[mh00h]

Thank you HonzaKral.

[avelis]

@HonzaKral Can the properties be defined within each InnerObjectWrapper or does it have to be explicitly passed into the Nested class as a properties dictionary key/value argument?

[HonzaKral]

@avelis not on the InnerObjectWrapper but on the Object/Nested field instead:

class Company(DocType):
    department = Nested()
    department.field('title', Text())
    department.field('tags', Keyword(multi=True))

[petergaultney]

Is it possible to define multilevel nesting by breaking it out into multiple classes? This syntax starts to be very unwieldy with multiple levels of nesting.

Example:

class BurrowKafkaPartitionWindowMarker(Nested):
    offset = Integer()
    timestamp = Date()
    lag = Integer()

class BurrowKafkaPartitionStatus(Nested):
    topic = String()
    partition = Integer()
    status = String()
    window_start = BurrowKafkaPartitionWindowMarker()
    window_end = BurrowKafkaPartitionWindowMarker()

class BurrowKafkaStatus(Nested):
    cluster = String()
    group = String()
    status = String()
    complete = Boolean()
    partitions = BurrowKafkaPartitionStatus(multi=True)
    partition_count = Integer()
    maxlag = BurrowKafkaPartitionStatus()
    totallag = Integer()

class BurrowKafkaLagReport(DocType):
    error = Boolean(required=True)
    message = String(required=True)
    status = BurrowKafkaStatus()
    request_url = String(required=True)

If I used the properties = { ... } syntax for this, it would look horrendous.

Unfortunately my cleaner syntax doesn't seem to work, for whatever reason. Nothing in the BurrowKafkaLagReport.status object gets saved to Elasticsearch.

[Audace]

@petergaultney I agree. The best solution I have found is the one detailed by @JoshCoady here. It would be nice if there were documentation or support for doing this natively.

[HonzaKral]

@petergaultney thank you for the inspiration, I started an implementation as https://github.com/elastic/elasticsearch-dsl-py/pull/790

any feedback is more than welcome!