Elasticsearch: Multi level nested objects `inner_hits` not working (only root level working)

Created on 24 Aug 2015 · 9Comments · Source: elastic/elasticsearch

Hi,

Here is the mapping:

{
    "mappings": {
        "people": {
            "properties": {
                "cars": {
                    "type": "nested",
                    "properties": {
                        "manufacturers": {
                            "type": "nested",
                            "properties": {
                                "country": {
                                    "type": "string"
                                },
                                "name": {
                                    "type": "string"
                                }
                            }
                        },
                        "model": {
                            "type": "string"
                        },
                        "make": {
                            "type": "string"
                        }
                    }
                },
                "last_name": {
                    "type": "string"
                },
                "first_name": {
                    "type": "string"
                }
            }
        }
    }
}

The test document is:

{
    "first_name": "Zach",
    "last_name": "Foobar",
    "cars": [{
        "make": "Saturn",
        "model": "SL",
        "manufacturers": [{
            "name": "Saturn",
            "country": "USA"
        }, {
            "name": "Honda",
            "country": "Canada"
        }]
    }, {
        "make": "Subaru",
        "model": "Imprezza",
        "manufacturers": [{
            "name": "Subaru",
            "country": "Japan"
        }, {
            "name": "Daimler",
            "country": "Germany"
        }]
    }]
}

and the query is:

{
    "query": {
        "nested": {
            "path": "cars",
            "query": {
                "nested" : {
                    "path" :  "cars.manufacturers",
                    "query" :  {
                       "match": {
                           "cars.manufacturers.country": "Japan"
                        }
                    },
                    "inner_hits": {}
                }
            },
            "inner_hits": {}
        }
    }
}

ElasticSearch gives the search result:

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 2.252763,
    "hits": [
      {
        "_index": "abcindex3",
        "_type": "people",
        "_id": "Zach",
        "_score": 2.252763,
        "_source": {
          "first_name": "Zach",
          "last_name": "Foobar",
          "cars": [
            {
              "make": "Saturn",
              "model": "SL",
              "manufacturers": [
                {
                  "name": "Saturn",
                  "country": "USA"
                },
                {
                  "name": "Honda",
                  "country": "Canada"
                }
              ]
            },
            {
              "make": "Subaru",
              "model": "Imprezza",
              "manufacturers": [
                {
                  "name": "Subaru",
                  "country": "Japan"
                },
                {
                  "name": "Daimler",
                  "country": "Germany"
                }
              ]
            }
          ]
        },
        "inner_hits": {
          "cars": {
            "hits": {
              "total": 1,
              "max_score": 2.252763,
              "hits": [
                {
                  "_index": "metaportal3",
                  "_type": "people",
                  "_id": "Zach",
                  "_nested": {
                    "field": "cars",
                    "offset": 1
                  },
                  "_score": 2.252763,
                  "_source": {
                    "make": "Subaru",
                    "model": "Imprezza",
                    "manufacturers": [
                      {
                        "name": "Subaru",
                        "country": "Japan"
                      },
                      {
                        "name": "Daimler",
                        "country": "Germany"
                      }
                    ]
                  }
                }
              ]
            }
          },
          "cars.manufacturers": {
            "hits": {
              "total": 0,
              "max_score": null,
              "hits": []
            }
          }
        }
      }
    ]
  }
}

The root level inner_hits is working, but the multi-level nested objects inner_hits are not working. The cars.manufacturers inner hits is empty. Am I doing something wrong or is this a bug? From this doc it says that multi level inner_hits are supported and should return non-root level inner_hits.

:SearcSearch >bug

Source

RanadeepPolavarapu

Most helpful comment

@black-square The bug was only fixed in 5.0.x. Fixing it required a refactoring that didn't make its way back in 2.4

martijnvg on 3 Oct 2016

🎉2

All 9 comments

@RanadeepPolavarapu the inner_hits syntax in the query dsl doesn't support proper nested. For now you should use the top level inner_hits syntax: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-inner-hits.html#top-level-inner-hits

This issue is related to this ticket: #11118
Although this ticket is about parent/child inner hits, the the nested inner_hits in the query dsl doesn't work there too and this should be fixed

martijnvg on 24 Aug 2015

Closing as duplicate of #11118

clintongormley on 25 Aug 2015

@martijnvg How could I replicate that query correctly using top level inner_hits?

RanadeepPolavarapu on 27 Aug 2015

@RanadeepPolavarapu the following query should work:

curl -XGET "http://localhost:9200/_search" -d'
{
  "query": {
    "nested": {
      "path": "cars",
      "query": {
        "nested": {
          "path": "cars.manufacturers",
          "query": {
            "match": {
              "cars.manufacturers.country": "Japan"
            }
          }
        }
      }
    }
  },
  "inner_hits": {
    "cars": {
      "path": {
        "cars": {
          "query": {
            "nested": {
              "path": "cars.manufacturers",
              "query": {
                "match": {
                  "cars.manufacturers.country": "Japan"
                }
              }
            }
          },
          "inner_hits": {
            "manufacturers": {
              "path": {
                "cars.manufacturers": {
                  "query": {
                    "match": {
                      "cars.manufacturers.country": "Japan"
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}'

Downside that It is more verbose, because the query needs to be repeated on each nested level. When the embedded query dsl inner_hits syntax also properly supports multiple level of nested object fields and parent child relations then your query should work too.

martijnvg on 27 Aug 2015

+1
I am wondering why the reference includes this paragraph:
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-inner-hits.html#hierarchical-nested-inner-hits
which seems exactly the part that is not working

abulhol on 1 Feb 2016

This bug is closed as well as #11118, but the original example from @RanadeepPolavarapu still doesn't work on 2.4. Should it be reopen again?

black-square on 3 Oct 2016

@black-square The bug was only fixed in 5.0.x. Fixing it required a refactoring that didn't make its way back in 2.4

martijnvg on 3 Oct 2016

🎉2

Hi Guys,
In this data when i am trying with one level nested data then it's working fine but when i am adding one more nested data then it's not working if anybody will be knows about that then please reply me.I am sending my document data and mapping file also.

POST /test_word14/doc/1
{
"name": "Diseases of the blood and blood-forming organs and certain disorders involving the immune mechanism (D50-D89)",
"depth": 1,
"children": [{
"name": "Nutritional anemias (D50-D53)",
"depth": 2,
"children": [{
"code": "D50",
"name": "Iron deficiency anemia",
"depth": 3,
"children": [{
"code": "D50.0",
"name": "Iron deficiency anemia secondary to blood loss (chronic)",
"depth": 4
}, {
"code": "D50.1",
"name": "Sideropenic dysphagia",
"depth": 4
}, {
"code": "D50.8",
"name": "Other iron deficiency anemias",
"depth": 4
}, {
"code": "D50.9",
"name": "Iron deficiency anemia, unspecified",
"depth": 4
}]
}]
}]
}

(mapping file)
PUT /test_word16
{
"mappings": {
"doc": {
"properties": {
"name": { "type": "string" },
"depth": {"type":"long"},
"children": {
"type": "nested",
"properties": {
"name": { "type": "string" },
"depth": { "type": "long" },
"children": {
"type": "nested",
"properties": {
"code": { "type": "string" },
"name": { "type": "string" },
"depth": { "type": "long" },
"children": {
"type": "nested",
"properties": {
"code": { "type": "string" },
"name": { "type": "string" },
"depth": { "type": "long" },
}
}
}
}
}
}
}
}
}
}

please tell me what is the query i am applying here.i am applying the query like this:
GET /icd10_codes/doc/_search
{
"_source": false,
"query": {
"nested": {
"path": "children",
"query": {
"nested": {
"path": "children.children",
"query": {
"nested": {
"path": "children.children.children",
"query": {
"bool": {
"must": [
{
"match": {
"children.children.name": "Iron"
}
}
]
}
},
"inner_hits": {
"_source": {
"excludes":["name"]
}
}
}
}
}
}
}
}