Elasticsearch: _source doesn't work on parent inner_hits

[ericofusco]

Elasticsearch version: 5.0.0

Plugins installed: []

JVM version: Oracle JRE Server 8u112

OS version: Ubuntu 14.04

Description of the problem including expected versus actual behavior:

When _source is defined on inner_hits, it doesn't work the same way it works on outer hits. It gives the same error for any type I use. I guess it should work the same way is described on https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-source-filtering.html.

For example "inner_hits":{ "_source": "name" } it doesn't work.

Query example:

GET local/document/_search
{  
      "_source": "name",
      "query":{  
         "bool":{  
            "must":[  
               {  
                  "bool":{  
                     "should":[  
                        {  
                           "has_parent":{  
                              "type":"document_category",
                              "inner_hits":{  
                                "_source": "name"                                                   
                              },
                              "query":{  
                                 "multi_match":{  
                                    "fields":[  
                                       "name",
                                       "description"
                                    ],
                                    "query":"test",
                                    "operator":"and"
                                 }
                              }
                           }
                        }
                     ]
                  }
               }
            ]
         }
      }
   }
}

Error: [inner_hits] _source doesn't support values of type: VALUE_STRING

Steps to reproduce:

1. Create child and parent documents
2. Use _source within inner_hits in has_parent

[clintongormley]

Inner hits expects the object form of _source, not the short form accepted by the top-level _source pararameter

[ericofusco]

@clintongormley,

Thanks for the reply. Could you please give me a quick example for the "_source" ?

[clintongormley]

Sorry @ericofusco, I was just agreeing with you and adding some detail for @martijnvg

This form of _source works in inner hits:

"_source": { "includes": [...], "excludes": [...] }

[ericofusco]

@clintongormley ,

Alright. I guess I should keep this issue open until this is updated to accept the short form.

Using the object form as shown in your example works as expected.

Thanks again.

[sean-cherbone]

Additionally,
"_source": ["field1","field2"]
fails with error

[inner_hits] _source doesn't support values of type: START_ARRAY

but works as:
"_source": { "includes":["field1","field2"] }
I will assume this is the same issue.

If this is the intended behavior from here on out, it just needs to be stated as such in the documentation. The inner hits options documentation for 5.0 currently provides a link to the top level source filtering API; indicating that @ericofusco's code and mine should work.

I should also point out that this type of filtering worked in 2.4

[vincentfretin]

With elasticsearch 5.1.1, "_source": { "includes":["uid"] } indeed doesn't give an error but the returned _source is actually an empty {} instead of {"uid": "8599f82f97684c72a9620f5552358b5d"} I'm waiting in my case.

[Tobion]

I can confirm @vincentfretin. Source filtering on inner hits is bugged:

• Using the short source filtering syntax creates an error
• Using "_source": { "includes":["uid"] } returns an empty _source even when the field exists
• Using "_source": { "includes":["uid"], "exludes":["foo"] } returns the full _source and ignores the excluded field. Also the _source structure seems to have changed then. When part of a nested query the nested property is included in the source which didn't seem to be the case in earlier ES version, e.g. _source: { "nested property": {...fields...} }

Tested with version 5.0.2 and 5.1.2

[JohPa8696]

Hi! I am new to the project. Can I try to tackle this issue? Any advice or suggestion would be much appreciated.

[polyfractal]

As reported by @Chakrygin in #23829 (see ticket for reproduction), this appears to be broken in field collapsing as well. And also applies to disabling source entirely, not just various forms of filtering.

[clintongormley]

@talevy could you take a look at this please?

[abhishek5678]

Hi Guys,
In this data when i am trying with nested data then it's getting fine but when i am adding one more nested data then it's not working if anybody will be knows about that then please reply me.

POST /icd10_codes/doc/1
{
"name": "Diseases of the blood and blood-forming organs and certain disorders involving the immune mechanism (D50-D89)",
"depth": 1,
"children": [{
"name": "Nutritional anemias (D50-D53)",
"depth": 2,
"children": [{
"code": "D50.0",
"name": "Iron deficiency anemia secondary to blood loss (chronic)",
"depth": 4
}, {
"code": "D50.1",
"name": "Sideropenic dysphagia",
"depth": 4
}, {
"code": "D50.8",
"name": "Other iron deficiency anemias",
"depth": 4
}, {
"code": "D50.9",
"name": "Iron deficiency anemia, unspecified",
"depth": 4
}]
}]
}

Also sending my mapping file and query also
mapping file:
PUT /icd10_codes
{
"mappings": {
"doc": {
"properties": {
"name": { "type": "string" },
"depth": {"type":"long"},
"children": {
"type": "nested",
"properties": {
"name": { "type": "string" },
"depth": { "type": "long" },
"children": {
"type": "nested",
"properties": {
"code": { "type": "string" },
"name": { "type": "string" },
"depth": { "type": "long" }
}
}
}
}
}
}
}
}

QUERY:
GET /icd10_codes/doc/_search
{
"_source": false,
"query": {
"nested": {
"path": "children",
"query": {
"nested": {
"path": "children.children",
"query": {
"bool": {
"must": [
{
"match": {
"children.children.name": "Iron"
}
}
]
}
},
"inner_hits": {
"_source": {
"excludes":["name"]
}
}
}
}
}
}
}

[talevy]

Finally got around to fixing this... and realized someone beat me to it 3 days ago 💥!

Looks like this commit fixed it: https://github.com/elastic/elasticsearch/commit/70e14d5e3fb78a74a89e812fc019882e781c61e4
So now the behavior is consistent in how the top-level and the inner-hits _source fields are parsed.

Tested against 5.x with the shorthand string, array, and includes/excludes.

test CONSOLE script:

PUT company
{
  "mappings": {
    "branch": {},
    "employee": {
      "_parent": {
        "type": "branch" 
      }
    }
  }
}

POST /company/_bulk
{ "index": { "_type": "branch", "_id": "london" }}
{ "name": "London Westminster", "city": "London", "country": "UK" }
{ "index": { "_type": "employee", "_id": "london", "_parent": "london" }}
{ "name":  "Alice Smith", "dob":   "1970-10-24", "hobby": "hiking"}

GET company/_search
{
  "_source": "name",
  "query": {
    "has_parent": {
      "parent_type": "branch",
      "inner_hits": { "_source" : "city"},
      "query": {
        "match": {
          "country": "UK"
        }
      }
    }
  }
}