Elasticsearch: "didn't store _source" when reindexing into ES 5.2

[PhaedrusTheGreek]

• Occurs when reindexing from ES 1.7.5 -> 5.2
• Does not occur with ES 1.7.5 -> 5.1.1

Steps to reproduce:

1. Index data into 1.7.5:

POST /i/d?pretty
{
    "test": "doc"
}

1. Spin up 5.2.0 and run reindex:

reindex.remote.whitelist: ["127.0.0.1:9201"]

POST /_reindex?wait_for_completion&pretty
{
   "conflicts": "proceed",
   "source": {
      "remote": {
         "host": "http://127.0.0.1:9201"
      },
      "index": "i",
      "type": [
         "d"
      ]
   },
   "dest": {
      "index": "i2"
   }
}

Results in:

{
   "error": {
      "root_cause": [
         {
            "type": "illegal_argument_exception",
            "reason": "[i][d][AVn1_zFoaRGPrSCAf9Gw] didn't store _source"
         }
      ],
      "type": "illegal_argument_exception",
      "reason": "[i][d][AVn1_zFoaRGPrSCAf9Gw] didn't store _source"
   },
   "status": 400
}

[jimczi]

It's a side effect of a bug fix #22507, reindex does not explicitly request _source in the remote index (automatically add _source:true in the source like in 5.1). Though for cluster older than 2.0.0 we automatically add metadata stored fields (ttl, parent, ...) to retrieve. The combo makes a request that filters _source from the response.
The workaround is to add "_source":true in source ;) and it works fine but it should not be a requirement. Although we'll have to document this behavior for the 5.2.0 timeframe.
@nik9000 any thoughts ?

[nik9000]

@nik9000 any thoughts ?

I'm happy to put together a fix sometime in the next few days. I'll have to dig up my old versions of elasticsearch and have a look.

[sly7-7]

I try to reindex from 1.7.4 to 6.1, and I encounter this error. The documents are stored without _source on purpose. I just wonder if the reindex should work, or if it's impossible to do such a reindex.

[dadoonet]

No if you don’t have source you can’t reindex (which reads source).