Elasticsearch: Unable to use stored scripts in _update_by_query

[luizgpsantos]

Elasticsearch version (bin/elasticsearch --version):
Version: 6.1.1, Build: bd92e7f/2017-12-17T20:23:25.338Z, JVM: 1.8.0_151

Plugins installed:
["x-pack"]

JVM version (java -version):
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)

OS version (uname -a if on a Unix-like system):
Darwin MacPro.local 17.3.0 Darwin Kernel Version 17.3.0: Thu Nov 9 18:09:22 PST 2017; root:xnu-4570.31.3~1/RELEASE_X86_64 x86_64

Description of the problem including expected versus actual behavior:

I'm unable to use stored scripts to update by query. Elasticsearch is returning "lang cannot be specified for stored scripts error.

Steps to reproduce:

1. Create a stored script

POST _scripts/dummy_script
{
  "script": {
    "source": "ctx._source.my_field = 8",
    "lang": "painless"
  }
}

Please note that "lang" parameter is obligatory here.

1. Index a test document:

PUT test/foo/1
{
  "my_field": 10,
  "user_id": 12
}

1. Try to update by query:

POST test/_update_by_query
{
  "query": {
    "term": {
      "user_id": 12
    }
  },
  "script": {
    "id": "dummy_script"
  }
}

Response:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "lang cannot be specified for stored scripts"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "lang cannot be specified for stored scripts"
  },
  "status": 400
}

Provide logs (if relevant): No logs are emitted.

[danielmitterdorfer]

Thanks for the report @luizgpsantos! I could reproduce this issue with the provided reproduction scenario on a fresh install of Elasticsearch 6.1.1.

This is probably a side-effect of #25610. @jdconrad can you please have a look?

[jdconrad]

Will take a look at this next week.

[jughead]

The current workaround that I use is to pass:

"script": {
  "id": "dummy_script",
  "lang": null,
}

[marczerbo]

Hello, I'm trying scripting & _update_by_query on my data but, with "lang": null on elastic cloud (6.1.1), I still get the same error ("lang cannot be specified for stored scripts").

[marczerbo]

Hello, just to tell that the workaround of jughead is working (probably was my fault on last tentative, sorry) and that I succeeded in using it in elasticsearch-py passing python "lang": None instead of keyword ‘null’ (see below for whom is interested).
Bye.

es.put_script (id="dummy_script", body={ "script": { "lang": "painless", "source": "ctx._source.xxx = params.param1; …" } } )

es.update_by_query (index=dummy_index, body={"script": {"id": "dummy_script", "lang": None, "params": { "param1": value_yyy, …}}, "query": {"term": {… } } } )

[TheFireCookie]

Hello,
I'm encountering the same bug using NEST and I cannot find a workaround since I don't handle the serialization myself.
It's blocking the upgrade of my product to ElasticSearch 6.X
Matthias.

[rjernst]

I looked at this with @jdconrad. The bug here is in RestUpdateByQueryAction.parseScript. It defaults lang to the default lang, but for stored scripts this should be null. This code should really use Script.parse.

/cc @nik9000

[tom10271]

I am using ES 6.2.3,

{
    "script": {
        "id": "merge-asset-profiles",
        "lang": null,
        "params": {
            "updated_at": 1527048588
        }
    }
}

is not working. I got

{
    "error": {
        "reason": "[script] lang doesn't support values of type: VALUE_NULL",
        "root_cause": [
            {
                "reason": "[script] lang doesn't support values of type: VALUE_NULL",
                "type": "parsing_exception"
            }
        ],
        "type": "parsing_exception"
    },
    "status": 400
}