Elasticsearch: percolator: null_pointer_exception when storing query with date range query using "gt" or "gte" operator

[ghost]

Elasticsearch version: 5.1.1

Plugins installed:

[{
  "name" : "x-pack",
  "version" : "5.1.1",
  "description" : "Elasticsearch Expanded Pack Plugin",
  "classname" : "org.elasticsearch.xpack.XPackPlugin"
}]

JVM version: 1.8.0_111

OS version: Ubuntu 16.04 (Linux amd64 4.4.0-57-generic)

Description of the problem including expected versus actual behavior:

Attempting to index a percolate query that uses gte or gt in a range query against a field of type date fails with the following error:

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "failed to parse"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "failed to parse",
    "caused_by": {
      "type": "null_pointer_exception",
      "reason": null
    }
  },
  "status": 400
}

Range queries using gte or gt fail to index; however both lte and lt work properly. Note that the queries listed below are using fixed dates. Thus this note regarding breaking changes in ES 5.0 DOES NOT APPLY:

The percolator no longer accepts percolator queries containing range queries with ranges that are based on current time (using now).

If it helps to know, our setup is a new cluster installed from scratch on ES 5.1.1. There was no index upgrade done from a previous version, nor was any of the data copied. Fresh install, new mapping definitions and data inserts as per the example below.

Steps to reproduce:

1. Create a new index (named searches) with the type mapping for percolate pre-processing (named item, with a date field named created), as well as the type mapping for storing the percolate queries themselves (named search, with the percolate field named query).

PUT searches
{
  "mappings": {
    "item": {
      "properties": {
        "created": {
          "type": "date"
        }
      }
    },
    "search": {
      "properties": {
        "query": {
          "type": "percolator"
        }
      }
    }
  }
}

1. Attempt to index the following queries. The only difference is the range operator used (lt, lte, gt, and gte). The first two (lte and lt) will index, while the other two (gte and gt) throw the null_pointer_exception error.

PUT searches/search/1
{ "query": { "range": { "created": { "lte": "2016-01-01" } } } }

PUT searches/search/2
{ "query": { "range": { "created": { "lt": "2016-01-01" } } } }

PUT searches/search/3
{ "query": { "range": { "created": { "gte": "2016-01-01" } } } }

PUT searches/search/4
{ "query": { "range": { "created": { "gt": "2016-01-01" } } } }

[martijnvg]

@reol-nbessette Thanks for reporting, this is a bug in the now range validation for percolator queries containing range queries.

[ghost]

@martijnvg Now that's a fast debug session and commit. Thank you so much!

Should someone else stumble across this issue before the next patch release, here's a client-side workaround if you're not wanting to patch the server directly. The bug is that when a "from" date (via gt or gte) is specified, a "to" date (via lt or lte) is erroneously expected as well. We can work around by attaching an lte range check with a large value exceeding all your real date values. The year 9999 seems to work even with strict date formats. Thus, as an example:

{
  "query": {
    "range": {
      "created": {
        "gte": "2016-01-01"
      }
    }
  }
}

can be worked around with:

{
  "query": {
    "range": {
      "created": {
        "gte": "2016-01-01",
        "lte": "9999-12-31"
      }
    }
  }
}