Elasticsearch: MatchNoDocsQuery in simple_query_string query with multiple fields and AND operator returning no results

[bartdegoede]

Elasticsearch version Version: 6.4.0, Build: default/deb/595516e/2018-08-17T23:18:47.308994Z

Plugins installed: [analysis-icu, analysis-smartcn, analysis-stempel, repository-s3]

JVM version: 1.8.0_131

OS version: Linux 4.4.0-83-generic #106-Ubuntu SMP Mon Jun 26 17:54:43 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux (Vagrant box)

Description of the problem including expected versus actual behavior:
Related to the issue described in https://github.com/elastic/elasticsearch/issues/28856, when I execute a simple_query_string with the following conditions:

• the query string is analyzed with an analyzer that has a stopword filter
• the query contains one of the stopwords on the list
• the query has multiple fields configured
• the default_operator is AND

The token is replaced by a MatchNoDocsQuery instead of omitted. The combination with the AND operator ensures that no result will match, regardless of what is queried.

The expected behavior (at least, the behavior in Elasticsearch 6.3.0 after this merge https://github.com/elastic/elasticsearch/pull/28871) is for the stop word to be removed from the token stream, not to be replaced by MatchNoDocsQuery.

Steps to reproduce:

Please include a minimal but complete recreation of the problem, including
(e.g.) index creation, mappings, settings, query etc. The easier you make for
us to reproduce it, the more likely that somebody will take the time to look at it.

1. Run Elasticsearch 6.4.0 (bin/elasticsearch)
2. Create an index with multiple fields:

curl -XPUT "localhost:9200/stopword-query-string-test" -H "Content-Type: application/json" -d '{
    "mappings": {
        "doc": {
            "properties": {
                "title": { 
                    "type": "text",
                    "analyzer": "english"
                },
                "body": {
                    "type": "text",
                    "analyzer": "english"
                }
            }
        }
    }
}'

1. Validate the query:

curl -XGET "localhost:9200/stopword-query-string-test/doc/_validate/query?explain" -H "Content-Type: application/json" -d '{
  "query": {
    "simple_query_string": {
      "query": "the quick brown fox",
      "fields": ["title", "body"],
      "analyzer": "english",
      "default_operator": "AND"
    }
  }
}'

The response will be:

{
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "valid": true,
  "explanations": [
    {
      "index": "stopword-query-string-test",
      "valid": true,
      "explanation": """+(+MatchNoDocsQuery("[multi_match] list of group queries was empty") +(title:quick | body:quick)~1.0 +(title:brown | body:brown)~1.0 +(title:fox | body:fox)~1.0) #*:*"""
    }
  ]
}

1. To verify, I would expect the query to match the following document:
   Document:

curl -XPUT "localhost:9200/stopword-query-string-test/doc/1" -H "Content-Type: application/json" -d '{
    "title": "the quick brown fox jumps over the lazy dog",
    "body": "the quick brown fox jumps over the lazy dog"
}'

Query:

curl -XGET "localhost:9200/stopword-query-string-test/_search?pretty" -H "Content-Type: application/json" -d '{
  "query": {
    "simple_query_string": {
      "query": "the quick brown fox",
      "fields": ["title", "body"],
      "analyzer": "english",
      "default_operator": "AND"
    }
  }
}'

Response:

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 0,
    "max_score": null,
    "hits": []
  }
}

I tested with querying just one field too, and there behaviour is as expected (the stopword is simply omitted):

GET stopword-query-string-test/doc/_validate/query?explain
{
  "query": {
    "simple_query_string": {
      "query": "the quick brown fox",
      "fields": ["title"],
      "analyzer": "english",
      "default_operator": "AND"
    }
  }
}

Response

{
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "valid": true,
  "explanations": [
    {
      "index": "stopword-query-string-test",
      "valid": true,
      "explanation": "+(+title:quick +title:brown +title:fox) #*:*"
    }
  ]
}

Provide logs (if relevant):
No relevant logs

[jimczi]

There is already a fix for this bug that will be available in 6.4.1 so I am going to close this issue. Thanks for reporting @bartdegoede .