Elasticsearch: Context Suggester category filtering is not cumulative

[kabcampbell]

Elasticsearch version: 5.0.0

Plugins installed: None

JVM version: 1.8.0_101

OS version: Ubuntu 16.04

Description of the problem including expected versus actual behavior:
I have added multiple context mappings to a completion suggester field. In version 2.x and earlier, when filtering on those contexts, suggestions were only returned if they matched all of the contexts. But that does not seem to be the case with 5.0.0.

Steps to reproduce:

1. Mapping for my suggest field:

"suggest": {
  "type": "completion",
  "contexts": [
      {
        "name": "companyContext",
        "type": "category"
     },
     {
        "name": "usersContext", 
        "type": "category"
     }                     
  ]
}

1. Index a document with suggest field contexts populated:

"suggest": {
    "input": "test team site",
    "contexts": {
        "companyContext": "vendor",
        "usersContext": ["chad", "fred"]            
    }        
}

1. Run a query supplying only one matching context:

"suggest": {
   "suggestions": {
        "text": "tes",
        "completion": {
            "field": "suggest",
            "contexts":{
                "companyContext": "vendor",
                "usersContext": "bill"
            }
        }
    }
}

I would expect that this would not return the indexed document, because the usersContext is not a match. And that was the behavior in 2.x. But in 5.0.0, the document that matches only the companyContext is returned.

[clintongormley]

@areek what are your thoughts?

[areek]

when multiple context types are defined they are ORed instead of ANDed in 5.0, they are meant to add boosts rather than be cumulative.
We can do a post filtering to return suggestions that match all the specified contexts. I will try to get this in in the next minor release.

[kabcampbell]

Thanks @areek.

[kabcampbell]

Hi @areek - Did this make it into 5.1.1? I just noticed that my previous comment seems to somehow have unassigned this task, so I'm hoping it didn't drop off your radar.

[khalidabuhakmeh]

Hello @areek I am running into the same issue as @kabcampbell where I have similar documents within the same index, but only want to suggest from a subset of those documents.

I am finding that the contexts are resulting in OR logic which is returning irrelevant suggestions.

[Robooto]

Hi, I recently updated from 2.x to 5.0 and am experiencing this same issue where filtering by multiple contexts is resulting in OR filtering instead of AND.

[jacktuck]

Hi @areek Any updates? Had this made it into a minor release yet?

Thanks

[jacktuck]

Forgot about this issue and made #23568

@kabcampbell Did you come up with a workaround, i'm actually in a position where i have to migrate to v5 so looking for a solution?

[wjonesy]

I am migrating from 2.x to 5 at the moment. And I have just come across this issue, I have been stumped for a while.

The docs are not very clear.
"These suggestions will be associated with cafe and food category."
Sounds like the context types are AND, not OR.

An update would be useful.

Thanks

[cara-dev]

I'm also migrating from 2.x to 5.x and have come across this issue. Any updates?

[antoineberthelin]

Hello, do you have any news about this issue ? Tks

[khalidabuhakmeh]

[gnudle]

Hey, here is some progress
https://github.com/elastic/elasticsearch/pull/24565

[jacksontbryan]

This just nailed me as well. There is a chain of issues all closed with a recommended work around at the very bottom of the thread in #26407. It appears that it isn't going to be "fixed" to have an option to AND contexts.

The recommended work around is to create a combined context. This will not work for me because I am doing a geography and category combined and its ORing it right now which was very unclear from the documentation.

[michael-wirth]

+1 This feature would be very helpful

[adamhadani]

+1

[blackchestnut]

+1

[jimczi]

cc @elastic/es-search-aggs

[jimczi]

Closing as duplicates of https://github.com/elastic/elasticsearch/issues/24421#issuecomment-375689139
The workaround is to create the combinations of the contexts at index time like described here:
https://github.com/elastic/elasticsearch/pull/26407#issuecomment-326771608

Any other solution would considerably slow down queries which is the reason why I am closing this issue as won't fix. For more flexibility it is preferable to use a normal field and to apply edge ngram on it.

[johnfrades]

Thanks @jimczi for the workaround, this is what im looking for.