Elasticsearch-net: Term Filter with empty value compiles to nothing

[SLavrynenko]

NEST/Elasticsearch.Net version:
2.0.5

Elasticsearch version:
"version" : {
"number" : "2.1.1",
"build_hash" : "40e2c53a6b6c2972b3d13846e450e66f4375bd71",
"build_timestamp" : "2015-12-15T13:05:55Z",
"build_snapshot" : false,
"lucene_version" : "5.3.1"
},

Description of the problem including expected versus actual behavior:
I've the following code for filter:
filter &= !f.Verbatim().Term(p => p.MobileNumber, "") && f.Exists(m => m.Field(p => p.MobileNumber));

but in the result search query I see just:
{"exists":{"field":"MobileNumber"}}
without expected part with:
{"bool":{"must_not":[{"term":{"MobileNumber":{"value":''}}}]}}

Is the way I'm using Verbatim() correct? Don't understand why it doesn't work.

[gmarz]

Hey @SLavrynenko this was fixed in https://github.com/elastic/elasticsearch-net/pull/2021. You're on an older version that doesn't contain the fix. I recommend upgrading directly to 2.3.1, which also fixes a pretty bad memory leak issue (https://github.com/elastic/elasticsearch-net/issues/2048).

After upgrading, you'll also have to change your code slightly to make this work by calling Verbatim() inside the term query rather than at the outer query level. So it'll become:

filter &= !f.Term(t => t.Verbatim().Field(p => p.MobileNumber).Value("")) && f.Exists(m => m.Field(p => p.MobileNumber));

[SLavrynenko]

It's strange but looks like I can't upgrade NEST via Nuget because of the following error:

Severity    Code    Description Project File    Line    Suppression State
Error       Unable to find a version of 'Microsoft.CSharp' that is compatible with 'Elasticsearch.Net 2.3.1 constraint: Microsoft.CSharp (>= 4.0.1-beta-23225)'.            0   

[gmarz]

Make sure you're using the v3 nuget.org feed (see https://github.com/elastic/elasticsearch-net/issues/2079).

[SLavrynenko]

Now everything is working, thanks!

[SLavrynenko]

At my previous post I meant that I've upgraded the NEST. Just tried 2.3.1 version of NEST with your code:

filter &= !f.Term(t => t.Verbatim().Field(p => p.MobileNumber).Value("")) && f.Exists(m => m.Field(p => p.MobileNumber));

and it doesn't work - there is still "exists" part of the query.

[gmarz]

@SLavrynenko mind posting the entire NEST query here a long with the actual and expected JSON?

[SLavrynenko]

I've the following code:

        var response = client.Search<Contact>(
         s => s.Query(
             q =>
             {
                 return q.Bool(
                     b=> b.Must(
                         m => m.MatchAll())
                     .Filter(
                         f => !f.Term(t => t.Verbatim().Field(p => p.MobileNumber).Value("")) && f.Exists(m => m.Field(p => p.MobileNumber))
                     ));
             })
           .Index(IndexName));

I'm expecting to get the following search query:

{
   "query": {
      "bool": {
         "must": [
            {
               "match_all": {}
            }
         ],
         "filter": [
            {
               "bool": {
                  "must": {
                     "exists": {
                        "field": "MobileNumber"
                     }
                  },
                  "must_not": {
                     "term": {
                        "MobileNumber": ""
                     }
                  }
               }
            }
         ]
      }
   }
}

But I get the following:

Request:

{"query":{"bool":{"must":[{"match_all":{}}],"filter":[{"exists":{"field":"MobileNumber"}}]}}}

[gmarz]

Thanks @SLavrynenko

This looks like a bug in how we create the bool query from the overloaded operators. Will work on a fix for the next release.

For now, you can achieve this by building the bool query explicitly. Also, the outer bool query with match_all in the must clause isn't necessary. Try this:

Query(q => q
    .Bool(b => b
        .Filter(f => f
            .Bool(bb => bb
                .Must(m => m
                    .Exists(e => e.Field(p => p.MobileNumber))
                )
                .MustNot(mn => mn
                    .Term(t => t.Verbatim().Field(p => p.MobileNumber).Value(""))
                )
            )
        )
    )
)

[SLavrynenko]

I will try this solution, thanks a lot for the help!