Kibana: KQL: When using terms with dashes, I am warned about using Lucene syntax.

[jordansissel]

Kibana version: 7.2.0

Elasticsearch version: 7.2.0

Server OS version: Google Container OS / GKE 1.12.8-gke.10

Browser version: Firefox 68.0.1

Browser OS version: WIndows 10

Original install method (e.g. download page, yum, from source, etc.): Docker

Describe the bug:

Kibana mistakenly warns me that I am using Lucene syntax when I'm trying to use KQL.

Steps to reproduce:

1. Use KQL
2. Type test-123hello in the query bar
3. Click "update"

Expected behavior:

Don't warn? I'm trying to use KQL! ;P

Is it possible that Kibana doesn't treat - (dash) as part of a term?

This also happens when I'm searching for a specific field with hostname values that have dashes, such as: beat.hostname:auditbeat-1234

Screenshots (if relevant):

[elasticmachine]

Pinging @elastic/kibana-app

[timroes]

cc @Bargs @TinaHeiligers

[TinaHeiligers]

@jordansissel If you're using KQL without using the drop down menu and want to omit the query field, please add your query term as a string (in quotation marks). Please try "test-123hello" rather than test-123hello and beat.hostname : "auditbeat-1234" rather than beat.hostname:auditbeat-1234.

If the query term is not provided as a string, we will get a syntax error:
_Incorrect syntax (unquoted):_

As a string, the query will be issued:
_Correct syntax (quoted):_

Please also note that when a field is not added, the input term will be matched against the default field in your index settings. If there isn't a default field, terms are matched against all fields. With KQL, the term needs to be provided as a string. For more information, please see https://www.elastic.co/guide/en/kibana/current/kuery-query.html.

KQL with autocomplete is there to help you with selecting valid fields and values in your index pattern. I highly encourage you to take advantage of selecting items from these drop downs:

Please let us know if there are any issues otherwise I'm going to close this issue.

[Bargs]

There are definitely some edge cases where valid KQL queries will trigger our lucene detection. We could look into the possibility of fixing this particular edge case, but in general when a user runs into false positives like this we assume they know they're using KQL at that point and will hit the "Don't show again" button. This warning is really just to prevent the total beginner from using lucene on accident in the KQL query bar.

[jordansissel]

@TinaHeiligers @Bargs thank you both for your excellent answers <3

[timroes]

I am wondering, if we should fix those specific edge cases, or maybe consider, since we made KQL the default in 7.0, that we could remove the warning completely in 8.0, assuming KQL is our default and we don't treat Lucene queries with a high enough priority anymore to trigger false positive errors for KQL users.

[montanaflynn]

I was wondering why it kept saying this!

Query: user-id:10

It looks like you may be trying to use Lucene query syntax, although you have Kibana Query Language (KQL) selected. Please review the KQL docs here.

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[iankoetter]

Recently came across this bug in Kibana 7.8.1

[nyetwurk]

Even worse, you can't mix hyphens and wildcards it seems, e.g. none of these work:
hostname:*-stuff-*
hostname:"*-stuff-*"
hostname:*"-stuff-"*

[rw-access]

@nyetwurk if you put * inside your quotes, it is interpreted as a literal asterisk * and not a wildcard

[nyetwurk]

@nyetwurk if you put * inside your quotes, it is interpreted as a literal asterisk * and not a wildcard

hostname:*"-stuff-"* and hostname:*"-"stuff"-"* should work then. They do not.

[rw-access]

Those are surrounded by quotes, so * is not interpreted as a wildcard.

[nyetwurk]

Those are surrounded by quotes, so * is not interpreted as a wildcard.

No, they are not. The *'s are not inside of the quotes.

[rw-access]

Sorry I think I chose words poorly. By "surrounded by quotes," I meant to say "in a quoted string.". Like "this*example", not like this*example. I'm sorry for the confusion.

And for this example: hostname:*"-stuff-"* looks like invalid syntax, but it does look like a clever idea.

[nyetwurk]

Considering things like foo-bar-baz are a common pattern (e.g in index patterns), having a way to do *-bar-* seems like a pretty obvious requirement.