Graylog2-server: Regress in queries in Graylog 3.0

[Nklya]

I think there might be regress in queries in Graylog 3.0.
I updated my Graylog setup from Graylog 2.5.1/Elasticsearch 5.6 to Graylog 3.0/Elasticsearch 6.6 and discovered a problem with queries.

Expected Behavior

According to help, I can use parentheses in queries to find any of word
http://docs.graylog.org/en/3.0/pages/queries.html#syntax

For example NOT user:(gitlab-runner jenkins git). This worked as expected on Graylog 2.5.1 and doesn't on 3.0.
For now I had to change query to NOT (user:gitlab-runner OR user:git OR user:jenkins)

Current Behavior

Query NOT user:(gitlab-runner jenkins git) return all items.

Possible Solution

Change query to NOT (user:gitlab-runner OR user:git OR user:jenkins)

Steps to Reproduce (for bugs)

1. Run query with any_field: (value1 value2)
2. It doesn't work in ver 3.0
3. Change query to any_field:value1 OR any_field:value2
   4.It works

Context

I used these queries to filter some events from alerts. I had to change them to more complicated like NOT (user:gitlab-runner OR user:git OR user:jenkins)

Your Environment

• Graylog Version: 3.0.0-12
• Elasticsearch Version: 6.6.0
• MongoDB Version: 4.0.2
• Operating System: Ubuntu 18.04.1 LTS
• Browser version: Google Chrome 72.0.3626.109

P.S. I haven't found related issues

[edmundoa]

I could reproduce the issue in Elasticsearch 6.5.1, but not on 5.6.13. I guess there was some change in behaviour we weren't aware of.

[edmundoa]

We managed to find out why this behaviour changed:

The split_on_whitespace parameter for the query_string query has been removed. If provided, it will be ignored and issue a deprecation warning. The query_string query now splits on operator only.

Extract from the ES 6.0 breaking changes: https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.0.html

This was confirmed in https://github.com/elastic/elasticsearch/issues/31297#issuecomment-401238569, which also links to a new way of getting the same result that got introduced after the change. Unfortunately it needs to be set in index templates, so we would need to update Graylog's templates to be able to workaround the problem.

To summarize:

• @Nklya for now the easiest way to fix the query is to add OR operators around the different terms, e.g. NOT user:(gitlab-runner OR jenkins OR git).
• We need to update our documentation to point out this change of behaviour.
• We should consider modifying our ES templates to include the new split_queries_on_whitespace option, or try to find another way to make old queries keep their old behaviour.

[Nklya]

Thanks!
I already use OR to workaround this.

[dennisoelkers]

@edmundoa: Are there any follow up tasks for this issue or can it be closed?

[edmundoa]

@dennisoelkers as far as I know only the last point remains:

We should consider modifying our ES templates to include the new split_queries_on_whitespace option, or try to find another way to make old queries keep their old behaviour.

[dennisoelkers]

Should we create a new issue for that or modify the scope of this one?