Kibana: Enlarge the search input field to support security use cases

[randomuserid]

Describe the feature:

In recent builds, like the 8.x demo instance, the width of the search input field has been approximately halved from what it used to be. See comparison in the attached screenshots.

Describe a specific use case for the feature:

Long and / or compound searches used by security analytics and threat hunting use cases need a wider / larger input field.

In some cases, a two-line input field (or the ability to grow and shrink the size of the field) would be nice to have, for very long or complex searches.

[elasticmachine]

Pinging @elastic/kibana-app

[lukeelmers]

cc @lizozom since you may have looked in this area of code recently

@elastic/kibana-design In case y'all have already thought of this. IIRC there was a brief period of time where timepicker was stacked above the query bar, but it took up a ton of vertical space so they were merged into the same line. Perhaps there's a potential middle ground here?

[randomuserid]

So with the detection engine shipping, the size of the search box is now becoming a blocker to expanding search based rule production due to these reasons;

1. Users cannot see the search logic they are editing, while they are editing it, either in the Siem, or when working on a search in Discover. This has usability impact and introduces the potential for accidents - searches are inevitably going to be saved with unnoticed errors which may result in false negatives.
2. We are reluctant to ship lengthy searches for reasons above; they will be difficult for a user to edit and tune. Rule authors are also impacted and are using a complex workflow where rules are composed in text files, converted to json, and then imported.
3. These conditions effectively block us from shipping searches with complex logic. Complex logic is needed to work across disparate event pipelines; to improve signal to noise ratios; and to highlight our ability to ship more sophisticated detection logic.

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[snide]

EUI has docs and a working example of a short term, quick solution to this problem. Long term I think we need to use a real code editor in this area, with a different autocomplete solution (closer to the Canvas expression editor). We're happy to pair with any engineers that want to work on that project. Just let us know.

[randomuserid]

Let's discuss. We tend to use filters less and search strings more, esp. when there are complex nested booleans. This is more intuitive for SIEM users from other platforms and has some other usability advantages when editing rule files under current workflows. Also for interoperability with projects like Sigma, OpenSOC, others.

[dcode]

I'm not sure the search to filter functionality fits this particular use-case. I mean, if you look at some of the rules that we're shipping today, , they are completely unreadable in a search box.

event.action:"Network connection detected (rule: NetworkConnect)" and process.name:(Microsoft.Workflow.Compiler.exe or bginfo.exe or cdb.exe or cmstp.exe or csi.exe or dnx.exe or fsi.exe or ieexec.exe or iexpress.exe or odbcconf.exe or rcsi.exe or xwizard.exe)

(source link)

It would add a lot of value if users (analysts) could wrap their text and indent to make it easier to see grouping:

network.transport:tcp and destination.port:(6667 or 6697) and
   source.ip:(10.0.0.0/8 or 172.16.0.0/12 or 192.168.0.0/16) and
   not destination.ip:(10.0.0.0/8 or 127.0.0.0/8 or 172.16.0.0/12 or 192.168.0.0/16 or "::1")

(source link)

I understand the desire to have a consistent layout and responsive UI, but today, it's not a _useable_ UI for anyone trying to add specificity to their queries. Honestly, I think the ideal approach would to have an expandable text area, such as the text box that I'm typing this comment in. There's another example (here) that looks like a regular text input, but allows the user to resize it to fit their text. This one auto-resizes, which may be unnecessary, but I could at least have the option to see my entire query at once.

The disappearing box in the EUI docs linked above don't really address the long queries.

EDIT: Added examples in-line for clarity.

[randomuserid]

Great example, and worth noting that while this is one of our longer rules atm, it needs to become even longer in order to have enough branching logic to consider network events from different pipelines which have different characteristics. Even if we did not have new lines, and the search text was one run-on string without line breaks, the ability for the user to be able to see the entire search would be an improvement. At present users have to copy / paste between a text editor and the search field.

[snide]

I'll restate what I mentioned before

Long term I think we need to use a real code editor in this area, with a different autocomplete solution (closer to the Canvas expression editor). We're happy to pair with any engineers that want to work on that project. Just let us know.

Does SIEM / Security want to work on this? We're happy to pair.

[spong]

Happy to get the ball rolling on this @snide -- I'll set something up for next week 🙂

[lizozom]

@spong are you planning to do this work for SIEM?
@snide do you think this is useful in other apps?
Because if so, I think it could be nice to have it integrated into data/ui/search_bar.
What do you think?

[snide]

Yes, it needs to be everywhere. Preferably we should build a single search bar, that can expand to larger queries and handle the type of AND / OR grouping that SIEM requires (since right now they are using two search bars for their queries). Whichever team wants to start on this just set up a kickoff with me and I'll get a designer assigned.

[lizozom]

I'll bring it up on the @elastic/kibana-app-arch dev sync so we can cooperate on this.
@alexh97

[spong]

The Band got together and we have some next steps for progressing on this issue! 🎉 🎶 We'll be pursuing both a short term (7.9) and long term solution (7.x/8.x?).

For the near-term, @snide & co are going to run a design cycle on what it would look like to swap the search input field to a text area, and SIEM will budget ~1wk LoE for implementing this change in 7.9 for the Rule Creation / Edit Rule query bars within SIEM Detections (perhaps exposing to the whole app depending on implementation). Will sync once the design cycle is complete.

For the long-term, we had chatted about what a full-featured editor would look like for the search input field, including auto-complete, syntax highlighting, the grouping of AND's/OR's, and support for additional languages like EQL. @lizozom will be starting the conversation on the Platform side in the next Kibana App Arch sync, and following up from the design/dev work in the above near-term solution we'll create a document for building out what a full-featured solution would look like when implemented across all areas of Kibana.

Thanks again everyone! Will continue to post updates to this issue once we loop back around with @snide & design! 🙂

[snide]

Did a quick exploration just to point out this should be possible. We'll work on the components and pattern work necessary on the EUI side. Issue to track that part of it here https://github.com/elastic/eui/issues/3434

[MichaelMarcialis]

@snide (et al): FYI, I’m planning to start kicking around some design ideas for this issue next week (5/11). Will keep this group updated as I make progress.

[elasticmachine]

Pinging @elastic/kibana-core-ui (Team:Core UI)

[MichaelMarcialis]

Per feedback from the SIEM and EUI folks, we've updated the short-term solution mockups and prototype to account for the following:

• On focus events, if the query bar has an adjacent sibling datepicker (i.e. Discover), that datepicker will be hidden to allow the horizontal space for the query bar to grow to fill that now-empty space.

• When the textarea is focused, watch for an enter key keydown event to submit the query, rather than create a line break. As this is more important than supporting line breaks in the query textarea, line breaks will not be supported in the short-term.

This appears to be agreeable to all concerned parties for 7.9 needs. Updated mockups and prototypes below.

• Figma Mockup
• Figma Prototype