Kibana: Kibana CRUD forms enhancement

[LeeDr]

Describe the feature:

I think it would be nice to have a feature (maybe a plugin) in Kibana where I can create and use simple forms to create docs of some ad-hoc type. These forms could be used from with Kibana or embedded in some other app just like Saved Searches, Visualizations, and Dashboards.

I think of this as "Manual Ingest". It would be the easiest way for a new user to get data into Elasticsearch (easier than installing/configuring Logstash or Beats).
But it could also be the basis of significant workflows in a business.

• It could be used for a Help Desk application where users could easily search all the history of cases.
• It could be used for a sign-up or user registration.
• Add and edit content for news stories by category, location, writer, etc, that are then used by the web site to show the newest stories.
• Add new products for sale on an online shopping site.

These are all use cases that need a user to enter data and get it into Elasticsearch (not something Logstash or Beats are appropriate for). Right now, a user would have to spend real development effort to build an app to enter this data.

Here's a mock-up of the view in Kibana where a user would click Forms app, select an existing form from the list, and start adding a new doc. Obviously if you have X-Pack Security, not everyone would have privileges to create a new index, or to delete a form.

On the right side we have 2 options.

• The first option wizard walks you through creating a new form and a new index for storage.
• The second option wizard for existing index would show you a list of your indexes (or index patterns?) that you would select from. You could potentially create multiple forms for the same index. For example, a "For Office Use Only" version of a form that has a few fields which are hidden or non-editable in another form.
  

I could have an index pattern for my docs, filter in Discover like we do now, but I would also want to be able to click on a doc and open a form with edit fields, delete button, a button to add a new doc. After all, I'm going to make typos in my news story and need to be able to correct them.

Have a way to enable/disable editing per field (probably not required for the first release);

[pemontto]

:+1:

[chenryn]

👍

[lmangani]

:+1:

[LeeDr]

I had some thoughts and hacked together a rough prototype of the form to create the new index.
Radio buttons let you select if you want a time-based index, and if so, where the time comes from.

Give it an index name (it should probably check at this point that it doesn't already exist).
Or maybe if you give the name of an existing index it populates this page with all the data types and lets you assign form widgets for any of those you want to allow editing of. So this field should come before the "Time-based Index field" options.

Name the first field, and select a data type. I didn't take the time to figure out everything that should be in this list.
If it's a String, I make indented radio buttons for Keyword or Text, but now I realize they should be checkboxes because you could have both.
If it's a number, maybe you pick a plain text field for manual entry, or maybe there's some other options.
If it's a Date, offer a Date/Time picker from Kibana.

/cc FYI @lukasolson

[mjeewantha]

+1

[kosalads]

+1

[LeeDr]

Rearranged content...

[LeeDr]

Scratch my previous lame form editor mock-up. I realize now that the form editor should use widgets similar to the visualization editor which allows drag-drop, enable, delete of metrics or buckets (in this case form fields).

• In this screenshot, image each "Metric" block is a form field.
• Instead of "Aggregation" it would be "Field type" with choices like "Text Field", "Date Picker", "Password", "Password with second entry", "Text Area", "Check boxes", "Radio Buttons".
• For some of these choices additional options would appear. For example, for "Radio Buttons" it would let you add text labels like "Home","Mobile","Work" for a phone number field.
• "Custom Label" would just be "Label"
• If you added address fields but forgot a field for apartment number you could just add it at the end and drag it up into the position you want.

[mr-mos]

+1

[xycloud]

awesome, +1

[fjiang212]

+1

[jantoniuk]

+1

[Martin-Andersen]

Any progress? I would love to see this implemented. Then I could stop writing custom web apps

[Fredoid]

+1000
This feature would be just awesome.
Kibana is a powerfull tool to visualize but it's very frustrating to not be able to update data directly in the user interface.

[rjof]

+1

[chenyun]

+1

[sunnytambi]

+1

[laurencegill]

+1

[LeeDr]

This is coming up on 3 years old and I think I'm going to start working on it soon since I have a real need for it now!

[LeeDr]

I'm looking at the simplest first phase right now. In this phase you would only have a choice to make existing fields in an existing index pattern editable. And the editing would be in-place in Discover (no capability to add new fields, or an editing form, or to define a new index pattern). In this simple case, we assume there is either no security or other Elasticsearch settings preventing us from updating the doc in Elasticsearch.

• [ ] Add a checkbox to the reactFieldEditor to appear here (but above the [Save] button, not below it. (I hacked this one into src\legacy\core_plugins\kibana\public\management\sections\index_patterns\edit_index_pattern\create_edit_field\create_edit_field.html but I think it needs to go into the reactFieldEditor)
  

• [ ] When checked, add a value to the index-pattern for that field. Currently a field looks something like this;

{
    "name": "customer_first_name",
    "type": "string",
    "esTypes": ["text"],
    "count": 1,
    "scripted": false,
    "searchable": true,
    "aggregatable": false,
    "readFromDocValues": false
}

Initially this could be

        "editable": true

But to support different types of input (text, checkbox, selection list, datepicker, etc,) it could describe the input type and if the "edit" field doesn't exist then it's not editable;

         "edit": "datepicker"

• [ ] In Discover, when rendering the column data of selected fields, if the index pattern indicates it's editable, then show the input with the current value in that input type. I think that conditional test might go in src\legacy\core_plugins\kibana\common\field_formats\types\source.js
  But I've temporarily hacked src\legacy\core_plugins\kibana\public\discover\doc_table\components\table_row\truncate_by_height.html to try to show a text input (but it shows it wrong and for every element);
  

• [ ] Do we want one save icon on one end of the row for the case where there are editable field(s)? Make this actually update the doc with the new value for this field. I don't know if one of these 2 ways is better than the other?
  Update the field(s) that are changed this way;

POST kibana_sample_data_ecommerce/_update/gzceV2sBN4PF12sMfKVN
{
  "doc": {"customer_last_name": "Drengenberg"}
}

Or update with query like this;

POST kibana_sample_data_ecommerce/_update/_DceV2sBN4PF12sMdp6K
{
  "script" : "ctx._source.customer_last_name += \"Lee\""
}

[sunnytambi]

This is coming up on 3 years old and I think I'm going to start working on it soon since I have a real need for it now!

Thats going to be killer @LeeDr .. I had the same need few months ago and had to drop it because Kibana doesn't have a native edit flow.

[Oliverax]

If you need any more motivation: We use the ELK stack for logging data. Our simple but powerful use case would be to annotate error-entries in the discover/search mode in an additional field along the lines of: "Who is looking into this", or "is it already captured in our issue tracker"? Maybe simply by altering a LogLevel field from ERROR to "NOTIFIED" or "HANDLED" would be sufficient, too.

[nimocks]

+1

[3dog]

+1

[tonypottakkal]

+1

[dsigel]

+1

[surjray]

+1

[nimocks]

I've started this a few weeks ago. No clue when I will get back to it but can see screen shots of how I was imagining it working.

https://github.com/aaron-nimocks/kibana-index-crud