The ability to drill down from one dashboard to another view creates a custom workflow for troubleshooting and analyzing data. When speaking with experienced users, this is a fairly common ask. When speaking with new users, the expectation is that the behavior is already available within Kibana.
In the definition of a dashboard panel, we will allow a user to define one or multiple custom drilldown links that can be easily accessed from that dashboard panel. Clicking into this drilldown link will navigate users to the preconfigured view.
cc: @Stacey-Gammon @snide @thomasneirynck @kobelb
It would also be nice to be able to drill down and add a dynamic filter based on a click. This would be for where I have a table/line that shows multiple servers and high level values. I'd like to click on the server and then go to a different dashboard to see details for that specific server. Looking at your mockups and description, it sounds like I would need to first click on the table/line for that server to add it as a filter, wait for the query to load (since the visualization panel will gray out), then click on these new links to drill down.
Also, it kind of fixes the click handler metric visualization but not entirely since that issue is more about getting metric visualization on par with the rest of the visualizations where you can add a filter based on the data in the visualization. These links might get it part of the way but it will still be kind of weird since most visualizations allow clicking on the data to filter while metric doesn't.
Here's another design, but for what a "Dashboard link menu" would look like. Essentially same UI as the panel drilldowns in the above design, but rather than having a visualization, the view mode for this is just a list of exposed links. Figured I'd through it in here since the concepts are so similar and will likely reuse a lot of the same code.
+1
@trevan you are correct, in the proposed approach above you would need to click into a filter and then navigate to the URL. This implementation requires the least amount of change to the current users workflow. However, I think both approaches could be beneficial as having links that are unrelated to a filter may not necessarily need to be accessed via a data point. From a user experience perspective, what would your expectation be? Some thoughts below
Regarding the metric visualization. If you were able to define a link with pre-defined filters, you'd technically be able to provide one or multiple drilldowns with that metric pre-filtered. However, I agree - it's not exactly the same as clicking into the metric to navigate.
I think this is a great idea, especially because most users need to integrate kibana with several other websites/programs that their system is connected to. +1
We need this feature just yet.
Suggest to add a option in the dashboard design, allow a internal URL which can transfer the query string & filter to the next automatically.
@snide - as I was implementing the mocks I came across a situation that might crop up with the way we do the panel flipping for the editing screen.
When the panel is really small, the link editing space will be really small too, and it's a little weird to make the user expand it just to edit links, and then have to resize it small again so the dashboard goes back to the way they want it.
I like the panel flipping idea, but this might be a blocker. Sounds like our other option is just using a modal?
Here are a couple screenshots that show my concern. Obviously the contents of the add/edit links panel will be a lot nicer, but you can get the idea that it will all be very squished.
I just thought of something else. The location of the link icon that shows up on hover is in the same spot as the location of the spy toggle. The spy toggle is internal to the visualization - it's not added by dashboard. So if we want to add a clickable panel icon on top of the actual visualization space, we can't make any assumptions about what it is hovering over (anyone can write anything to be an embeddable and put stuff there - so just moving the spy toggle may not be a sufficient solution).
We could add a footer to the panel that doesn't overlap the vis, but people might be unhappy with the added space it takes up. We could have it pop out on hover, below the visualization. Perhaps if we do that and make it "floatable" it'll work - shouldn't matter if the footer floats over a visualization below it, because the mouse is in the top visualization so the user isn't possibly trying to click something and can't get to it.
@Stacey-Gammon, is it going to be possible for the same visualization to have different "links" depending on the dashboard it is in? If not, why not leave the editing of the links in the visualization editor? It feels weird to edit a visualization specific item inside of the dashboard.
Also, why not put the "link icon" in the next to the "expand panel" icon? Or always show the dropdown in the top left that shows up in edit mode and have 'expand panel' and 'links' be in that menu? You could even move the spy link there as well.
And are these links going to be dynamic per my comments above? Will I be able to click/hover/interact on a row/bar/line/etc and get a link that includes the data that I'm interacting with?
@trevan
is it going to be possible for the same visualization to have different "links" depending on the dashboard it is in?
Yes
If not, why not leave the editing of the links in the visualization editor? It feels weird to edit a visualization specific item inside of the dashboard.
Another reason we went this route is because we wanted it to be a feature that any "embeddable", not just visualizations. For instance, we have visualizations and saved searches. Without making it part of the dashboard panel, you'd have to implement the same logic in a visualization and a saved search.
This topic of a pluggable dashboard embeddable vs a pluggable visualization type is somewhat complicated (and probably off topic for this specific feature). We have gone back and forth on whether everything should be implemented as a visualization plugin, or whether we should allow a more flexible "embeddable" plugin. We went with the later, though jury is still out on whether that's the best route. For now, with all the refactoring our visualize code was going through, it was easier to extract saved search logic out of dashboard code with a new embeddable plugin rather than implement it as a visualization plugin.
Also, why not put the "link icon" in the next to the "expand panel" icon? Or always show the dropdown in the top left that shows up in edit mode and have 'expand panel' and 'links' be in that menu? You could even move the spy link there as well.
I think this is a good option! Spy toggle is part of the embeddable so dashboard doesn't know anything about it. We could consider moving it out but it not sure we want to assume every embeddable will have a spy option.
And are these links going to be dynamic per my comments above? Will I be able to click/hover/interact on a row/bar/line/etc and get a link that includes the data that I'm interacting with?
No. You create the links up front, but after that they are static. Having the links be part of the hovering options on a visualization would be a visualization specific feature. I think this would be a useful next step, but we'd have to reconcile it with how we do filtering now.
So I think for your use case, you probably want visualization specific drill down links that are customizable on the visualization itself, and are specific to the type of visualization shown, so clicking might not filter, but would open up a new object. Perhaps we should have two, separate issues to cover both those use cases.
@Stacey-Gammon, yeah I've been following the external discussions about embedded options. I also re-read the description here and see where it said this should be stored on the dashboard state instead of the visualization state.
So I think for your use case, you probably want visualization specific drill down links that are customizable on the visualization itself, and are specific to the type of visualization shown, so clicking might not filter, but would open up a new object. Perhaps we should have two, separate issues to cover both those use cases.
They don't have to be visualization specific drill down links. If you look at the "related issues" in the description, almost all of them talk about clicking on something in the visualization/search/etc that then takes them to a different dashboard/visualization/search filtered based off of what they just clicked on. These aren't links that you define before hand but links that are dynamic based off of the data currently shown in the visualization/search.
@trevan
They don't have to be visualization specific drill down links.
I don't know if I follow this. What I mean to say is that those types of links would have to be configured during creation of the visualization. For instance your request to make a data table cell clickable to a dynamic link by inserting the value you clicked into a url template - that is something that would be very specific to data tables, and not applicable for a bar chart.
I've thought a bit about whether we could generalize this somehow at the dashboard level by having embeddables send their filtering through dashboard, which could then intercept it and rather than add a filter, it would redirect the user to a link, stuffing the filter into a url template defined at the panel level. I'm not convinced this is the right solution though because it requires more coupling of the embeddable to the dashboard, and it'd still be possible for someone to write an embeddable and apply the filter directly.
@snide
Some more questions regarding the UI/UX flow:
Add link
more than once? Do I get additional input lines, or should I be forced to finish, or delete the one I just started first?Add link
but before I select a saved object, should we give the user a way to cancel and remove the newly added link? Otherwise I think they have to pick an object then remove it via the edit tool? What if there are no objects to pick, how would a user remove the line then?@Stacey-Gammon
They don't have to be visualization specific drill down links.
I don't know if I follow this. What I mean to say is that those types of links would have to be configured during creation of the visualization. For instance your request to make a data table cell clickable to a dynamic link by inserting the value you clicked into a url template - that is something that would be very specific to data tables, and not applicable for a bar chart.
I was just trying to say that if links will be per visualization per dashbard (which is what I gathered from your response), then these "dynamic" links also can be per visualization per dashboard. You would just edit them in the dashboard and then a visualization can have different dynamic links depending on the dashboard it is in. That is what it sounds like is going to happen for the "static" links that you are currently designing.
I've thought a bit about whether we could generalize this somehow at the dashboard level by having embeddables send their filtering through dashboard, which could then intercept it and rather than add a filter, it would redirect the user to a link, stuffing the filter into a url template defined at the panel level. I'm not convinced this is the right solution though because it requires more coupling of the embeddable to the dashboard, and it'd still be possible for someone to write an embeddable and apply the filter directly.
I've actually thought about adding that "intercept" ability so that plugins can hook into it and do exactly what you said.
What would really be nice is that instead of causing a filter when you click on an item (either it is a visualize table row, a search table row, a visualize bar chart, or a different embeddable thing), a context menu appears. This context menu then gives you the option of "filtering" (with both a positive and negative option), go to a link, create watch, create ml job, etc (all of these actions are defined by the embeddable or by plugins). Dashboard creators would then specify what the link templates should look like (with, preferably, easy shortcuts to other dashboards/visualizations/etc) and on click, the embeddable would request the app state and fill in the link.
Good thoughts @trevan, I'd like to explore them more. The static links will probably remain a first step, but it sounds like most use cases will require the dynamic filtering context menu. btw, we do have plans for a "create ml job" link in the panel, so will need to expose more pluggability in the dashboard panel anyhow.
btw @trevan - it should technically be the same amount of clicks to accomplish the same goal. E.g.
Looking at your mockups and description, it sounds like I would need to first click on the table/line for that server to add it as a filter, wait for the query to load (since the visualization panel will gray out), then click on these new links to drill down.
Because the links will be on the panel, not the embeddable, you should be able to click the drill down links before the embeddable finishes rendering. Same with if you built a "Dashboard link menu" - that won't require any network requests so the panel should finish loading nearly instantly and then dashboard level drill down links will be available immediately after a filter is clicked and applied at the dashboard level.
It might be a little bit of a clunkier UI/UX than your suggestion, but it does retain the benefit of one click filtering for people that don't want to use drill down links.
@Stacey-Gammon, that is definitely clunky and isn't very intuitive (how do you tell people to make sure they add a filter before they click on the link?). You also still have the issue of sending a request to ES to rebuild the dashboard that you are just going to throw away. That increases the load on ES for little benefit.
I don't understand what you mean by the "Dashboard link menu".
For REF:
Its called "dynamic drill-downs" in Splunk. We used advanced XML , leveraging tokens within the code of the app. The next piece that is relative and missing here is called "workflow actions" in Splunk - being able to pull token data from the page and use it in a workflow URL - IE: passing to an external REST API..
Workflow actions has its own admin UI...
Cheers!
thanks for the additional info @RANGERBEE! We've taken a step back and are hoping to approach this in a dynamic sense as most users here have requested. I plan on putting together more detailed specs with mocks in the coming weeks. Will update this issue when ready.
no problem... To really complete the workflow and surpass Splunk, making available a customizable secure landing page for each Workflow action genre that allows for some JS would be seriously trick! When we get those REST API response headers they need to be handled, the user does not need to see it, so a landing page with an iframe was our solution to both handle the request/responses and forward to a proper URL then after... ;)
Hi @alexfrancoeur,
Any timeline on this feature? It would be a really nice feature to have.
@Braj-120 Unfortunately we do not have a specific timeframe at the moment but we are hoping to deliver in 6.x. I'll be updating the original description of this issue in the coming days to reflect our most recent requirements.
Also hoping to use functionality like this. Being able to go directly from a dashboard with visualizations to the underlying searches would make troubleshooting so much easier.
REF
I am explaining exact requirement with example .
Let's say we have 2 dashboards
Dashboard 1. Operating System
Dashboard 2. Browsers
In Dashboard 1 we have pie chart showing multiple Operating System like Windows,linux,Android .
In Dashboard 2 we have pie chart showing multiple Browsers like Chrome ,IE , FF etc.
When I click on Windows in Dashboard 1 then it should redirect to Dashboard 2 and should show Browsers having Operating System as Windows .
If you have any doubt let me know.
+1
+1
Hello Team
Any update on the release?
It’s been a big hurdle for us to continue with kibana. As we already have extensive reports may be we need to plan for alternatives.
Any info on the release will really help me to plan for the alternatives for drill down feature ?
There is a lot of stuff discussed in this issue that would be really useful. However I wanted to point out to those waiting for these features, that there are things that can be done in Kibana today to dynamically link dashboards and facilitate a number of drill-down scenarios. The relevant features might not be obvious, and it does involve taking some time to figure out the structure of a Kibana URL, including how to set filters by manipulating the URL directly. While this requires a bit of work, the results can be very useful. Let me share a few examples...
The markdown visualization can be used to navigate between dashboards, which is especially useful for dashboards related to the same use-case.
This is achieved using simple hyperlinks to the dashboards:
TIP: We recommend modifying the Kibana advanced setting filters:pinnedByDefault
, setting it to true
. This will keep filters set on one dashboard, when navigating to another. This is very useful when filtering data to drill-down and then changing dashboards for a different perspective on that same data.
Clicking on the color goes to one dashboard...
And clicking on the number links to another...
Notice how the filters are set via the URL. This is how we make dashboards to launch into certain devices, applications and use-cases, like these...
default
:Url
.Link
{{value}}
{{value}}
inserted where the field value will be.Together this looks like this...
If you use these fields in a visualization such as a table, you will notice the values are now hyperlinks.
Clicking on them will launch the configured URL. In the example above the Talos page is launched for that IP address.
This same method can be used to launch to another dashboard, leveraging the Kibana URL methods showed in the Markdown example...
While these techniques do not handle all of the scenarios that the things discussed in this issue hopefully will. They can be used to significantly improve users' interactions with Kibana.
We use these techniques to allow the users of our Network and Security Analytics solution to use Kibana without ever having to open the full list of dashboards. The navigation and drill-down options we provide allows them to navigate to any perspective they need, and to rarely need to manually set filters when doing so.
Hopefully some of you find these tips helpful as you wait for future enhancements.
+1
+1
+1
This would solve the "way too much data to load on a single set of charts" issue we're currently experiencing. We have dashboards with about ten charts, each loading potentially thousands of segments (lines, bars, etc). Usually the dashboard has one or two simple pie or metric charts at the top for click based filtering. We currently use a hack that saves a bogus filter that returns almost no data for all the charts with the dashboard. After initial load of the dashboard, we then click one of the filter charts at the top, or manually edit the saved search filter. That then loads data into the charts below.
Having drill downs would allow us to separate "click to filter" charts, away from "data intensive charts" - which would avoid the "way too much data to load..." issue.
Here are some commercial products with drill down capabilities
Today we can make a field a URL and use the field's value in the URL Template. I'd also like to be able to use meta data such as the current time range. Additionally, it would be helpful if other fields from the same document could be used.
For example, let's say I'm looking at intrusion detection events like this for the 24 hours starting 2018-08-16:
{
"source_ip" : 1.2.3.4,
"dest_ip" : 5.6.7.8,
"signature_id" : 9101112
}
Now I want to pivot to my forensic packet capture tool, which has a URL scheme like this:
https://toolname.mycompany.com/search?query=src:1.2.3.4+AND+dst:5.6.7.8&start=2018-08-16&end=2018-08-17
In order to craft a URL template for that, I need access to the value of both the source_ip and the dest_ip fields from the document as well as the start and end times of the date range.
+1
+1
+1
+1
In this thread https://discuss.elastic.co/t/access-first-location-of-all-in-visual-builder-markdown/150500 we discuss a possible minor enhancement to Visual Builder that would enable end users to effectively create all types of drill down displays.
+1 Whats the timeline on this feature?
+1
Ok, this issue is now more than 2 years old. Can anyone suggest a decent work-around?
Edit: Found This: https://thenewstack.io/inserting-links-kibana-dashboards/
@hdave, if you find that link useful take a look a little further up the thread: https://github.com/elastic/kibana/issues/12560#issuecomment-404449251
@alexfrancoeur @trevan I just wanted to circle back and provide some field feedback. The most common request I get regarding drill-down from prospects and customers is the ability to click on a datapoint within a chart and use that datapoint to link out to another related dashboard. Then, have that datapoint automatically populated as a filter for the next dashboard. Finally, easily go backwards and retrace steps to the origin of your click path if you do this a few times. I wasn't sure if this is exactly how this would work, but wanted to provide additional feedback in case it's helpful.
@bradquarry thanks for sharing and this is exactly one of the workflows we plan to support with Kibana Actions (https://github.com/elastic/kibana/issues/32371). There is still a lot of work to do, but I'd subscribe to that meta issue to track progress on the implementation of actions.
Finally, easily go backwards and retrace steps to the origin of your click path if you do this a few times. I wasn't sure if this is exactly how this would work, but wanted to provide additional feedback in case it's helpful.
@bradquarry would the using back button work or are you expecting that breadcrumbs would be populated?
cc: @stacey-gammon @AlonaNadler
@alexfrancoeur Got it, thanks. Breadcrumbs would provide a smoother experience. If they stand out well and people use them they might even decrease ES search workload in aggregate. If you take 4 drill in steps and can go right back to the beginning vs hitting the back button 3 times you wont have to potentially perform unneeded dashboard refreshes. I think this is good as cluster size is mostly driven by data volumes, and we like highly efficient fast queries.
+1
+1
+1
+1 from me. This feature would be brilliant to have
Hi alexfrancoeur,
when this enhancement cab be released?
There's a user scenario from me(please check below). Does it include in this requirement?
I have data like below in Kibana and created a pie chart based on field 'ktf2'. In the pie chart, I want to view corresponding 'cn' information by clicking one of the slices of the pie chart.
cn,ci,at,ktf2
Siemens, SiemensPS, Search, "Competency_Criteria,Profile_Criteria"
HSBC, HSBCPROD, Search, "Keyword, Competency_Criteria, Background_Criteria"
......Thanks,
Cherie
+1
Let's do this, please... without it, Kibana is pretty limited. My users are desperate for this type of functionality +1
There is a lot of stuff discussed in this issue that would be really useful. However I wanted to point out to those waiting for these features, that there are things that can be done in Kibana today to dynamically link dashboards and facilitate a number of drill-down scenarios. The relevant features might not be obvious, and it does involve taking some time to figure out the structure of a Kibana URL, including how to set filters by manipulating the URL directly. While this requires a bit of work, the results can be very useful. Let me share a few examples...
Use the markdown visualization to group related dashboards
The markdown visualization can be used to navigate between dashboards, which is especially useful for dashboards related to the same use-case.
This is achieved using simple hyperlinks to the dashboards:
TIP: We recommend modifying the Kibana advanced setting
filters:pinnedByDefault
, setting it totrue
. This will keep filters set on one dashboard, when navigating to another. This is very useful when filtering data to drill-down and then changing dashboards for a different perspective on that same data.Use the Markdown feature of Visual Builder
- Define the data.
- Use the data to dynamically set filters via the URL that is used in the Markdown links (once you have figured out the URL structure it is not as hard as looks here).
- Set any relevant filters and even use CSS to tweak how stuff is displayed.
- The result
Clicking on the color goes to one dashboard...
And clicking on the number links to another...
Notice how the filters are set via the URL. This is how we make dashboards to launch into certain devices, applications and use-cases, like these...
Use the URL field formatter
- Goto the Index Pattern configuration within Kibana, and edit the settings for the field that you would like to use within a URL. The format will be
default
:
- Change it to
Url
.- Ensure the type is
Link
- Label Template should be set to
{{value}}
- The Url Template should include the URL to open with
{{value}}
inserted where the field value will be.- You can also select whether to launch the URL in a new tab or not.
Together this looks like this...
If you use these fields in a visualization such as a table, you will notice the values are now hyperlinks.
Clicking on them will launch the configured URL. In the example above the Talos page is launched for that IP address.
This same method can be used to launch to another dashboard, leveraging the Kibana URL methods showed in the Markdown example...
While these techniques do not handle all of the scenarios that the things discussed in this issue hopefully will. They can be used to significantly improve users' interactions with Kibana.
We use these techniques to allow the users of our Network and Security Analytics solution to use Kibana without ever having to open the full list of dashboards. The navigation and drill-down options we provide allows them to navigate to any perspective they need, and to rarely need to manually set filters when doing so.
Hopefully some of you find these tips helpful as you wait for future enhancements.
very cool
At GoodLabs Studio, one of the most asked-for features we get from our clients that use Kibana is the drill-down feature. This thread was started 3 years ago and not much has changed, so we decided to take a crack at it and open source it for others to use.
Our drill-down plugin allows users to access as many targets as they want; targets can range from other Kibana dashboards or visualizations to external URLs. All of which are presented in a simple popup. In addition the targets can be configured to include filters that are applied on the next view. Users configure the drill-down feature the same way they configure index patterns by simply applying the drill-down field formatter provided by our plugin to their data. The drill-down field formatter can be applied to multiple data types and as discussed above, there are no limits to the number of drill-down targets that can be added.
Filters are applied through URL templates that the user configures when they setup the drill-down field formatter. Within this post thread, one of the posts states - “add a dynamic filter based on a click. This would be for where I have a table/line that shows multiple servers and high level values. I'd like to click on the server and then go to a different dashboard to see details for that specific server.” This post highlights exactly what we have implemented, this feature allows users to analyze their data with less actions—making kibana easier to use. For those that are interested, the plugin can be found at: https://github.com/goodlabs-studio/kibana-plugin-drilldownmenu. Any feedback is welcome!
Few updates on this issue. Few weeks Kibana team merged drilldown into master. You can now create drilldowns from one dashboard to the other configuring what you wish to carry over to the desitnation (global filters, search query, and timerange). For each panel you can have multiple drilldowns for different destinations. On interaction users will have options to choose where to go (or if to perform the regular filter).If all goes well that will be part of the upcoming release. Please let us know if you find it useful and if anything is missing
In addition we already are working on the next steps with different more advanced drilldown including to external urls and other solutions within Kibana https://github.com/elastic/kibana/pull/63710
Thanks you for continuing providing feedback 👍
cc: @elastic-jb @streamich @Dosant
@AlonaNadler Is there any easy way (like a docker image) to try the merged drilldown feature?
Closing this as drilldowns has been released in Kibana.
Most helpful comment
There is a lot of stuff discussed in this issue that would be really useful. However I wanted to point out to those waiting for these features, that there are things that can be done in Kibana today to dynamically link dashboards and facilitate a number of drill-down scenarios. The relevant features might not be obvious, and it does involve taking some time to figure out the structure of a Kibana URL, including how to set filters by manipulating the URL directly. While this requires a bit of work, the results can be very useful. Let me share a few examples...
Use the markdown visualization to group related dashboards
The markdown visualization can be used to navigate between dashboards, which is especially useful for dashboards related to the same use-case.
This is achieved using simple hyperlinks to the dashboards:
TIP: We recommend modifying the Kibana advanced setting
filters:pinnedByDefault
, setting it totrue
. This will keep filters set on one dashboard, when navigating to another. This is very useful when filtering data to drill-down and then changing dashboards for a different perspective on that same data.Use the Markdown feature of Visual Builder
Clicking on the color goes to one dashboard...
And clicking on the number links to another...
Notice how the filters are set via the URL. This is how we make dashboards to launch into certain devices, applications and use-cases, like these...
Use the URL field formatter
default
:Url
.Link
{{value}}
{{value}}
inserted where the field value will be.Together this looks like this...
If you use these fields in a visualization such as a table, you will notice the values are now hyperlinks.
Clicking on them will launch the configured URL. In the example above the Talos page is launched for that IP address.
This same method can be used to launch to another dashboard, leveraging the Kibana URL methods showed in the Markdown example...
While these techniques do not handle all of the scenarios that the things discussed in this issue hopefully will. They can be used to significantly improve users' interactions with Kibana.
We use these techniques to allow the users of our Network and Security Analytics solution to use Kibana without ever having to open the full list of dashboards. The navigation and drill-down options we provide allows them to navigate to any perspective they need, and to rarely need to manually set filters when doing so.
Hopefully some of you find these tips helpful as you wait for future enhancements.