Kibana: Tasks UI

[cjcenizal]

Replaces old discussion: https://github.com/elastic/kibana/issues/18411

CC @cachedout @elastic/stack-monitoring

Overview

Long-running tasks can leak processes and kill a cluster. Users need a way to detect slow queries and kill these tasks. We can start down the road of addressing this problem by building a “Tasks” app that show a list of all tasks that are running, indicate ones that are cancellable, display any available information that the API provides, and give users the ability to cancel these tasks.

Note that we’ll have to disable cancel functionality if the user’s roles don’t support it (it’s likely that cluster monitoring and cluster manage are the required roles, but we need to double check this). Some of the other information this UI will surface:

• How long a task has been running
• Which nodes they’re executing on
• Cross-link to the node a task is running on in Monitoring, if Monitoring is available

Prototype

In 2016, @ycombinator built a prototype UI. Here are some of the notable features demonstrated by the prototype.

Parent-child relationship

There is a parent-child relationship between tasks, represented by indentation in the prototype.

Task-specific states, e.g. reindexing

The particular task demonstrated by the prototype was a reindex task, which had multiple states that the UI could represent. Note that there are many different types of tasks with their own variety of states.

"Documents created" state

When reindexing into an empty index, documents are created. This is surfaced by the reindex task's progress bar.

"Documents updated" state

If a simultaneous reindex task is started with the same target index, documents get updated as well as created, also indicated by the progress bar.

"Documents created and updated" state

If the original reindex task is canceled, then eventually the new task will being creating new documents instead of just updating existing ones. In this case, the progress bar indicates both states.

Task cancellation

Requests for cancelling tasks are asynchronous, so the prototype notifies the user of when the request to cancel a task has been sent. For the new UI, we should probably just gray out the row in the table and replace the action buttons with a spinner and the text "Deleting".

Once cancelled, the row is removed from the table and a toast notifies the user of the successful cancellation.

[elasticmachine]

Pinging @elastic/es-ui

[ycombinator]

So happy to see this coming to life! Thanks @elastic/es-ui team! ❤️

[ycombinator]

What follows is way in the weeds but just wanted to dump my thoughts since I was here already.

Cross-link to the node a task is running on in Monitoring, ...

You will need to know two pieces of data to form the link to the Monitoring Node Overview page:

• The Elasticsearch cluster's ID (CLUSTER_ID below). This is the value of the cluster_uuid field as returned by the GET / ES API response.
• The Elasticsearch node's ID (NODE_ID below).

Then you can form a link to the Monitoring Node Overview page like so: /app/monitoring#/elasticsearch/nodes/NODE_ID?_g=(cluster_uuid:CLUSTER_ID).

... if Monitoring is available

At the risk of possibly over-engineering a bit 🤓:

The Stack Monitoring plugin could expose a JS API that:

• informs if Stack Monitoring UI is available (and perhaps why if it's not), and
• if the UI is available, a function to generate the correct Monitoring Node Overview page link given a ES cluster UUID and node UUID. That way, if Stack Monitoring wants to change the structure of the Node Overview page URL, the ES Management code base doesn't have to change anything on it's end.

[cachedout]

@ycombinator Such an API is something I've been thinking a little bit about as well. What do you think about starting a doc or another GH issue to track that independently?

[chrisronline]

if the UI is available, a function to generate the correct Monitoring Node Overview page link given a ES cluster UUID and node UUID. That way, if Stack Monitoring wants to change the structure of the Node Overview page URL, the ES Management code base doesn't have to change anything on it's end.

This is how Infra team did this with the Logs UI.

See this issue and this PR

[ycombinator]

@cachedout Yeah, lets take it to a separate GH issue.

[cachedout]

@ycombinator New issue opened: https://github.com/elastic/kibana/issues/43100

[alisonelizabeth]

Closing this issue. After discussing with the ES team, we've decided not to create a generic tasks UI. We plan on creating more use-case specific UIs for managing tasks (e.g., reindex, search).

cc @alexfrancoeur @lukasolson @epixa

[cjcenizal]

@jkelastic built a similar prototype in EUI. He and other SEs helped us understand the value prop of a generic Tasks UI like this:

• Visibility into the system helps users understand what's going on with their cluster and understand the bigger picture.
• Seeing these tasks gives people something concrete that can explain cluster behavior, e.g. degraded performance.
• Having the ability to cancel and throttle tasks in a UI means giving users greater control over their cluster.
• A UI is easier to consume than an API.

I'm reopening this so we can move forward with adding the Tasks UI.

[jzzfs]

I took the liberty of implementing this, albeit outside of Kibana. Here's the web app and here's the repo.