Kibana: Metricbeat default visualization "Disk Usage [Metricbeat System]" broken

[CodingSpiderFox]

Accessing the default Kibana Disk Usage [Metricbeat System] vizualization shows no data while the is suitable data in the index:

Editing the visualization shows that there is no field selected.

Selecting "beat.hostname.keyword" made it show something again. (also see #16122)

However, as there is actual values as shown above, the value "0%" is wrong.

As there are a lot of values being zero, I wanted to make sure to test this with data that is surely over 0%.

So I changed the aggregation to "Max"

Still 0% are shown. At least 7.83 % would have to be shown. (see first screenshot)

Kibana version:
6.1.2
Elasticsearch version:
6.1.2

Original install method (e.g. download page, yum, from source, etc.):
Docker

Description of the problem including expected versus actual behavior:

Steps to reproduce:
Install ELK Stack 6.1.2 with docker

Loaded metricbeat index and dashboard trough following command

#!/bin/bash

COMPOSE_NET=dockercomposeelkstack_default
STACK_VERSION=6.1.2

echo "Start initializing metricbeat"

echo "Set file permissions for metricbeat configuration file"
sudo chown root:root ./setup/init_configs/metricbeat.yml
sudo chmod go-w ./setup/init_configs/metricbeat.yml

docker run --net ${COMPOSE_NET} --rm --cap-add NET_ADMIN \
    -v $(pwd)/setup/init_configs/metricbeat.yml:/usr/share/metricbeat/metricbeat.yml:ro \
     docker.elastic.co/beats/metricbeat:${STACK_VERSION} \
     /usr/share/metricbeat/metricbeat setup

echo "metricbeat initialization finished"

with the metricbeat.yml being

# we need at least one module so metricbeat can start up

metricbeat.modules:

#------------------------------- System Module -------------------------------
- module: system
  metricsets:
    # CPU stats
    - cpu

#============================== Dashboards =====================================
# These settings control loading the sample dashboards to the Kibana index. Loading
# the dashboards is disabled by default and can be enabled either by setting the
# options here, or by using the `-setup` CLI flag or the `setup` command.
setup.dashboards.enabled: true

# The URL from where to download the dashboards archive. By default this URL
# has a value which is computed based on the Beat name and version. For released
# versions, this URL points to the dashboard archive on the artifacts.elastic.co
# website.
#setup.dashboards.url:

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  host: "kibana:5601"

#================================ Outputs =====================================

# Configure what outputs to use when sending the data collected by the beat.
# Multiple outputs may be used.

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["elasticsearch:9200"]


  #protocol: "https"
  username: "<HIDDEN>"
  password: "<HIDDEN>"

[epixa]

@simianhacker Any ideas on this one?

[simianhacker]

‘beat.hostname.keyword’ is only available when your mapping’s for Metricbeat are not setup correctly. My best guess is that the Metricbeat process didn’t install the index template correctly. @elastic/beats-team

[exekias]

Hi @CodingSpiderFox, you detailed the steps you use to set up the dashboards. I'm wondering how you actually run Metricbeat to ship metrics, as there is where the index template is installed. Could you detail that and dump the log output you get while starting it?

[CodingSpiderFox]

I think the reason was that other nodes on the network were sending data to my dev instance of ELK stack. Maybe that's why it got messed up before the index templates were loaded. So this ist fixed for me now

[faulander]

I have exactly the same problem. Disk Usage doesn't show anything correct.
What information do you need?

[internetztube]

Same problem here.

[wanghanlin]

Same problem here.

[buithehien1991]

Same problem

[frankfoti]

Hitting this as well. Kibana 6.6, metricbeat index was crated directly from metricbeat 6.6 then data is being shipped though logstash.

[elasticmachine]

Pinging @elastic/kibana-app

[zetser]

Change period to 10s in modules.d/system.yml

- module: system
  period: 10s
  metricsets:
    - filesystem
    - fsstat
  processors:
  - drop_event.when.regexp:
      system.filesystem.mount_point: '^/(sys|cgroup|proc|lib)($|/)'

[immaciek]

Same problem - 'Disk used' gauge also doesn't work.

[davidbernard]

same problem 6.7

[zetser]

All you have to do is that the update times coincide with the data delivery time of metricbeat.

My configuration is as follows:

system.yml

- module: system
  period: 10s
  metricsets:
    - filesystem
    - fsstat
  processors:
  - drop_event.when.regexp:
      system.filesystem.mount_point: '^/(sys|cgroup|proc|lib)($|/)'

I have set the period parameter in 10 seconds

Now go to the Visualize option in the menu and type Disk in the search field.

And click on the one called Disk Usage [Metricbeat System]

This graph is of the Top N type

To make a test change the graph type to Time Series

1. This value can affect the way your data is graphed globally
2. This value is the one that you must adjust according to the configuration you have in your system.yml file in the Period parameter

If you put this value too low the following will happen

If the graph has spikes that is what is causing that sometimes do not show data.

All you have to do is look for a value in which the graph shows up like this:

You just have to select again the Top N graph type and save the changes and you're done

I hope I've helped and sorry for my Google Traductor English :V

[Chernomazov]

I have something to add.
ELK: 6.7
MetricBeat: 3.4.1
Period for metricsets "filesystem" is default 1 min. Autorefresh period for Kibana is 5 sec.
Firstly I had null values and full-sized green bars.

After few refreshs I got picture with right values and colored value-sized bars.

After few refreshs I got picture with null values and green value-sized bars (order is reversed).

[Chernomazov]

5s/60s=12 steps. On 1st and 2nd steps bars are colored (2nd picture). On 3-12 steps bars are green (3rd picture).

[Chernomazov]

I kept autorefresh period for Kibana equals 5 sec.
When MetricBeat's period equals 1 minute I tried to change interval at Kibana's Panel Options to 1m, 2m, 3m, and 5m. Bars keep colored n-1 minutes but switch to green for 1 minute.
I changed MetricBeat's period equals 45 and 50 seconds, and tried interval at 1m and 2m. Bars keep colored for some time but always switch to green. Bars are switch to green at first 5-10 seconds of minute.

[simianhacker]

@Chernomazov Have you tried setting the interval to >=10s? That prevent the auto bucketing from going below 10 seconds, since it looks like your data is being recorded in 10 second intervals.

[Chernomazov]

@simianhacker I've tried this setting. Data comes at 47th second of minute, bars become colored at 50th second, and switch back to green at 05th second of new minute.

Data is recorded in 1 minute interval. It's default interval value for "filesystem" metricset.

[simianhacker]

@Chernomazov Then you would need to treat it as >=1m. The idea being you need your bucket size to encompass the minimum number of data points. If your data comes in on the 47th second then you need to round up to 1 minute.

[Chernomazov]

@simianhacker I tried >=1m earlier, and it didn't work. I've tried it today, and it works. Interval 1m is working today too. Interval 1m didn't work yesterday as I wrote earlier. Interval auto isn't working today.
I save interval to >=1m, and keep watching.

[Chernomazov]

Metricbeat 7.0 visualization "Disk Usage [Metricbeat System] ECS" has same issue.
I had to change Interval to >=1m to make it works the proper way.

[maulinglawns]

I can confirm this behaviour in metricbeat 7.3 as well.
Have experimented with different intervals in visualization to no avail. It sometimes work, mostly it doesn't!

Update: Actually, setting the panel update to >=1m seems to be working now for my usecase.
Thanks @simianhacker