Beats: [Metricbeat] - vsphere module request

[EcHoOfInSaNiTy]

Hello,

I love the vsphere module that was added to metricbeat, however there isn't enough different kinds of metrics being pulled for me to move away from my current metric collection methods. Would it be possible to include metrics that for example Zabbix can pull by default? This would be huge for me in terms of moving everything over to metricbeat for data collection.

I.E:
_Where hv is hyper-visor metrics and vm is virtual-machine metrics__
vmware.hv.cluster.name
vmware.hv.cpu.usage
vmware.hv.datacenter.name
vmware.hv.fullname[
vmware.hv.hw.cpu.freq
vmware.hv.hw.cpu.model
vmware.hv.hw.cpu.num
vmware.hv.hw.cpu.threads
vmware.hv.hw.memory
vmware.hv.hw.model
vmware.hv.hw.uuid
vmware.hv.hw.vendor
vmware.hv.memory.size.ballooned
vmware.hv.memory.used
vmware.hv.network.in
vmware.hv.network.out
vmware.hv.sensor.health.state
vmware.hv.status
vmware.hv.uptime
vmware.hv.version
vmware.hv.vm.num
vmware.hv.datastore.read
vmware.hv.datastore.write
vmware.hv.datastore.size
vmware.hv.datastore.size
vmware.hv.cluster.name
vmware.hv.cpu.usage
vmware.hv.datacenter.name
vmware.hv.fullname
vmware.hv.hw.cpu.freq
vmware.hv.hw.cpu.model
vmware.hv.hw.cpu.num
vmware.hv.hw.cpu.threads
vmware.hv.hw.memory
vmware.hv.hw.model
vmware.hv.hw.uuid
vmware.hv.hw.vendor
vmware.hv.memory.size.ballooned
vmware.hv.memory.used
vmware.hv.network.in
vmware.hv.network.out
vmware.hv.sensor.health.state
vmware.hv.status
vmware.hv.uptime
vmware.hv.version
vmware.hv.vm.num
vmware.vm.cluster.name
vmware.vm.cpu.num
vmware.vm.cpu.ready
vmware.vm.cpu.usage
vmware.vm.datacenter.name
vmware.vm.hv.name
vmware.vm.memory.size.ballooned
vmware.vm.memory.size.compressed
vmware.vm.memory.size.private
vmware.vm.memory.size.shared
vmware.vm.memory.size.swapped
vmware.vm.memory.size.usage.guest
vmware.vm.memory.size.usage.host
vmware.vm.memory.size
vmware.vm.powerstate
vmware.vm.storage.committed
vmware.vm.storage.uncommitted
vmware.vm.storage.unshared
vmware.vm.uptime
vmware.cluster.status

[ruflin]

It seem for the virtual machine metrics we already have a subset on what you suggest above. This means here the number of metrics which are collected would have to be expanded.

We don't have a hyper-visor metricset yet. This would have to be created. The key here is if this API is exposed by the github.com/vmware/govmomi library we use.

Want to try to contribute the above metrics? You seem to have all the environment already available which is needed for manual testing.

[EcHoOfInSaNiTy]

Sure!

[PauliniMrtns]

This a cool topic, I need this as well. I would appreciate some help from the community with this because I too need this implementation for VMWare monitoring!

[MnrGreg]

@cblomart has done a great deal of work collecting vsphere metrics using GOVMOMI and the vSphere Metric Groups / perfManager method: vsphere.go

With this implementation one is able to define a list metrics deemed relevant to the administrator. One could apply this same logic to the MetricBeats vsphere module by adding a parameter similar to get_custom_fields.

For example:

metricbeat.modules:
- module: vsphere
   get_custom_metrics:
     - cpu.ready.summation
     - virtualDisk.totalWriteLatency.average
     - virtualDisk.totalReadLatency.average

I expect there would be a significant increase in adoption of Elastic/MetricBeats if this could be implemented. Large organizations are always looking to rationalize and standardize their portfolio of tools. In terms of on-premise hypervisor deployments VMware must be in the top 3.

This would presumably be more efficient then collecting OS level metrics from the MetricBeats System agent installed within the OS. In a few scenarios this eliminates the need for the MetricBeats OS agent altogether.

To get a true performance monitoring picture, such as evaluating CPU oversubscription, one will always need the hypervisor level stats.

I would love to see the community jump onto this. Also hopeful that Elasticsearch BV can dedicate some people!?

Addresses #5180

[MnrGreg]

For those interested, we've added an Elastic backend to vsphere-graphite. With this binary one is able to specify which metrics should be collected from vSphere's realtime performance counters. This includes useful metrics like guest disk capacity usage and guest CPU Ready.

Grafana dashboards also available here

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.