Beats: Create Nats modules

[ruflin]

Beats Modules / Datasets release checklist

This checklist is intended for Devs which create or update a module to make sure modules are consistent.

Modules

For a metricset to go GA, the following criterias should be met:

• [x] Initial PR with metricset https://github.com/elastic/beats/pull/9825
• [x] Supported versions are documented
• [x] Supported operating systems are documented (if applicable)
• [x] Integration tests exist
• [x] System tests exist
• [x] Automated checks that all fields are documented
• [x] Documentation
• [x] Fields follow ECS and naming conventions
• [x] Kibana Home Tutorial (if applicable)

Filebeat module

• [x] Test log files exist for the grok patterns
• [x] Generated output for at least 1 log file exists
• [x] Dashboards exists (if applicable)

Metricbeat module

• [x] Example data.json exists and an automated way to generate it exists (go test -data)
• [x] Test environment in Docker exist for integration tests
• [x] Dashboards exists (if applicable)

[ChrsMark]

@ruflin just a design question, as far as the addition of the remaining metricsets is concerned.

Initially we had 4 metricsets, one for each of the NATS' monitoring URIS, varz, connz, routez, subsz. On #9825, varz was introduced but it was decided to be called stats. So the question now is how do you suggest we should call the remaining three? Is it ok to go with the name of their URI, or do you consider something else as more meaningful?

[ruflin]

I don't know much about nats but I assume connz stands for connection? routez for route and subz ...? So in general if we can make it a bit more meaninful for non nats experts great, if not, I'm also ok with the original names.

[ruflin]

@skatsaounis @ChrsMark Any chance you could also contribute some dashboards for this module?

[skatsaounis]

Hi @ruflin,

We are going to do it after merging the last metricset patch. In terms of completeness of NATS module is there something else we are missing?

[ruflin]

@skatsaounis Will go through the checklist above as soon as we got all the PR's in. I think dashboards and Kibana Home Tutorial (if applicable) is the only thing missing. But Kibana Home Tutorials I hope we can take care of as this has to happen in the Kibana repo. It's to make the modules more visible to our users.

Is there also potential here for a filebeat modules for Nats? I assume it has some logs ...

[ruflin]

@ChrsMark I updated this issue to not only track the metricbeat Nats module but also the Filebeat one.

For creating the Filebeat module the most important first step is to get some logs from nats to play around with and figure how many different log types there are.

[skatsaounis]

@ruflin I wan to inform you that I am going to create a PR to Kibana regarding Nats Metrics Home Tutorial. As soon as it is ready I am going to mention this issue to make it visible here.

[skatsaounis]

@ruflin If you could find some time it would be nice to have this issue's checklist updated.

[ruflin]

@skatsaounis Sorry for the late reply, was of for a bit. It seems from the above, only thing missing is dashboards for the Filebeat modules?

[skatsaounis]

@ruflin Indeed. When we close the enhancements PR we are going to open another one to add a dashboard

[skatsaounis]

@ruflin Hi Nicolas! Thank you for merging the Filebeat Dashboard related PRs. May I ask you what is missing to consider the NATS work as complete? We would love to see NATS Filbeat module become part of version 7 as well.

PS: The last checkpoint can be marked as complete :smile:

[ruflin]

Thanks for all the work on this. I'm going to close this issue.

For the releases: The metricbeat module will be in 7.0 but the Filebeat module came in after feature freeze so will only make it into 7.1.