Openjdk-infrastructure: Number of hosts require adding to Nagios monitoring

[tellison]

We have brought a number of new machines on-line recently, but they need adding to the Nagios monitoring service.

e.g.
build-win2012-1 (37.58.103.195)
build-win2012-2 (37.58.103.196)
jck-softlayer-x86-ubuntu-16.04-1 (159.122.210.205)
jck-softlayer-x86-ubuntu-16.04-1 (159.122.210.194)
build-cloudcone-x64-ubuntu-16.04-1 (??)
build-plymouth-x64-solaris11-1 (offline?)

possibly more

[tellison]

P'haps the zOS machines can't play in this scheme?
build-marist-s390x-zOS-1
build-marist-s390x-zOS-2

[tellison]

The host following host should be removed from Nagios
build-joyent-x64-ubuntu-16.04-1

[tellison]

New host for JCK services. WOuld be good to have Nagios check these keep working to avoid strange JCK failures.

jck-softlayer-x64-ubuntu-16.04-3 (aka jckservices.adoptopenjdk.net) (158.176.72.83)

[geraintwjones]

As far as I can tell, build-joyent-x64-ubuntu-16.04-1 is not in Nagios. Did someone remove it?

[geraintwjones]

jck-softlayer-x86-ubuntu-16.04-1 (159.122.210.205)

Logged on to 159.122.210.205. It reports itself as jck-softlayer-x64-ubuntu-1604-1, so that's what I've added.

[geraintwjones]

Added jckservices.adoptopenjdk.net (158.176.72.83)

[geraintwjones]

Working through the list of servers documented in the wiki here, adding those not already in Nagios.

[karianna]

Is this something we could add to ansible bootstrapping for future boxes?

[geraintwjones]

I was thinking about that. I think it is something we should be doing with Ansible. I'll take a look at what we've already got there tomorrow.

[bblondin]

yes, last year I wrote a bash script for Nagios Admins to easily create machine templates.
Today I told another look at, I have rewrote it Nagios_Ansible_Config_tool.sh
It is now working in my environment.

A few changes to the playbook are required and some infra configuration, but it will now install the nagios plugins then contact the nagios master and create the template for the machine and load it.

[bblondin]

@geraintwjones if you can, we need playbooks for nagios plugins
we currently only have plays for Ubuntu and RHEL

[bblondin]

Pull request created #161

Nagios master auto config

[bblondin]

Created play for Nagios Plugins on SLES #164

[bblondin]

New playbooks with 'Nagios master auto config' are working nicely.
now when the playbook is run on a system it will automatically get added and monitored by nagios :)

[geraintwjones]

Working on the CentOS playbook...

[karianna]

@geraintwjones I believe your PR is done and this can be closed?

[geraintwjones]

@karianna I decded to get all playbooks added first, and then run them against any machine that needs to be added. That PR only added the CentOS playbook, so there's still some work to be done.

[geraintwjones]

Comparing the list of machines documented in infrastructure.md with the list of machines currently monitored by Nagios at https://nagios.adoptopenjdk.net/nagios/, I've observed the following...

• Machines listed in infrastructure.md NOT monitored by Nagios
  
  
  
  • build-1and1-win2008r2-1 (Win Server 2008 R2)
  
  
  • build-joyent-x64-smartOS-14-1 (SmartOS 14.4.0)
  
  
  • build-joyent-x64-ubuntu-16.04-1 (Ubuntu 16.04)
  
  
  • build-marist-s390x-zOS-1 (zOS 2.1)
  
  
  • build-marist-s390x-zOS-2 (zOS 2.1)
  
  
  • build-osuosl-ppc64le-fedora-24-2 (Fedora 24)
  
  
  • build-win2012-1 (Win Server 2012 R2)
  
  
  • build-win2012-2 (Win Server 2012 R2)
  
  
  • jck-macstadium-macos1010-1 (MacOS 10.10)
  
  
  • jck-macstadium-macos1010-2 (MacOS 10.10)
  
  
  • jck-packet-armv8-ubuntu-16.04 (Ubuntu 16.04)
  
  
  • test-packet-x64-windows-2012r2-1 (Win Server 2012 R2)
  
  

We don't have Ansible playbooks for adding Fedora 24, FreeBSD 11-1, MacOS 10.10, SmartOS 14.4.0, Win Server 2008 R2, Win Server 2012 R2 or zOS 2.1 yet.

I'll run the Ubuntu playbook on build-joyent-x64-ubuntu-16.04-1 and jck-packet-armv8-ubuntu-16.04, and then start writing playbooks for the missing operating systems if Nagios supports them.

[geraintwjones]

build-joyent-x64-ubuntu-16.04-1 has been deleted, so needs to be removed from infrastructure.md and from KeyBox.

[geraintwjones]

Deleted build-joyent-x64-ubuntu-16.04-1 from KeyBox.

[geraintwjones]

Removed build-joyent-x64-ubuntu-16.04-1 from infrastructure.md under pull request #191. Waiting for it to be approved/merged.

[bblondin]

240 Unix playbook now supports Nagios plugin installation on:

SLES 12 x
SLES 11 x
SLES 11 p
SLES 12 s390x

rhel6 p
rhel6 x
rhel7 x
rhel7 le
rhel7 p
rhel7 s390x

Raspbian 8 armv7l

Ubuntu 14 x
Ubuntu 14 le

Ubuntu 16 X
Ubuntu 16 s390x
Ubuntu 16 LE

[bblondin]

I think we good on this, closing