netdata 🚀 - Netdata Unbound dns not Working

Just to clarify, are you having issues with Unbound, or with PowerDNS? They are two different pieces of software and the Netdata modules for them are completely unrelated.

Ferroin on 2 Oct 2018

@Ferroin im trying to install plugin of unbound on netdata. I havent got any powerdns service on my os.

askd77 on 2 Oct 2018

OK, just making sure, because the log errors you posted seem to be related to PowerDNS (and a handful of other DNS modules), not Unbound.

Can you try running the following command as the user which you run Netdata as (usually netdata)?

/usr/libexec/netdata/plugins.d/python.d.plugin unbound debug trace 1

That will run just the unbound module by itself in debug mode. If that exits by itself, post all the output here, otherwise, post any error messages you see in the output.

Ferroin on 2 Oct 2018

@Ferroin im attaching you the error code after run of debugger

2018-10-02 14:54:30: python.d INFO: plugin: main: Using python 2 2018-10-02 14:54:30: python.d DEBUG: plugin: main: loading '/etc/netdata/python.d.conf' 2018-10-02 14:54:30: python.d ERROR: plugin: main: cannot load '/etc/netdata/python.d.conf': [Errno 2] No such file or directory: '/etc/netdata/python.d.conf'. Will try stock version. 2018-10-02 14:54:30: python.d DEBUG: plugin: main: loading '/usr/lib/netdata/conf.d/python.d.conf' 2018-10-02 14:54:30: python.d INFO: plugin: main: No modules to run. Exit...

askd77 on 2 Oct 2018

It looks like you probably don't have the plugin enabled.

You can enable it by copying /usr/lib/netdata/conf.d/python.d.conf to /etc/netdata/python.d/conf and editing this line (should be around line 91):

unbound: no

to instead read:

unbound: yes

Assuming you've got the other configuration mentioned in https://github.com/netdata/netdata/blob/master/python.d/README.md#unbound taken care of, it should just work.

Ferroin on 2 Oct 2018

@Ferroin thank you for you help. Please note the plugin are running well. Check the below export:
Also restart the netdata service but on web interface im not able to locate the section of DNS. Could you please assist?

`BEGIN unbound_local.num_queries 1000046
SET 'num.query.type.A' = 0
SET 'num.query.type.AAAA' = 0
SET 'num.query.type.CNAME' = 0
SET 'num.query.type.MX' = 0
SET 'num.query.type.NS' = 0
SET 'num.query.type.PTR' = 0
SET 'num.query.type.SOA' = 0
SET 'num.query.type.SRV' = 0
SET 'num.query.type.TXT' = 0
SET 'num.query.type.other' = 0
END

BEGIN unbound_local.cache 1000046
SET 'total.num.queries' = 0
SET 'total.num.cachehits' = 0
SET 'total.requestlist.overwritten' = 0
SET 'total.requestlist.exceeded' = 0
SET 'unwanted.queries' = 0
SET 'unwanted.replies' = 0
SET 'num.query.tcp' = 0
SET 'num.query.ipv6' = 0
END

BEGIN unbound_local.num_queries_flags 1000046
SET 'num.query.flags.QR' = 0
SET 'num.query.flags.AA' = 0
SET 'num.query.flags.TC' = 0
SET 'num.query.flags.RD' = 0
SET 'num.query.flags.RA' = 0
SET 'num.query.flags.Z' = 0
SET 'num.query.flags.AD' = 0
SET 'num.query.flags.CD' = 0
SET 'num.query.edns.present' = 0
SET 'num.query.edns.DO' = 0
END

BEGIN unbound_local.answers 1000046
SET 'num.answer.rcode.NXDOMAIN' = 0
SET 'num.answer.rcode.nodata' = 0
SET 'num.answer.secure' = 0
SET 'num.answer.bogus' = 0
SET 'num.rrset.bogus' = 0
END

BEGIN unbound_local.memory 1000046
SET 'mem.cache.rrset' = 0
SET 'mem.cache.message' = 0
SET 'mem.cache.dnscrypt_shared_secret' = 0
SET 'mem.cache.dnscrypt_nonce' = 0
SET 'mem.mod.iterator' = 0
SET 'mem.mod.validator' = 0
`

askd77 on 2 Oct 2018

Netdata doesn't display charts on the web interface that have only ever had zero values for all their dimensions. If you send some DNS queries through Unbound, the charts should show up. It may take a couple of tries reloading the web interface though (some browsers are really aggressive about caching it.

Ferroin on 2 Oct 2018

@Ferroin this issue include the icon of unbound dns? because this is different from diagrams with statistics. In my proposal when a plugin is enabled the icon may to appear on web interface.

askd77 on 2 Oct 2018

👍1

@Ferroin i see all dimension ids have dots in name. This is wrong, should be _ instead.

ilyam8 on 3 Oct 2018

I have no idea how I missed this, but parsing is broken somehow here.

@askd77 Did you install netdata using one of the upstream installation methods (docker image, or one of the kickstart scripts), from a local build, or from your distribution's package manager?

Ferroin on 3 Oct 2018

@Ferroin installed the net data from Centos7 package manager. I confirmed today nothing changed with netdata statistics doesn’t show anything and I have hits on my unbound. Also refreshed a couple of times different browsers

askd77 on 3 Oct 2018

@askd77 I'm going to look at getting a CentOS VM set up to try and reproduce this, since I've been unable to on anything else I've tested on.

Ferroin on 4 Oct 2018

😕1

@Ferroin did you find any useful?

askd77 on 6 Oct 2018

Unfortunately no.

@paulfantom Any chance you could take a look at this given that you appear to be our resident RHEL/CentOS expert? So far I've had no luck reproducing it on the CentOS VM I set up or on any other system's I've tried with.

Ferroin on 8 Oct 2018

installed the net data from Centos7 package manager.

@askd77 which package? Have you build it yourself or downloaded from somewhere? (we don't ship any prebuilt CentOS7 packages [source]).

paulfantom on 8 Oct 2018

@paulfantom netdata package where installed from package lib of Centos (yum install netdata) and then made configuration as upper comments.

askd77 on 10 Oct 2018

Neither official CentOS repositories nor EPEL ship netdata. Where did you install this package from, did you build it yourself from our repo (if yes, then which git commit revision?) or did you install it from some other repository? I cannot help you without that information as I won't be able to reproduce your error.

paulfantom on 10 Oct 2018

@paulfantom I follow this guide to install all necessary packages. ‘https://www.google.gr/amp/s/www.tecmint.com/monitor-nginx-performance-using-netdata-on-centos-7/amp/‘
For Nginx I didn’t face any problem. But with unbound something is not going well.

askd77 on 11 Oct 2018

That guide doesn't say anything about installing netdata from package (using yum install netdata) and it uses our installation script. Did you configure unbound to expose metrics by following our guide here: https://github.com/netdata/netdata/tree/master/python.d#unbound?

paulfantom on 12 Oct 2018

@paulfantom i made a lot of images and tests to locate the issue. To conclude now on my image i have install the link where i have mention on my previous comment but from debugger receiving the below error:

`/usr/libexec/netdata/plugins.d/python.d.plugin unbound debug trace 1
2018-10-13 17:10:31: python.d INFO: plugin: main: Using python 2
2018-10-13 17:10:31: python.d DEBUG: plugin: main: loading '/etc/netdata/python.d.conf'
2018-10-13 17:10:31: python.d DEBUG: plugin: main: module load source: 'unbound' => [OK]
2018-10-13 17:10:31: python.d DEBUG: plugin: main: loading '/etc/netdata/python.d/unbound.conf'
2018-10-13 17:10:31: python.d DEBUG: unbound: local: No unix socket specified. Trying TCP/IP socket.
2018-10-13 17:10:31: python.d DEBUG: unbound: local: No host specified. Using: "localhost"
2018-10-13 17:10:31: python.d DEBUG: unbound: local: No port specified. Using: "None"
2018-10-13 17:10:31: python.d DEBUG: unbound: local: No valid TLS client certificate configuration found.
2018-10-13 17:10:31: python.d DEBUG: unbound: local: No request specified. Using: "UBCT1 stats
"
2018-10-13 17:10:31: python.d DEBUG: unbound: local: Unbound config: /etc/unbound/unbound.conf
2018-10-13 17:10:31: python.d DEBUG: unbound: local: Extended stats: True
2018-10-13 17:10:31: python.d DEBUG: unbound: local: Per-thread stats: False
2018-10-13 17:10:31: python.d DEBUG: unbound: local: Connecting to: localhost:8953
2018-10-13 17:10:31: python.d DEBUG: unbound: local: Using key: /etc/unbound/unbound_control.key
2018-10-13 17:10:31: python.d DEBUG: unbound: local: Using certificate: /etc/unbound/unbound_control.pem
2018-10-13 17:10:31: python.d DEBUG: plugin: main: job initialization: 'unbound local' => ['OK']
2018-10-13 17:10:31: python.d DEBUG: plugin: main: module status: 'unbound' => [OK] (jobs: 1)
2018-10-13 17:10:31: python.d DEBUG: unbound: local: Creating socket to "::1", port 8953
2018-10-13 17:10:31: python.d DEBUG: unbound: local: Encapsulating socket with TLS
2018-10-13 17:10:31: python.d DEBUG: unbound: local: connecting socket to "::1", port 8953
2018-10-13 17:10:31: python.d ERROR: unbound: local: Failed to connect to "::1", port 8953, error: [Errno 111] Connection refused
2018-10-13 17:10:31: python.d ERROR: unbound: local: Traceback (most recent call last):
File "/usr/libexec/netdata/python.d/python_modules/bases/FrameworkServices/SocketService.py", line 87, in _connect2socket
self._sock.connect(sa)
File "/usr/lib64/python2.7/ssl.py", line 867, in connect
self._real_connect(addr, False)
File "/usr/lib64/python2.7/ssl.py", line 854, in _real_connect
socket.connect(self, addr)
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
error: [Errno 111] Connection refused

2018-10-13 17:10:31: python.d DEBUG: unbound: local: closing socket
2018-10-13 17:10:31: python.d DEBUG: unbound: local: Creating socket to "127.0.0.1", port 8953
2018-10-13 17:10:31: python.d DEBUG: unbound: local: Encapsulating socket with TLS
2018-10-13 17:10:31: python.d DEBUG: unbound: local: connecting socket to "127.0.0.1", port 8953
2018-10-13 17:10:31: python.d ERROR: unbound: local: Failed to connect to "127.0.0.1", port 8953, error: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:579)
2018-10-13 17:10:31: python.d ERROR: unbound: local: Traceback (most recent call last):
File "/usr/libexec/netdata/python.d/python_modules/bases/FrameworkServices/SocketService.py", line 87, in _connect2socket
self._sock.connect(sa)
File "/usr/lib64/python2.7/ssl.py", line 867, in connect
self._real_connect(addr, False)
File "/usr/lib64/python2.7/ssl.py", line 858, in _real_connect
self.do_handshake()
File "/usr/lib64/python2.7/ssl.py", line 831, in do_handshake
self._sslobj.do_handshake()
SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:579)

2018-10-13 17:10:31: python.d DEBUG: unbound: local: closing socket
2018-10-13 17:10:31: python.d INFO: unbound: local: check() => [FAILED]
2018-10-13 17:10:31: python.d INFO: plugin: main: FINISHED
`

askd77 on 13 Oct 2018

What is your python version? If it is python < 2.7.9, then please upgrade at least to 2.7.9

paulfantom on 13 Oct 2018

@paulfantom my version is 2.7.15 but the error its the same.

python2.7 -V Python 2.7.15

askd77 on 14 Oct 2018

@paulfantom please note final run the process successful. Locate the error on do-not-query-localhost: no Please check the below status of unbound

`num.query.flags.TC=0
num.query.flags.RD=15
num.query.flags.RA=0
num.query.flags.Z=0
num.query.flags.AD=0
num.query.flags.CD=0
num.query.edns.present=15
num.query.edns.DO=0
num.answer.rcode.NOERROR=15
num.answer.rcode.FORMERR=0
num.answer.rcode.SERVFAIL=0
num.answer.rcode.NXDOMAIN=0
num.answer.rcode.NOTIMPL=0
num.answer.rcode.REFUSED=0
num.query.ratelimited=0
num.answer.secure=0
num.answer.bogus=0
num.rrset.bogus=0
unwanted.queries=0
unwanted.replies=0
msg.cache.count=56
rrset.cache.count=529
infra.cache.count=90
key.cache.count=0

BEGIN unbound_local.recursion 1000032
SET 'recursive_avg' = 0
SET 'recursive_med' = 0
END

BEGIN unbound_local.reqlist 1000032
SET 'reqlist_avg' = 0
SET 'reqlist_max' = 0
SET 'reqlist_overwritten' = 0
SET 'reqlist_exceeded' = 0
SET 'reqlist_current' = 0
SET 'reqlist_user' = 0
END

BEGIN unbound_local.cache 1000032
SET 'cache_message' = 56
SET 'cache_rrset' = 529
SET 'cache_infra' = 90
SET 'cache_key' = 0
END

BEGIN unbound_local.queries 1000032
SET 'ratelimit' = 0
SET 'cachemiss' = 0
SET 'cachehit' = 15
SET 'expired' = 0
SET 'prefetch' = 0
SET 'recursive' = 0
END
`
Now the only issue where i have is on web interface of netdata. It doesnt show the unbound category

askd77 on 14 Oct 2018

Have you tried refreshing web interface? If a python plugin is sending data you pasted here, then it should work.

paulfantom on 18 Oct 2018

@paulfantom i have refreshed the browser, used different browsers, restart the service but nothing changed. Still category unbound inst appear on web interface

askd77 on 18 Oct 2018

@Ferroin @l2isbad any ideas?

paulfantom on 18 Oct 2018

Not really. Given the fact that the module running by itself behaves fine, I'm inclined to think it's some interaction between different modules, but I'm not really sure.

Ferroin on 18 Oct 2018

@askd77 please restart netdata and post here grep unbound error.log output

ilyam8 on 19 Oct 2018

@l2isbad please check the below screenshot. Follow your suggestion but i didnt receive any error

askd77 on 21 Oct 2018

please do grep python

ilyam8 on 21 Oct 2018

@l2isbad im attaching the txt file with the error codes of grep python
New Text Document.txt

askd77 on 21 Oct 2018

It seems unbound is disabled in `/etc/netdata/python.d.conf'

ilyam8 on 21 Oct 2018

@l2isbad unbound is enables as per instructions

samba: yes
smartd_log: yes
squid: yes
springboot: yes
tomcat: yes
unbound: yes
varnish: yes
web_log: yes
w1sensor: yes

askd77 on 21 Oct 2018

please do
ls -l /etc/netdata/python.d.conf

ilyam8 on 21 Oct 2018

@l2isbad please find the result

ls -l /etc/netdata/python.d.conf

-rw-r----- 1 root root 1831 Oct 20 21:19 /etc/netdata/python.d.conf

askd77 on 21 Oct 2018

I see

2018-10-20 19:20:57: python.d ERROR: plugin: main: cannot load '/etc/netdata/python.d.conf': [Errno 13] Permission denied: '/etc/netdata/python.d.conf'. Will try stock version.

ilyam8 on 21 Oct 2018

@askd77
as you can see conf file is not readable for user netdata

ilyam8 on 21 Oct 2018

sudo chown root:netdata /etc/netdata/python.d.conf

ilyam8 on 21 Oct 2018

@l2isbad
ls -l /etc/netdata/python.d.conf

-rw-r----- 1 root netdata 1831 Oct 20 21:19 /etc/netdata/python.d.conf

still nothing changed, to the category

askd77 on 21 Oct 2018

restart netdata.service
should be fixed after thatr

if not
grep unbound error.log

ilyam8 on 21 Oct 2018

restart the service, but didnt work.
Now it show me 2 error on error.log

`cat /var/log/netdata/error.log | grep unbound

2018-10-21 10:15:00: python.d INFO: unbound: local: check() => [FAILED]
2018-10-21 10:26:04: python.d INFO: unbound: local: check() => [FAILED]
`

askd77 on 21 Oct 2018

sudo su -s /bin/bash netdata
/usr/libexec/netdata/plugins.d/python.d.plugin unbound debug trace 1

ilyam8 on 21 Oct 2018

sudo su -s /bin/bash netdata
bash-4.2$ /usr/libexec/netdata/plugins.d/python.d.plugin unbound debug trace 1
2018-10-21 10:38:34: python.d INFO: plugin: main: Using python 2
2018-10-21 10:38:34: python.d DEBUG: plugin: main: loading '/etc/netdata/python.d.conf'
2018-10-21 10:38:34: python.d DEBUG: plugin: main: module load source: 'unbound' => [OK]
2018-10-21 10:38:34: python.d DEBUG: plugin: main: loading '/etc/netdata/python.d/unbound.conf'
2018-10-21 10:38:35: python.d DEBUG: unbound: local: No unix socket specified. Trying TCP/IP socket.
2018-10-21 10:38:35: python.d DEBUG: unbound: local: No host specified. Using: "localhost"
2018-10-21 10:38:35: python.d DEBUG: unbound: local: No port specified. Using: "None"
2018-10-21 10:38:35: python.d DEBUG: unbound: local: No valid TLS client certificate configuration found.
2018-10-21 10:38:35: python.d DEBUG: unbound: local: No request specified. Using: "UBCT1 stats
"
2018-10-21 10:38:35: python.d DEBUG: unbound: local: Unbound config: /etc/unbound/unbound.conf
2018-10-21 10:38:35: python.d DEBUG: unbound: local: Extended stats: True
2018-10-21 10:38:35: python.d DEBUG: unbound: local: Per-thread stats: False
2018-10-21 10:38:35: python.d DEBUG: unbound: local: Connecting to: localhost:8953
2018-10-21 10:38:35: python.d DEBUG: unbound: local: Using key: /etc/unbound/unbound_control.key
2018-10-21 10:38:35: python.d DEBUG: unbound: local: Using certificate: /etc/unbound/unbound_control.pem
2018-10-21 10:38:35: python.d DEBUG: plugin: main: job initialization: 'unbound local' => ['OK']
2018-10-21 10:38:35: python.d DEBUG: plugin: main: module status: 'unbound' => [OK] (jobs: 1)
2018-10-21 10:38:35: python.d DEBUG: unbound: local: Creating socket to "::1", port 8953
2018-10-21 10:38:35: python.d DEBUG: unbound: local: Encapsulating socket with TLS
2018-10-21 10:38:35: python.d INFO: unbound: local: check() => [FAILED]
2018-10-21 10:38:35: python.d INFO: plugin: main: FINISHED

askd77 on 21 Oct 2018

The module is trying to connect using IPv6, but I think Unbound preferentially uses IPv4 instead of IPv6 if you specify localhost for the control-interface key in it's configuration. Try updating the Unbound configuration 9not the config for the Netdata module, but the config for unbound itself) to explicitly use an IP address instead of localhost. 127.0.0.1 if you want it to use IPv4, or ::1 if you want it to use IPv6.

Ferroin on 22 Oct 2018

@Ferroin nothing changed. I have already try to add both options with "127.0.0.1" and "::1" also all interfaces "0.0.0.0" and "::0".

askd77 on 22 Oct 2018

Just had another thought about this while working on setting up a CentOS VM. If you have the system set up with any of the CentOS security policy options, there's a pretty distinct possibility that that is imposing some restriction which is causing this issue. The system I set up to try and reproduce things on was configured with no security policy, so if the issue is because of a security policy, that would explain why I couldn't reproduce it.

Ferroin on 25 Oct 2018

@Ferroin thank for your help, i have already check this parameter and was disable.

askd77 on 25 Oct 2018

Hi @askd77
Have you solved it?

ilyam8 on 19 Nov 2018

Hello @l2isbad
unfortunately no, the netdata tool it doesnt work with unbound dns

askd77 on 19 Nov 2018

@askd77 :cry:

@Ferroin any ideas how to trace the issue?

ilyam8 on 21 Nov 2018

@l2isbad At this point, I'm thoroughly stumped. I tried again yesterday to reproduce this in a CentOS 7 VM , both with a build of Netdata created from the commit that added Unbound support, and the most recent tree, using both methods of configuring the Unbound control socket (both listening on loopback, and using a UNIX socket), and was unsuccessful again at reproducing it. As far as I can tell, it's _something_ interacting oddly inside python.d.plugin, but it doesn't seem to happen when the plugin is run manually either.

Ferroin on 21 Nov 2018

@askd77

please post here (i know you did it, but please do again and please do new debug)

unbound module configuration file
unbound module debug log with trace

ilyam8 on 21 Nov 2018

@l2isbad
im posting again the debug log of netdata below

bash-4.2$ /usr/libexec/netdata/plugins.d/python.d.plugin unbound debug trace 1
2018-11-24 08:36:57: python.d INFO: plugin: main: Using python 2
2018-11-24 08:36:57: python.d DEBUG: plugin: main: loading '/etc/netdata/python.d.conf'
2018-11-24 08:36:57: python.d DEBUG: plugin: main: module load source: 'unbound' => [OK]
2018-11-24 08:36:57: python.d DEBUG: plugin: main: loading '/etc/netdata/python.d/unbound.conf'
2018-11-24 08:36:57: python.d DEBUG: unbound: local: No unix socket specified. Trying TCP/IP socket.
2018-11-24 08:36:57: python.d DEBUG: unbound: local: No host specified. Using: "localhost"
2018-11-24 08:36:57: python.d DEBUG: unbound: local: No port specified. Using: "None"
2018-11-24 08:36:57: python.d DEBUG: unbound: local: No valid TLS client certificate configuration found.
2018-11-24 08:36:57: python.d DEBUG: unbound: local: No request specified. Using: "UBCT1 stats
"
2018-11-24 08:36:57: python.d DEBUG: unbound: local: Unbound config: /etc/unbound/unbound.conf
2018-11-24 08:36:57: python.d DEBUG: unbound: local: Extended stats: True
2018-11-24 08:36:57: python.d DEBUG: unbound: local: Per-thread stats: True
2018-11-24 08:36:57: python.d DEBUG: unbound: local: Connecting to: localhost:8953
2018-11-24 08:36:57: python.d DEBUG: unbound: local: Using key: /etc/unbound/unbound_control.key
2018-11-24 08:36:57: python.d DEBUG: unbound: local: Using certificate: /etc/unbound/unbound_control.pem
2018-11-24 08:36:57: python.d DEBUG: plugin: main: job initialization: 'unbound local' => ['OK']
2018-11-24 08:36:57: python.d DEBUG: plugin: main: module status: 'unbound' => [OK] (jobs: 1)
2018-11-24 08:36:57: python.d DEBUG: unbound: local: Creating socket to "::1", port 8953
2018-11-24 08:36:57: python.d DEBUG: unbound: local: Encapsulating socket with TLS
2018-11-24 08:36:57: python.d INFO: unbound: local: check() => [FAILED]
2018-11-24 08:36:57: python.d INFO: plugin: main: FINISHED

below is the module conf file

New Text Document (5).txt

and final im attaching below the error log of unbound

cat /var/log/netdata/error.log | grep unbound
2018-11-24 08:42:43: python.d INFO: unbound: local: check() => [FAILED]
2018-11-24 08:46:54: python.d INFO: unbound: local: check() => [FAILED]

askd77 on 24 Nov 2018

Well, i have manage to identified issue.
2018-12-18 18:59:28: python.d DEBUG: unbound: local: Creating socket to "127.0.0.1", port 8953
2018-12-18 18:59:28: python.d DEBUG: unbound: local: Encapsulating socket with TLS
2018-12-18 18:59:28: python.d ERROR: unbound: local: Failed to wrap socket.
2018-12-18 18:59:28: python.d ERROR: unbound: local: Traceback (most recent call last):
File "/usr/libexec/netdata/python.d/python_modules/bases/FrameworkServices/SocketService.py", line 78, in _connect2socket
cert_reqs=ssl.CERT_NONE)
File "/usr/lib64/python3.6/ssl.py", line 1149, in wrap_socket
ciphers=ciphers)
File "/usr/lib64/python3.6/ssl.py", line 747, in __init__
self._context.load_cert_chain(certfile, keyfile)
PermissionError: [Errno 13] Permission denied

2018-12-18 18:59:28: python.d DEBUG: unbound: local: closing socket
2018-12-18 18:59:28: python.d INFO: unbound: local: check() => [FAILED]

Reason for that is, that netdata always use TLS to access control port 8953. However to access that port it needs access to /etc/unbound/unbound_control.* . By default this file has mod 644, so netdata process cant access it - > cant connect to 8953. If i make these files readable for netdata user, it start to work.
Quick fix would be to disable tls. (I havent seen such option in /etc/netdata/python.d/unbound.conf)
Proper fix would be to have rights to access pem and key file.

tomascharvat on 18 Dec 2018

Quick fix would be to disable tls. (I havent seen such option in /etc/netdata/python.d/unbound.conf)

There isn't one because Unbound won't let you use the control protocol over a non-TLS socket unless you're using a UNIX socket.

Proper fix would be to have rights to access pem and key file.

Correct, and this is covered in the documentation for the unbound module in the section on configuration.

Ferroin on 18 Dec 2018

Since the issue is identified and the proper solution is already in the module documentation i think we can close it
@Ferroin

ilyam8 on 18 Dec 2018

I would like to hear back from the OP about whether this solves their issue or not first, but assuming it does, then yes, and I'll probably make a PR to update the documentation to make that section a bit more prominent.

Ferroin on 18 Dec 2018

@askd77 please have a look
https://github.com/netdata/netdata/issues/4350#issuecomment-448315232

ilyam8 on 18 Dec 2018

@Ferroin
Sorry, I'm a bit confused. How can i set the rights for pem and key file on python.d/unbound.conf?

askd77 on 22 Dec 2018

@tomascharvat gave full rights on pem and key file without result. As the local check on port 8953 still doesn't respond.

askd77 on 26 Dec 2018

I was able to get this to work after having the above issues as well.. Where it would run with the module ran manually, then the service would FAIL the check. I had added the "Group" to the unbound_control.key file but not the .pem file. Once I chgrp "groupName" unbound_control.pem : then restarted the netdata service, it checked and started working.. Thanks for all the tidbits here folks!

HammaBamma on 16 Feb 2019

Fixed for me
sudo chmod 644 unbound_control.pem
sudo chmod 644 unbound_control.key

As well as
sudo chown root:netdata unbound_control.pem
sudo chown root:netdata unbound_control.key

Netdata will be able to read and access these 2 files afterwards.

Unbound Unbound now shows op on the dashboard.

Yinchie on 19 Feb 2019

Netdata will be able to read and access these 2 files afterwards.

I see it in the module README

Once you have the Unbound control protocol enabled, you need to make sure that either the certificate and key are readable by Netdata (if you're using the regular control interface)

@Ferroin

i think we should accent it somehow, maybe a separate section with howto and bold font.

ilyam8 on 19 Feb 2019

I think we should accent it somehow, maybe a separate section with howto and bold font.

Agreed, I'll look at getting a PR to do so in the near future.

Ferroin on 19 Feb 2019

🚀1

@Ferroin I would be very grateful if you update the docs, so we can close the issue.

ilyam8 on 28 Feb 2019

@ilyam8 Sorry, this slipped off my radar and I never got back to it. Preparing a PR to update the docs right now.

Ferroin on 28 Feb 2019

I too have this issue under CentOS 7 and the troubleshooting here helped some - netdata couldn't read the key and pem files - so I added netdata to the unbound group, but I still get the check FAILED result:

bash-4.2$ /opt/netdata/netdata-plugins/plugins.d/python.d.plugin unbound debug trace 1
2019-02-28 22:27:23: python.d INFO: plugin: main: Using python 2
2019-02-28 22:27:23: python.d DEBUG: plugin: main: loading '/opt/netdata/etc/netdata/python.d.conf'
2019-02-28 22:27:23: python.d DEBUG: plugin: main: module load source: 'unbound' => [OK]
2019-02-28 22:27:23: python.d DEBUG: plugin: main: loading '/opt/netdata/etc/netdata/python.d/unbound.conf'
2019-02-28 22:27:23: python.d DEBUG: unbound: local: No unix socket specified. Trying TCP/IP socket.
2019-02-28 22:27:23: python.d DEBUG: unbound: local: No valid TLS client certificate configuration found.
2019-02-28 22:27:23: python.d DEBUG: unbound: local: No request specified. Using: "UBCT1 stats
"
2019-02-28 22:27:23: python.d DEBUG: unbound: local: Unbound configuration not found.
2019-02-28 22:27:23: python.d DEBUG: unbound: local: Extended stats: True
2019-02-28 22:27:23: python.d DEBUG: unbound: local: Per-thread stats: True
2019-02-28 22:27:23: python.d DEBUG: unbound: local: Connecting to: localhost4:8953
2019-02-28 22:27:23: python.d DEBUG: unbound: local: Using key: /etc/unbound/unbound_control.key
2019-02-28 22:27:23: python.d DEBUG: unbound: local: Using certificate: /etc/unbound/unbound_control.pem
2019-02-28 22:27:23: python.d DEBUG: plugin: main: job initialization: 'unbound local' => ['OK']
2019-02-28 22:27:23: python.d DEBUG: plugin: main: module status: 'unbound' => [OK] (jobs: 1)
2019-02-28 22:27:23: python.d DEBUG: unbound: local: Creating socket to "127.0.0.1", port 8953
2019-02-28 22:27:23: python.d DEBUG: unbound: local: Encapsulating socket with TLS
2019-02-28 22:27:23: python.d INFO: unbound: local: check() => [FAILED]
2019-02-28 22:27:23: python.d INFO: plugin: main: FINISHED

SELinux is disabled.

my unbound.conf:

local:
#ubconf: /etc/unbound/unb.conf
extended: yes
per_thread: yes
host: localhost4
port: 8953

Same result with explicitly using python3 to invoke the plugin. I even tried running as root Whatever I do, no TCP connection is ever setup - I know this because I was watching for it:

/usr/sbin/tshark -i lo -n port 8953

and unbound is definitely listening:

[root@voyageur python.d]# netstat -plutan | grep 8953
tcp 0 0 127.0.0.1:8953 0.0.0.0:* LISTEN 21321/unbound
tcp6 0 0 ::1:8953 :::* LISTEN 21321/unbound

unbound-control stats_noreset works and shows output - even as the netdata user.

I will look into using a unix socket, but I am at a bit of a loss as to why the plugin isn't working...

stahr on 1 Mar 2019

I can't get a unix socket to work with netdata either. It works fine with the unbound-control utility, but not with python.d/unbound. I think the auto-conf is failing to parse the ubconf: file, however, I can't figure out the syntax to hard code the necessary settings in python.d/unbound.conf.

Despite the claim of the documentation, setting up unbound to utilize a socket was as simple as changing "control-interface" to a file:

control-interface: /etc/unbound/unb.sock

but it seems like this plugin requires additional changes to the configuration file so... Does anyone have a working example?

stahr on 5 Mar 2019

@stahr Unfortunately i can't suggest anything useful apart from

reading trubleshooting section in the module readme file
reading this thread (link, link, link)

i guess you did all that and still have the problem :cry:

@Ferroin

2019-02-28 22:27:23: python.d DEBUG: unbound: local: Creating socket to "127.0.0.1", port 8953
2019-02-28 22:27:23: python.d DEBUG: unbound: local: Encapsulating socket with TLS
2019-02-28 22:27:23: python.d INFO: unbound: local: check() => [FAILED]

I see we don't have enough log info, there is no reason why check is failed.

ilyam8 on 5 Mar 2019

@stahr I've actually been having issues getting it working with a UNIX socket recently myself, I'm not sure if something changed on the Unbound side of things or not. I've been trying to debug, but have thus far had exactly zero success. As far as the config itself for the module, for a UNIX socket it should just be something like this:

local:
    socket: /etc/unbound/unb.sock

That said, I would very much encourage you to use /run or /var/run for the socket, as that will help ensure that it gets cleaned up properly when the system reboots.

@ilyam8 Yeah, logging could definitely be improved,

Ferroin on 6 Mar 2019

I had the same issue, it took me sometime to find the problem.

The key and cert used by the unbound-control aren't accessible from the outside of the root account and group.

To fix it, I make them belong to the group netdata:

chown root:netdata /etc/unbound/unbound_control.*

After that, netdata can access the key and do the request to unbound to gather the data.

Belphemur on 26 Sep 2019

Netdata: Netdata Unbound dns not Working

All 71 comments

Related issues