Collectd: ntpd plugin: private mode (7) is now deprecated

[shastah]

After upgrading my ntpd from 4.2.6p5 to 4.2.8, ntpd plugin stopped collecting any data:

Feb  7 16:58:44 hostname collectd[11344]: ntpd plugin: ntpd_do_query returned unexpected data. (ik = (nil); ik_num = 0; ik_size = 0)
Feb  7 16:58:44 hostname collectd[11344]: read-function of plugin `ntpd' failed. Will suspend it for 20.000 seconds.

Turns out ntpdc (which collectd mimics on the wire) is now deprecated, and "mode7" must be explicitly enabled in ntp.conf to keep it working:

  mode7  Enables  processing  of NTP mode 7 implementation-specific requests
         which are used by the deprecated ntpdc(1) program. The default for this flag
         is disable.  This flag is excluded from runtime configuration using ntpq(1).
         The ntpq(1) program provides the same capabilities as ntpdc(1) using standard 
         mode 6 requests.

It would probably be the best to rework collectd ntpd plugin to talk to ntpd using mode 6, like ntpq, but if that proves tricky and/or time consuming, at least it's worth mentioning the issue in ntpd plugin documentation, I think.

[jettero]

It took me quite a while to figure out how to enable this and what the implications might be. I'm still not totally sure about the implications, but I wanted to post a bunch of URLs here for the next chump in danger of spending an hour trying to figure out what's going on.

From my ntp.conf —

# see http://doc.ntp.org/4.2.8p5/miscopt.html
# and see http://questions.ntp.narkive.com/ebjq59Qq/ntp-questions-ntpdc-is-not-working-in-ntp-4-2-8p1
# and also http://support.ntp.org/bin/view/Dev/DeprecatingNtpdate
# and lastly: https://github.com/collectd/collectd/issues/932
# (we need this depricated bit for collectd ntpd stats)
enable mode7

[matwey]

@shastah what is the difference between mode 7 and mode 6? How can collectd be reimplement to mimic ntpq behaviour?

[shastah]

@matwey mode 6 is described in RFC 1305 as "reserved for NTP control messages", whereas mode 7 is "reserved for private use", so I guess might be implementation-specific. The most striking difference between them in terms of parsing responses is that mode 6 is supposed to be human-readable, so it's mostly plaintext.

The difference on the wire can be seen by snooping NTP traffic while doing ntpq -c pe compared to ntpdc -p (the latter uses mode 7 packets).

Appendix B of RFC 1305 gives an overview of mode 6.

[jcberthon]

I stumble on this problem too. And it took me several days to figure out why I could not monitor ntpd when run in an container (with collectd on the host), until I realise that the container had a newer version of ntpd which does not support mode 7 anymore. I was very puzzle that ntpdc -c kerninfo would work on the host but not on the container, but that is simply due because ntp.org has deprecated it and the mode it uses. See also this man page for the current stable of ntpdc: http://doc.ntp.org/current-stable/ntpdc.html where it is stated that this command is deprecated in favour of ntpq.

If someone run Ubuntu 16.04, they will stumble on the problem too. Version 14.04, Debian Jessie or CentOS 7 all have older version of ntpd still compatible with mode 7.

What kind of feedback is pending?

[Whissi]

@jcberthon: Are you sure you cannot re-enable _mode 7_ anymore? See comment https://github.com/collectd/collectd/issues/932#issuecomment-211453410.

[jcberthon]

@Whissi my comment was misleading sorry. Yes it is still possible to re-enable it by adding the config enable mode7, and it solves the problem for now. However, mode7 was disabled in order to prevent DDoS amplification attacks (see post by Miroslav here) which sounds like a valid point to deprecate this mode :-).

But:

1. because everyone migrating to Ubuntu 16.04 or newer distro with ntp 4.2.8 will encounter the problem, it should be documented somewhere that mode7 should be re-enabled (maybe in the wiki on the plugin?).
2. because mode7 is marked deprecated and disabled by default in 4.2.8, it probably will end up removed one day. So shouldn't this be anticipated and the plugin migrated to use mode6 instead of mode7 so that when either mode7 is disabled or completely removed, the plugin will still work?

I'm not criticizing here :-) I'm just humbly trying to contribute.

[Whissi]

mode 7 is only problematic because it brings the MON_GETLIST command which results in a large answer which can be abused in a DDoS. However it was never best practice to open a public NTP server that way for everyone. So if you have followed best practice recommendation your restrict line would prevent that your NTP server could be abused.

So there's nothing wrong with enabling mode 7 again if you do it right.

Any new way using a different method must be implemented first. I guess contributions are welcome however because we have a working way (which isn't bad if you do it right!) nobody really cares.

[jcberthon]

I agree with you, and most distro I've used set ntp properly in this regard for quite some time now.

I have implemented this workaround, so as you say I have now a working way. This is understood that for now nobody needs to care, but I don't think this issue should be _lost_ in the "pending feedback" state. Unless I'm misinterpreting the tag.

[shastah]

@jcberthon IMHO if Ubuntu doesn't include manual pages relevant to the software they ship, it's a bug that should be reported to Ubuntu (not saying collectd's wiki should not have that information)

[octo]

Discussed this in the community call today. Would be nice to fix. Needs investigation into the NTP protocol specifics. Not high priority since a workaround exists.

[mnlipp]

Just a note for someone "passing by": Fedora has moved to NTPsec (already starting in fc33, but in fc34 for sure) which does not support mode7. Kind of rises the priority...