heartbeat_timeout_secs is known via the system table cluster_config as documented here: https://rethinkdb.com/docs/system-tables/#clusterconfig
But I cannot find any info on what the heartbeat interval is. What I mean is that the servers must be sending raft heartbeats to each other at regular intervals but what is that interval? It doesn't seem to be documented anywhere. Is it 1 second? 2 seconds?
cc @VeXocide I brought up this question on Slack a couple of days ago but I figured I'd just open an issue for it because it's difficult to sync up on Slack sometimes and having a GitHub issue for it at least makes a permanent record of the question for other people to find.
I found evidence in some old code that the heartbeat interval is 2 seconds. See here: https://github.com/rethinkdb/rethinkdb/commit/705b238802984ca6cc630866cced1e5c305412a4#diff-05a5f08e37ed1d80a7d85a6846512320L403
That deleted line of code mentions a HEARTBEAT_INTERVAL_MS constant whose value was 2000 (milliseconds I assume). But that constant no longer exists and I don't know that part of the codebase well enough to understand what's going on.
It looks like the heartbeat interval is 1/5 of whatever the heartbeat timeout is. It looks like the default value of the heartbeat timeout is 10 seconds (see src/clustering/administration/metadata.hpp -- you've got to chase down some other code to see that's 10000 milliseconds). So if your heartbeat timeout is still 10 seconds, then your heartbeat interval is 2 seconds.
@srh thanks! Do you know if there's a way that I can contribute a change to the documentation to clarify this?
The documentation is at https://github.com/rethinkdb/docs -- you might make a pull request there.
My guess is, it'll take a while for the change to make it onto the website.