Tailscale: Excessive tailscale logging in system log

[danderson]

Report from a friend, as well as a couple of users: tailscaled is the single most verbose thing in their system logs, by a very long way. This is a problem for them because they do centralized logs collection, and so extremely verbose tailscaled logs multiplied by many machines depletes the central logstore's disk space at alarming rates.

We've heard this from several quarters, and our general plan for this is "do less stuff, because this will generate less logs". Filing this bug (a) to track that and (b) to decide whether that's enough. Even with things like eager handshaking disabled, we're still likely to log a _lot_ more than the rest of the system combined, because we intend these logs to be debug-level stuff.

Unfortunately we don't use log levels, so it's not a simple matter of "lower the syslog spamminess and only push debug logs into logtail". Unless we add some kind of log levels, our only option is to turn off syslog output entirely.

┆Issue is synchronized with this Asana task by Unito

[bradfitz]

FWIW, I submitted https://github.com/tailscale/tailscale/commit/73249f8032c3553a914785f0c76294807a0d4bd3 yesterday which addresses a lot of the log spam.

Turning off automatic handshakes will do a lot too.

But, yes, our log spamminess is one of my pet peeves too.

[wogong]

Really hope there is an option to turn off these logs.

[wogong]

I open a window in tmux showing real-time logs, sudo journalctl -f.
tailscale output too many logs to journal making it meaningless.
I found no option in journalctl to filter out logs of certain service, but you can achieve the same by
sudo journalctl -f | grep -v 'tailscaled'