Loki: Add config reload endoint / signal to promtail

[tomwilkie]

So we don't have to restart all the pods just to change the scrape config etc.

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[hairyhenderson]

Seems like there's a PR for this pending - would be nice to see this happen instead of the issue going stale and disappearing 😉

[cyriltovena]

@hairyhenderson how this would be useful for you ? Can you give us more informations ?

[hairyhenderson]

@cyriltovena like the description says, it's annoying to restart processes to reload configs. I'm used to being able to send a signal or call a reload endpoint in Prometheus, and this is one area where Loki _isn't_ "Like Prometheus, but for logs" 😉

[cyriltovena]

yeah but Prometheus could miss some scrape and it makes sense to not restart to avoid this kind of behaviour, promtail though won't miss anything as everything is on disk.

[hairyhenderson]

Sure... But there are side-effects of processes terminating which may be undesirable. And, it's conventional for processes in UNIX to listen to a SIGHUP and reload config based on that.

I suppose my question is: why would you _not_ want this feature?

[pastatopf]

Upvote fromy my side :-)

[Perry-The-Cynic]

It's no longer true that (from promtail's perspective) "everything is on disk." The syslog endpoint has data pushed onto promtail, and depending on the retry policy of the sender restarting promtail may drop some messages.

[cyriltovena]

It's no longer true that (from promtail's perspective) "everything is on disk." The syslog endpoint has data pushed onto promtail, and depending on the retry policy of the sender restarting promtail may drop some messages.

Very good point. Although that seems easier to fix than implementing a reload.