Goaccess: Display stats from an unclosed STDIN

[tomzx]

Hello,

Apparently goaccess already supports displaying stats from cat/tail and if you pass in a file, it will update the stats as new lines are added to the file.

Currently however it is not possible to do something such as tail -f | goaccess -a -p goaccess.conf as it will stay in parsing mode indefinitely.

At this point I'd like to ask, since you already support live updating of stats, is the reason that stats are not displayed as parsing is done is that it is a lot less efficient (a lot less lines parsed/second)? If it has little impact, it would be nice to see the stats displayed as parsing occurs.

Second, it would be nice to support live stats through something such as tail -f | goaccess -a -p goaccess.conf, which would allow users to quickly diagnose issues by monitoring logs since goaccess was started.

[allinurl]

Hello,

Good question, currently the live or tail implementation makes use of fseeko to change the file position and achieve a 'follow mode'. This was shipped in one of the early version of goaccess. As you said, being able to read directly from stdin would be great, especially for the use-case you mentioned above.

I'll go back and take a look at this again to see what can be done and how feasible it is. I appreciate you bringing this up.

[allinurl]

I've pushed a commit that enables the ability to output live stats from a stream or an unclosed STDIN, i.e., tail -f. This allows to do:

tail -f access.log | goaccess --log-format=COMBINED -

or even

tail -f access.log | goaccess --log-format=COMBINED -o report.html --real-time-html -

It also opens the possibility for live data filtering from the pipe such as:

tail -f access.log | grep -i --line-buffered 'firefox' | goaccess --log-format=COMBINED -

One thing to note is that tail -f will keep the pipe opened even when goaccess has already exited. For instance, tail -f syslog | grep -q 'cron' tail will exit on SIGPIPE, however it will only get SIGPIPE until an extra byte is written to the tail'd file. SIGTERM, SIGINT should close tail fine though.

Feel free to build from master to test this out, otherwise it will be pushed out in the upcoming release. Thanks.