Goaccess: How to parse Labeled Tab-separated Values (LTSV) logs with GoAccess

[liyuankui]

LTSV is more flexible and becoming trending.
http://ltsv.org/
Is it possible to config goaccess to parse LTSV logs, or simply add it to first class log format for goaccess.

[liyuankui]

E.g., for nginx, the log config as ltsv might look like this

log_format ltsv 'time:$time_local\t'
        'status:$status\t'
        'request_time:$request_time\t'
        'upstream_addr:$upstream_addr\t'
        'upstream_response_time:$upstream_response_time\t'
        'upstream_cache_status:$upstream_cache_status\t'
        'body_bytes_sent:$body_bytes_sent\t'
        'remote_addr:$remote_addr\t'
        'host:$host\t'
        'request_method:$request_method\t'
        'request_uri:$request_uri\t'
        'protocol:$server_protocol\t'
        'http_referer:$http_referer\t'
        'http_user_agent:$http_user_agent';

[allinurl]

It should be able to parse it, can you please post a few lines from your access log to make sure goaccess get them right? Thanks.

[liyuankui]

Thanks for renaming.
For below conf:

    log_format ltsv 'time:$time_iso8601\t'
                  'hostname:$hostname\t'
                  'remote_addr:$remote_addr\t'
                  'request_method:$request_method\t'
                  'request_length:$request_length\t'
                  'request_uri:$request_uri\t'
                  'https:$https\t'
                  'uri:$uri\t'
                  'query_string:$query_string\t'
                  'status:$status\t'
                  'bytes_sent:$bytes_sent\t'
                  'body_bytes_sent:$body_bytes_sent\t'
                  'referer:$http_referer\t'
                  'useragent:$http_user_agent\t'
                  'forwardedfor:$http_x_forwarded_for\t'
                  'request_time:$request_time\t'
                  'upstream_response_time:$upstream_response_time';

The log looks like this:

time:2017-08-15T06:51:37+00:00  hostname:ip-192-168-50-0    remote_addr:192.168.192.165  request_method:POST    request_length:817  request_uri:/sample/websocket/539/eszhwtay/xhr?t=1502779891835   https:on   uri:/sample/websocket/539/eszhwtay/xhr  query_string:t=1502779891835    status:200   bytes_sent:699 body_bytes_sent:244 referer:https://www.example.com/sample/ useragent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36     forwardedfor:221.249.116.206   request_time:3.649  upstream_response_time:3.649

[allinurl]

This should do it:

goaccess access.log --log-format='%^:%dT%t+%^\t%^\t%^:%h\t%^:%m\t%^\t%^:%U\t%^\t%^\t%^\t%^:%s\t%^:%b\t%^\t%^:%R\t%^:%u\t%^\t%^:%T\t%^' --time-format=%T --date-format=%Y-%m-%d

[liyuankui]

Wow, it works, thanks a lot!