goaccess custom log format

Created on 5 Jan 2018  Â·  24Comments  Â·  Source: allinurl/goaccess

Hi all,

we are using the apache webserver with below log format.

LogFormat "%h %{True-Client-IP}i %l %u %t \"%m %{REALREQ}e %H\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{Cookie}i\" \"%{cookie}n\" %{UNIQUE_ID}e \"%r\" %X %D" Custom

when we try accessing the monitoring of Logs via Goaccess Log monitoring tool, we are unable to get the true-client IP ,

can you please help how to create the custom log format for goaccess for accessing the access logs of apache webserver ?

Thanks in advance,

lodattime format question
Source
picture kiranpkosti

Most helpful comment

It works with this format.
Thank you so much.

picture zoso65 on 8 Jan 2018
👍2

All 24 comments

Can you please post a few lines from your access log so I can take a look? Thanks

allinurl picture allinurl on 5 Jan 2018 
"91.98.215.125" - - [31/Dec/2017:09:04:20 +0330] "GET /templates/template_fa/assets/images/my-logo.png HTTP/1.1" 200 3775 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:20 +0330] "GET /statics/images/category/1511265302._ID4_4.gardeshgari,mohajerat.jpg HTTP/1.1" 200 103022 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:20 +0330] "GET /statics/images/category/1511253218._ID1_1.ghaza.jpg HTTP/1.1" 200 25196 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:20 +0330] "GET /statics/images/category/1511264942._ID1_2.jpg HTTP/1.1" 200 17283 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:21 +0330] "GET /statics/images/category/1511264727._ID1_2.jpg HTTP/1.1" 200 21486 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:21 +0330] "GET /statics/images/category/1511254176._ID3_2.ab,bargh.jpg HTTP/1.1" 200 51914 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:21 +0330] "GET /statics/images/category/1511263242._ID1_1.amlak,sakhto%20saz.jpg HTTP/1.1" 200 19086 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:21 +0330] "GET /statics/images/category/1514634444._ID1_1.amniyati.png HTTP/1.1" 200 110592 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:21 +0330] "GET /statics/images/category/1511263771._ID1_2.jpg HTTP/1.1" 200 22912 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:21 +0330] "GET /statics/images/category/1511254024._ID2_3.jpg HTTP/1.1" 200 29463 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:21 +0330] "GET /statics/images/category/1511263919._ID1_2.jpg HTTP/1.1" 200 20327 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:21 +0330] "GET /statics/images/category/1511253779._ID1_1.madan.jpg HTTP/1.1" 200 20545 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:21 +0330] "GET /templates/template_fa/assets/images/favicon.png HTTP/1.1" 200 1678 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:22 +0330] "GET /statics/images/article/90.90.16291361_tmp.png HTTP/1.1" 200 9466 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:22 +0330] "GET /statics/images/article/90.90.200759327_tmp.png HTTP/1.1" 200 8856 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:23 +0330] "GET /templates/template_fa/bower_components/font-awesome/fonts/fontawesome-webfont.woff2 HTTP/1.1" 200 77461 "http://mydomain.com/templates/template_fa/bower_components/font-awesome/css/font-awesome.min.css" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:29 +0330] "GET /index/event/ HTTP/1.1" 200 6215 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:42 +0330] "GET /aboutus HTTP/1.1" 200 5211 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"91.98.215.125" - - [31/Dec/2017:09:04:43 +0330] "GET /templates/template_fa/assets/css/style.css?v=84588 HTTP/1.1" 200 37424 "http://mydomain.com/aboutus" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"
"77.68.40.37" - - [06/Jan/2018:10:08:41 +0330] "GET /templates/template_fa/assets/css/jquery.mmenu.all.css HTTP/1.1" 200 6399 "http://mydomain.com/company/Detail/21981/%D8%AA%D9%88%D9%84%DB%8C%D8%AF%DB%8C%20%D8%B5%D9%86%D8%B9%D8%AA%DB%8C%20%D8%B2%D8%B1%DB%8C%D9%86%20%D8%AE%D9%88%D8%B4%D9%87%20%DA%A9%D9%88%D8%AB%D8%B1" "Mozilla/5.0 (Linux; Android 5.1.1; SAMSUNG SM-J500H Build/LMY48B) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/6.2 Chrome/56.0.2924.87 Mobile Safari/537.36"
"77.68.40.37" - - [06/Jan/2018:10:08:41 +0330] "GET /templates/template_fa/assets/css/normalize.css HTTP/1.1" 200 1181 "http://mydomain.com/company/Detail/21981/%D8%AA%D9%88%D9%84%DB%8C%D8%AF%DB%8C%20%D8%B5%D9%86%D8%B9%D8%AA%DB%8C%20%D8%B2%D8%B1%DB%8C%D9%86%20%D8%AE%D9%88%D8%B4%D9%87%20%DA%A9%D9%88%D8%AB%D8%B1" "Mozilla/5.0 (Linux; Android 5.1.1; SAMSUNG SM-J500H Build/LMY48B) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/6.2 Chrome/56.0.2924.87 Mobile Safari/537.36"
"77.68.40.37" - - [06/Jan/2018:10:08:42 +0330] "GET /templates/template_fa/assets/css/slick-theme.css HTTP/1.1" 200 1039 "http://mydoman.com/company/Detail/21981/%D8%AA%D9%88%D9%84%DB%8C%D8%AF%DB%8C%20%D8%B5%D9%86%D8%B9%D8%AA%DB%8C%20%D8%B2%D8%B1%DB%8C%D9%86%20%D8%AE%D9%88%D8%B4%D9%87%20%DA%A9%D9%88%D8%AB%D8%B1" "Mozilla/5.0 (Linux; Android 5.1.1; SAMSUNG SM-J500H Build/LMY48B) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/6.2 Chrome/56.0.2924.87 Mobile Safari/537.36"
zoso65 picture zoso65 on 6 Jan 2018

This is my accesslog sample.
Thanks

zoso65 picture zoso65 on 6 Jan 2018

Please try:

goaccess access.log --log-format='"%h" %^[%d:%t %^] "%r" %s %b "%R" "%u"' --date-format=%d/%b/%Y --time-format=%T
allinurl picture allinurl on 8 Jan 2018

It works with this format.
Thank you so much.

zoso65 picture zoso65 on 8 Jan 2018
👍2

I , have problem with my log :

Dec 18 10:06:14 apihost01 domain-ext x.x.x. via x.x.x.x- 18/Dec/2018:10:06:05 +0100 GET 200 /app.php /api/report/campaigns/354/summary.json /var/www/apiv1/web/app.php 0 cpu:6.82 allocatedmem:4

houssem-alt picture houssem-alt on 12 Dec 2019

@houssem-alt please describe each field from your access.log. Thanks

allinurl picture allinurl on 12 Dec 2019

Dec 18 10:06:14 apihost01 domain.com-ext x.x.x.x via x.x.x.x -
18/Dec/2018:10:06:05 +0100 GET 200 /app.php
/api/report/campaigns/354/summary.json /var/www/apiv1/web/app.php 0
cpu:6.82 allocatedmem:4

On 12/12/2019 16:55, Gerardo O. wrote:
>

@houssem-alt https://github.com/houssem-alt please describe each
field from your access.log. Thanks

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/allinurl/goaccess/issues/985?email_source=notifications&email_token=AOAV3QPJNGN6TO742WBDUEDQYJNHFA5CNFSM4EKQFAFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGXDYRA#issuecomment-565066820,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOAV3QLOQYJ445DJ245BPFLQYJNHFANCNFSM4EKQFAFA.

houssem-alt picture houssem-alt on 12 Dec 2019

@houssem-alt as I mentioned above, please describe what each field means.

allinurl picture allinurl on 12 Dec 2019
👍1

Dec 18 10:06:14 apihost01 dialog.apis.edatis.com-ext 158.255.69.87 via 83.136.214.164 - 18/Dec/2018:10:06:05 +0100 GET 200 /app.php /api/report/campaigns/354/summary.json /var/www/apiv1/web/app.php 0 cpu:6.82 allocatedmem:4

I try this command :+1:
goaccess access.log --log-format=COMBINED -a -o acces.html

Parsed 1 linesproducing the following errors:

Token 'Dec' doesn't match specifier '%h'

Format Errors - Verify your log/date/time format
Thanks

houssem-alt picture houssem-alt on 12 Dec 2019

@houssem-alt To help you out, I'd need to know what the following fields are:

/app.php /api/report/campaigns/354/summary.json /var/www/apiv1/web/app.php 0 cpu:6.82 allocatedmem:4
allinurl picture allinurl on 12 Dec 2019

in fact these are centralized logs with rsyslog

houssem-alt picture houssem-alt on 12 Dec 2019

This should do it:

goaccess access.log --log-format='%^ %^ %^ %^ %v %h %^ %^ %^ %d:%t %^ %m %s %U %R %^ %L %^' --date-format=%d/%b/%Y --time-format=%T --ignore-panel=OS --ignore-panel=BROWSERS
allinurl picture allinurl on 12 Dec 2019

@allinurl thank you very much sir .

houssem-alt picture houssem-alt on 13 Dec 2019

Hello again, please,
I still have problems with these two log files I need your help Thanks to you cordially

Feb  1 11:09:00 webhost07 houssem.v7.digi.com: [Thu Feb 01 11:09:00.756041 2018] [:error] [pid 29413] [client 83.136.214.179:48578] PHP Notice:  Trying to get property of non-

object in /var/www/V7/src/Dialog/Extension/Twig/DialogExtension.php on line 302

Feb  1 09:49:15 webhost07 houssem.v7.digi.com: 37.71.178.6 - - [01/Feb/2018:09:49:14 +0100] "GET /dialog/reporting/parameter HTTP/1.0" 200 8738 "https://houssem.v7.digi.com/d

ialog/reporting/emailing/home" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"

houssem-alt picture houssem-alt on 13 Dec 2019

@houssem-alt you can't parse error logs.

allinurl picture allinurl on 13 Dec 2019

@allinurl , Ok ,
and for this log ,
Feb 1 09:49:15 webhost07 houssem.v7.digi.com: 37.71.178.6 - - [01/Feb/2018:09:49:14 +0100] "GET /dialog/reporting/parameter HTTP/1.0" 200 8738 "https://houssem.v7.digi.com/d

houssem-alt picture houssem-alt on 13 Dec 2019

@houssem-alt seems it's missing the end part?

allinurl picture allinurl on 13 Dec 2019

This my log , :

Feb  1 09:49:58 webhost07 dialog.v7.houssem.com: 196.203.219.155 - - [01/Feb/2018:09:49:58 +0100] "GET /dialog/campaign HTTP/1.0" 200 27570 "https://1042066294.v7.houssem.com/dialog/reporting/emailing/home" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0"
houssem-alt picture houssem-alt on 13 Dec 2019

@houssem-alt 

goaccess log --log-format='%^ %^ %^ %^ %^ %v %h %^[%d:%t %^] "%r" %s %b "%R" "%u"' --date-format=%d/%b/%Y --time-format=%T
allinurl picture allinurl on 13 Dec 2019

@allinurl thank you in advance sir :+1:

houssem-alt picture houssem-alt on 13 Dec 2019

@allinurl Hello sir ,
How to monitor multiple websites with 'goaccess’？ , I tested to execute two log in parallel with the command --real-time I got an error as preview below
Config file: /etc/goaccess.conf

Fatal error has occurred
Error occurred at: src/websocket.c - ws_socket - 2782
Unable to set bind: Address already in use.

houssem-alt picture houssem-alt on 18 Dec 2019

@allinurl , it's good problem solved, thanks in advance :100:

houssem-alt picture houssem-alt on 18 Dec 2019

@houssem-alt https://github.com/allinurl/goaccess/issues/625#issuecomment-272905761

allinurl picture allinurl on 18 Dec 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

SergioDG-YCC picture SergioDG-YCC  Â·  3Comments
konungrl picture konungrl  Â·  3Comments
deosha picture deosha  Â·  3Comments
Tassleh0ff picture Tassleh0ff  Â·  4Comments
Gill-Bates picture Gill-Bates  Â·  3Comments