Goaccess: Date format issue on Cygwin and different locale

[ManosZoumpoulakis]

Built from development
Token '16/Jun/2017' doesn't match specifier '%d'

My log file:
10.82.136.5 - - [29/Jun/2017:20:48:07 +0300] "GET / HTTP/1.1" 200 17507

I am running Apache server on Windows (WAMP) and I have selected NCSA Combined Log Format.

[allinurl]

Please select Common Log instead

[ManosZoumpoulakis]

The same issue with CLF.

[allinurl]

Which version are you running? If I copy the line you posted above and run goaccess v1.2 selecting CLF, I get this:

[ManosZoumpoulakis]

It's version 1.2 and I used "Build from GitHub (Development)"

[allinurl]

Are you running goaccess through Cygwin? Also, would you be able to attach a file with a few lines from your access.log? Thanks

[ManosZoumpoulakis]

Yes, I use Cygwin on Windows 7 64bit.
apache_log.txt

[allinurl]

I cloned the repo and ran it through Cygwin with the log attached and went fine. Can you please try running the following command against the log you just posted?

$ ./goaccess.exe apache_log.txt --log-format=COMMON

[ManosZoumpoulakis]

$ goaccess.exe C:/apache_log.txt --log-format=COMMON
Parsed 9 linesproducing the following errors:

Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'

Format Errors - Verify your log/date/time format

`

[allinurl]

I'm guessing you have a config file overwriting this, please run:

$ goaccess.exe C:/apache_log.txt --log-format=COMMON --no-global-config

[ManosZoumpoulakis]

$ goaccess.exe C:/apache_log.txt --log-format=COMMON --no-global-config
Parsed 9 linesproducing the following errors:

Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'

Format Errors - Verify your log/date/time format

`

[allinurl]

This appears to be specific to your instance. Could you try this on a different machine? You can also try editing your config file to see if that helps. You can find the config file through goaccess --dcf.

[ManosZoumpoulakis]

I re-installed Cygwin on the same machine but on another path, nothing changes. I have no other server available to test right now. Could you please point me on a link so I am sure that I follow the right steps to install the right packets and give the right commands? Really thank you for your time!

Also, I forgot to mention that on command "autoreconf -fi" I had to manually install Gettext

[allinurl]

From the error you are getting, sounds like it's using a different log-format, perhaps from a config file. Can you please run the following test cases and post the outputs here:

goaccess.exe apache_log.txt -o report.html

and

goaccess.exe apache_log.txt -o report.html --log-format=COMMON

and

goaccess.exe --dcf

and please create two test conf files with these lines on it and point it to goaccess

#1.conf
time-format %H:%M:%S
date-format %d/%b/%Y
log-format %v:%^ %h %^[%d:%t %^] "%r" %s %b "%R" "%u"

and run it as

goaccess.exe apache_log.txt -o report.html -p 1.conf

and then

#2.conf
time-format %H:%M:%S
date-format %d/%b/%Y
log-format %h %^[%d:%t %^] "%r" %s %b

and run it as

goaccess.exe apache_log.txt -o report.html -p 2.conf

[allinurl]

Any updates from the previous post?

[ManosZoumpoulakis]

Sorry, I missed the last e-mail.

$ goaccess C:/apache_txt.log -o report.html
Parsing... [0] [0/s]
GoAccess - version 1.2 - Jun 30 2017 00:35:37
Config file: /usr/local/etc/goaccess.conf

Fatal error has occurred
Error occurred at: src/parser.c - parse_log - 2732
No time format was found on your conf file.

$ goaccess.exe C:/apache_log.txt -o report.html --log-format=COMMON
C:/apache_log.txt/s]
Parsed 9 linesproducing the following errors:

Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'

Format Errors - Verify your log/date/time format

$ goaccess.exe --dcf
/usr/local/etc/goaccess.conf

$ goaccess.exe C:/apache_log.txt -o report.html -p C:/1.conf
C:/apache_log.txt/s]
Parsed 9 linesproducing the following errors:

Token '+0300]' doesn't match specifier '%h'
Token '+0300]' doesn't match specifier '%h'
Token '+0300]' doesn't match specifier '%h'
Token '+0300]' doesn't match specifier '%h'
Token '+0300]' doesn't match specifier '%h'
Token '+0300]' doesn't match specifier '%h'
Token '+0300]' doesn't match specifier '%h'
Token '+0300]' doesn't match specifier '%h'
Token '+0300]' doesn't match specifier '%h'

Format Errors - Verify your log/date/time format

$ goaccess.exe C:/apache_log.txt -o report.html -p C:/2.conf
C:/apache_log.txt/s]
Parsed 9 linesproducing the following errors:

Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'
Token '16/Jun/2017' doesn't match specifier '%d'

Format Errors - Verify your log/date/time format

[allinurl]

Okay, so this is a locale issue between your access log/date format and your system current locale. Since your log appears to record the month in English (i.e., Jun), then please try setting this on your terminal and then run goaccess:

export LANG='en_US.UTF-8'

Otherwise, you will need to change your web server log/date format to use the current locale.

[allinurl]

Closing this. Feel free to reopen it as needed.

[rinopo]

Hi,
I've encountered the same issue on a Mac terminal where the LANG is set to ja_JP.UTF-8.
(So the issue is not limited to cygwin.)
I'm using goaccess 1.3 and I believe the problem didn't occur with 1.2.

The above workaround (export LANG='en_US.UTF-8') worked for me too, but that does affect other things like the language of the output report, and thus virtually eliminates the meaning of the L10N efforts of this project.

Is it really desirable for goaccess to take the LANG environment variable into consideration when parsing log dates? (I personally have never seen an apache log with localized dates.)

[allinurl]

@rinopo I think the issue here is strftime(3) and strptime(3). They would need to be updated to use the given locale. Maybe we can allow the user pass an option for time-locale, e.g., --date-locale=en_US.UTF-8

A possible workaround to avoid messing up with LANG, you could try (haven't tested):

export LC_TIME='en_US.UTF-8'

[rinopo]

@allinurl
Thank you for the clarification and the tip!

Indeed export LC_TIME='en_US.UTF-8' works as expected!
(No error on parsing logs, with the output report in Japanese.)