Goaccess: Add option for on-disk database size limit

[Dhuran]

Hello!

We are using configuration with database, load-from-disk and keep-db-files. In some cases database size can grow up to 8G or more. Is there any possibility to set maximum database size? So we can keep disk usage under control without removing goaccess database files and losing all of previously stored data. I could not find any options similar to "size limit", "database size" or "expires" in the official man page.

Thank you!

[allinurl]

Should this option prevent goaccess to keep writing to it? or should it just stop processing data and quit? I just wonder how should goaccess behave once it has reached its storage limit.

Thanks

[Dhuran]

Thank you for fast reply!
The most perfect way is to delete data older then X days and keep writing new data to disk. Is it possible ?

[allinurl]

Not sure if it would be possible since currently it doesn't timestamp every record saved in the DB. Have you tried compressing the data using --compression=<zlib|bz2>? should help though.

[SeLLeRoNe]

As far as i read there will be the check for already imported data with a timetamp, so i guess that once that function is implemented would be also possible to remove old data.

In that case would be also nice to have an option to set how back in the past the log should be analyzed during the log parsing.

[n0guest]

Not sure if it would be possible since currently it doesn't timestamp every record saved in the DB. Have you tried compressing the data using --compression=? should help though.

It will reduce summary size of DB files, but it isn't a solution for initial problem. I.e. even if compression is enabled, DB will grow over time.

Should this option prevent goaccess to keep writing to it? or should it just stop processing data and quit? I just wonder how should goaccess behave once it has reached its storage limit.

I vote also for scenario "delete data older then N days". It would be nice to have such feature in goaccess.

[allinurl]

Actually there are some plans to refactor the on-disk storage so I'll look into this and add it as one of the new features. Thanks.

[nils-schween]

Hi there,
I just started using goaccess and it is a really nice program. Unfortunately I ran into the same problem as the original issue filer. I checked the changelogs of the last two releases but I couldn't find a feature "delete data older than N days". @allinurl Did you already add it and I missed it? Thank you for all your work!

Nils

[allinurl]

@nils-schween I haven't as I plan to replace the existing on-disk storage with something more robust. Hoping to add this when the storage engine is replaced. Thanks for the reminder.

[nils-schween]

Thanks for the answer. Then, I'll just wait and check regularly for new versions.