Freshrss: Log files become to big!

[Wanabo]

We need to rotate some log files for example:
data\users\currentuser\log.txt

File was over 300 Mb's.

[Frenzie]

I'd suggest a default value of something like 500 KiB. That'd give you a few thousand lines.

[aledeg]

@Wanabo You're right! something has to be done for the logs. But I am not sure FreshRss should handle that. This is not its job. This is a job for logrotate or other programs. For me, this issue should be marked as "Won't fix". My 2¢.

[Frenzie]

Something about that doesn't sit right with me given how https://github.com/FreshRSS/FreshRSS/blob/17c8c039df675b3b0f8d88d14f7316a240eabe76/app/Models/LogDAO.php#L6-L17 is set up. I think that a rejection of the proposed solution should imply something other than won't fix because I for one had no idea I had a 30 MB log file forming in a "random" location.

[aledeg]

@Frenzie I didn't know how the logs were read and used. You're right, there's something fishy here. We need to fix that. But I maintain that we shouldn't have a log rotation system in FreshRss nor a limit on log size.
What we should have is a proper way of handling the log file.

[Frenzie]

I'd have to know more about the precise purpose and contents of this error log to fully form an opinion on that. I've got the impression that it's just a service to users so they can see what went wrong with a feed or directly pass on an error message to an admin. That error message is either also present in the regular PHP error log or very much should be if it's not.

If I'm wrong about that, I strongly question why those log files exist at all. If I'm right, I'm not entirely sure I understand your objection to the PHP equivalent of a tail -c 1234 log.txt > log.new.txt; mv log.new.txt log.txt.

[Alkarex]

FreshRSS needs logs. We can change the way they are stored, and we can change the way they are processed. We can also improve the documentation.
I think before choosing an approach, we should involve people doing packaging of FreshRSS.

What do you think from Cloudron (@dswd), DPlatform (@j8r), YunoHost (@plopoyop), Ansible (@oupala)?
Could FreshRSS do anything to help with log management in your respective cases?
What are the best practices in your respective cases?

[dswd]

Cloudron apps are expected to log to stdout and stderr as those can be accessed via the logs command. It would be great if the logging could be configurable, e.g. USER_LOG_PATH="{{DATA}}/users/{{USER}}/log.txt" where I could just overwrite this with USER_LOG_PATH="/dev/stdout".

At the minimum it would be good to have one folder for all log files and not having them scattered over multiple folders. Then I could simply move that folder (via symlink) to volatile storage and have a task that cleans that folder periodically.

[Alkarex]

@dswd Thanks for the quick answer. FreshRSS has a UI for showing the (latest) logs (per user). Can you see an approach that would allow FreshRSS to read (parts of) the logs back?

[dswd]

Not really, you could keep the latest log messages separately in some kind of ring buffer but I don't know how to read from stdout.

[aledeg]

At the minimum it would be good to have one folder for all log files and not having them scattered over multiple folders. Then I could simply move that folder (via symlink) to volatile storage and have a task that cleans that folder periodically.

We could have something similar to Symfony. They choose to have a tree similar to linux. So the logs are in var/logs, the cache in var/cache and so on.

[Alkarex]

Yes @aledeg , I think an organisation like you are proposing would be good. We just need to ensure it works well for various platforms and FreshRSS packaging.

[Frenzie]

I'd like to raise a voice for shared hosting users. Implementing @aledeg's excellent suggestion wouldn't make matters any worse for them, but they'd still be stuck with the issue.

[oupala]

@Frenzie We could imagine that the log storage could be prefixed. It would be / for dedicated hosting (resulting in /var/log/ and /var/cache/) and would be /path/to/shared/user/home/ for shared hosting (resulting in /path/to/shared/user/home/var/log/ and /path/to/shared/user/home/var/cache/).

[aledeg]

Actually, I didn't mean /var/log but /whatever/path/to/freshrss/var/log

[aledeg]

When you have your logs in the same directory, it's really easy to configure logrotate to rotate all your log files.
See here for more information on logrotate configuration on that matter.

[aledeg]

If you're using Apache, this is a really good example and explanation on how to configure logrotate.

[Frenzie]

@aledeg My point about shared hosting is that people may not have proper access to SSH or cron. Even if they are able to use SSH and cron it's quite likely that they don't access to logrotate, but as far as the issue goes that's much less problematic. Cf. #613.

[aledeg]

I think we could tackle that problem by reading the file starting from the end and display only a subset of the file. We could also indicate the size of the file to avoid growing file on the filesystem.
I found a way of doing that purely in php but it's really slow since I read the file one char at the time.

There is another solution but it uses a Linux tool, tac. How do you feel about that ?

[Frenzie]

Couldn't you just read, say, 100 kB at a time or at least line by line?

[aledeg]

That was my original idea but I couldn't make it work for now.
Here is my inspiration

[Alkarex]

https://stackoverflow.com/questions/15025875/what-is-the-best-way-in-php-to-read-last-lines-from-a-file

[Frenzie]

Search engining around a little this seems more like what I would write if I were so inclined :-P (I.e., read certain-sized blocks at a time and we'll see what happens.)

https://stackoverflow.com/a/6451391/2470572

[Frenzie]

@Alkarex Wow, that's pretty in-depth.

Indeed, this is basically the upgraded solution of what I just linked to: https://gist.github.com/lorenzos/1711e81a9162320fde20

(I.e., assume lines are approximately x bytes and multiply by line number as a starting point.)

[dswd]

Sorry if I sound like a know-it-all but why don't you do a simple logrotate on your own? If you are interested in the last N lines, then log until you have N lines and then start a new file. This way you will always have a file with up to N lines in it. If you also keep the last file around, you will always have between N and 2N lines. This should be easy to implement and effectively limit file sizes.

[aledeg]

The thing is that the logrotate mecanism should be done outside of FreshRSS. It's already done by dedicated software. I think that we should try to keep FreshRSS simple and not implement everything we need when there is existing tools.
You're approach is correct and the idea is just a fail-safe in case of logs that are too big.

[Frenzie]

I agree that real log rotation probably shouldn't be done within FreshRSS. But with regard to what I said before about shared hosting (https://github.com/FreshRSS/FreshRSS/issues/1562#issuecomment-305912762 and https://github.com/FreshRSS/FreshRSS/issues/1562#issuecomment-306131955), I still think some simple form of log limiting would make a lot of sense as an option. But regardless, I think we're missing the most important step in that the user should be notified of the problem.

1. Get filesize.
2. if $filesize > 500 kB (my proposed default value)
3. Display message to user (but preferably not on every log write): Your log has exceeded 500 kB. You should either set up a cron job for log rotation (+link to relevant docs) or enable auto-log limiting.
4. User chooses: Ignore warning, I know what I'm doing, Set up auto-lmit

Otherwise the weakest users might easily be left with an enormous log file taking up all of their shared hosting space. And more advanced users could also end up with a 300 MB file (see OP) before they ever notice anything's awry. I want to be able to fully recommend FreshRSS to people who are just taking their first steps outside of SAAS. :-)

[Alkarex]

A new behaviour has just landed in the /dev branch: if a given log file grows larger than 1MB (value defined in constants.php), then the file is truncated to keep only the newest 0.5MB part. Please give it a try :-) https://github.com/FreshRSS/FreshRSS/pull/1712

On can disable this auto-truncate behaviour, by setting MAX_LOG_SIZE to a negative value.
We might have to set it to a larger value than 1MB for easier interaction with logrotate.

P.S.: I plan to soon rethink the location of the logs, to be more standard.
P.P.S.: I personally had my own log processing outside FreshRSS, but given the number of reported issues, and given the fact that FreshRSS will currently not be able to show the logs if they are too large (we should have a proper tail approach), I think we need to have some kind of protection inside FreshRSS, at least fore the time being.
P.P.P.S. :-P @Frenzie I forgot your input about noticing the users. A quick-fix would be to add (yet) another line to the log when it is truncated :-)

[kevinpapst]

As stated in #1712 my feedback is: works great and does not suffer from performance issues, even for users with large logfiles (tested with a 420MB logfile) => can be closed?!

[Alkarex]

Message added to a log when a log is truncated: https://github.com/FreshRSS/FreshRSS/pull/1726

[Alkarex]

The auto-log-rotation can easily be disabled (or increased) thanks to the custom constants introduced in https://github.com/FreshRSS/FreshRSS/pull/1725