Mailu: mailu is continuously spamming temp files in "filter" folder

[adb-sh]

Hey,
I'm running the Mailu master branch on my server. I installed it with the Mailu docker compose setup file. I've no idea what happened, but since a while I'm getting pretty much tmp.* files/folders in the "filter" folder. So its probably a problem with rspamd? I first noticed this when my 30GB VM crashed, cause the main volume was full. So I moved the installation to an external 2TB nfs filesystem. But after a few days even the 2TB were full of temporary files?! Whats going on here?

[Diman0]

It could be Rspamd or Clamav. I'd rather think the problem is with Clamav. Rspamd uses sqlite and map files as temp/cache files in the filter folder.
If you restart the Clamav and Rspamd containers and check the logs, do you see any mention of the tmp files in the log?

[adb-sh]

Yes you're probably right with clamav, because after reinstalling all docker containers (same dockerfile), clamav were continuously restarting. After a few weeks from then I started it again now to get the logs and it says, that the database is outdated, but I think that it wasn't displayed before.

Fri Jul 17 21:42:16 2020 -> Limits: Global time limit set to 120000 milliseconds.
Fri Jul 17 21:42:16 2020 -> Limits: Global size limit set to 157286400 bytes.
Fri Jul 17 21:42:16 2020 -> Limits: File size limit set to 31457280 bytes.
Fri Jul 17 21:42:16 2020 -> Limits: Recursion level limit set to 10.
Fri Jul 17 21:42:16 2020 -> Limits: Files limit set to 15000.
Fri Jul 17 21:42:16 2020 -> Limits: MaxEmbeddedPE limit set to 10485760 bytes.
Fri Jul 17 21:42:16 2020 -> Limits: MaxHTMLNormalize limit set to 10485760 bytes.
Fri Jul 17 21:42:16 2020 -> Limits: MaxHTMLNoTags limit set to 2097152 bytes.
Fri Jul 17 21:42:16 2020 -> Limits: MaxScriptNormalize limit set to 5242880 bytes.
Fri Jul 17 21:42:16 2020 -> Limits: MaxZipTypeRcg limit set to 1048576 bytes.
Fri Jul 17 21:42:16 2020 -> Limits: MaxPartitions limit set to 128.
Fri Jul 17 21:42:16 2020 -> Limits: MaxIconsPE limit set to 200.
Fri Jul 17 21:42:16 2020 -> Limits: MaxRecHWP3 limit set to 16.
Fri Jul 17 21:42:16 2020 -> Limits: PCREMatchLimit limit set to 10000.
Fri Jul 17 21:42:16 2020 -> Limits: PCRERecMatchLimit limit set to 10000.
Fri Jul 17 21:42:16 2020 -> Limits: PCREMaxFileSize limit set to 26214400.
Fri Jul 17 21:42:16 2020 -> Archive support enabled.
Fri Jul 17 21:42:16 2020 -> AlertExceedsMax heuristic detection disabled.
Fri Jul 17 21:42:16 2020 -> Heuristic alerts enabled.
Fri Jul 17 21:42:16 2020 -> Portable Executable support enabled.
Fri Jul 17 21:42:16 2020 -> ELF support enabled.
Fri Jul 17 21:42:16 2020 -> Alerting on broken executables enabled.
Fri Jul 17 21:42:16 2020 -> Mail files support enabled.
Fri Jul 17 21:42:16 2020 -> OLE2 support enabled.
Fri Jul 17 21:42:16 2020 -> PDF support enabled.
Fri Jul 17 21:42:16 2020 -> SWF support enabled.
Fri Jul 17 21:42:16 2020 -> HTML support enabled.
Fri Jul 17 21:42:16 2020 -> XMLDOCS support enabled.
Fri Jul 17 21:42:16 2020 -> HWP3 support enabled.
LibClamAV Warning: **************************************************
LibClamAV Warning: ***  The virus database is older than 7 days!  ***
LibClamAV Warning: ***   Please update it as soon as possible.    ***
LibClamAV Warning: **************************************************
LibClamAV Warning: **************************************************
LibClamAV Warning: ***  The virus database is older than 7 days!  ***
LibClamAV Warning: ***   Please update it as soon as possible.    ***
LibClamAV Warning: **************************************************

But how can I manually update the database and why isn't it done automatically? Maybe some missing statement in the clamav confing? Greetings :)

[Nebukadneza]

Hi There,

The Mailu-Project is currently in a bit of a bind! We are short on man-power, and we need to judge if it is possible for us to put in some work on this issue.

To help with that, we are currently trying to find out which issues are actively keeping users from using Mailu, which issues have someone who want to work on them — and which issues may be less important. These a less important ones could be discarded for the time being, until the project is in a more stable and regular state once again.

In order for us to better assess this, it would be helpful if you could put a reaction on this post (use the :smiley: icon to the top-right).

• 👍️ if you need this to be able to use Mailu. Ideally, you’d also be able to test this on your installation, and provide feedback …
• 🎉 if you find it a nice bonus, but no deal-breaker
• 🚀 if you want to work on it yourself!
  We want to keep this voting open for 2 weeks from now, so please help out!

[lub]

But how can I manually update the database and why isn't it done automatically?

This is done automatically by freshclam, but this is a separate daemon. When clamav checks it's database before freschlam finished updating it you'll get the warning.

Could you maybe show a find or similar of the tmp structures?

[Diman0]

We have also seen clamav has issues when the system has a low amount of memory. If swap is disabled, please enable it and make swap at least 1 GB on the host.
Try increasing the amount of RAM of the host machine to at least 4GB.

See issue #470 for more information.

[adb-sh]

I'm currently using another Mailu instance where I deactivated the mailu_antispam container due to this error. I tried to reproduce the error (same VM, same dockerfile) but wasn't able to do so. Probably clamav sucessfully updated itself. The VM has and had 4GB of RAM and a 2GB swap file. The structure of the temp files looked like this: ./filter/tmp.*/*

[cneumaier]

I installed mailu in a kubernetes cluster and I am facing the same issue. My volume is constantly filling up because of this issue. I disabled the antispam container as well, just like @adb-sh. I would love to see this issue being resolved...

[Steedalion]

Is it a bad idea to make schedule cron to periodically delete the *.tmp files?