Server: Trash get fully emptied when Cron runs (v18.0.8 and v19.0.2)

[biva]

I updated yesterday to 18.0.8. My quota is unlimited (it shows 0 B). All my trashbin (deleted files) have been definitely removed.

Step to reproduce:

• Delete a file test.txt
• Go to deleted files in the web UI: the file is there
• Force the cron job to execute (sudo -u www-data php -f /var/www/html/nextcloud/cron.php)
• GO to deleted files: no file anymore

In the log, I see:
"remove "test.txt" (0B) to meet the limit of trash bin size (50% of available quota)"

Am I the only one with this critical issue?

[biva]

I tried to change the quota (100 GB, while I have 30 GB of data). Same result, the file is removed from the deleted files.

This is a critical issue: we have data losses :( !!

[biva]

Just updated to 19.0.2: same result

[ne20002]

Confirmed. All deleted files gone. :(
Also, newley deleted files get deleted from ttrashbin right away...

[SimplyCorbett]

19.0.2. Confirmed.

[eppfel]

I have somehow a similar problem, but I#m not sure if it is the same.

I reproduce the weird log message

remove "Android" (0B) to meet the limit of trash bin size (50% of available quota)

...which is odd, because obviously 0B.

My problem was that the web UI showed an empty trash bin under deleted files, but the files_trashbin folder on the server was not empty.

I just moved from 18.0.7 to 18.0.8 to 19.0.2, when I encountered this. The files I had deleted were on external storage.

Now I used occ trashbin:cleanup on my admin user, which cleaned up the files on the server.

However, I was not able to reproduce the steps as before, so maybe cleaning up the trashbin removed the behaviour. Could be related to the upgrade and something getting stuck.

[acsfer]

@icewind1991 can this be a regression caused by https://github.com/nextcloud/server/pull/21658?

Also, another similar issue here https://github.com/nextcloud/server/issues/10862

[jknockaert]

Setting trashbin_retention_obligation to 30,360 keeps the files in the trashbin for 30 days and deletes every thing else. So that seems to confirm that the determination of the available free space is not working correctly.

[biva]

Is there anybody from Nextcloud in this thread investigating this issue? This is a critical one!

[danielb42]

This affects me since upgrading to 19.0.2 a few hours ago. ~It was not happening with any 18.x version.~ // edit: Sorry, don't know if 18.0.8 was affected since the updates 18.0.8->19 came so fast. Benefit of doubt for "was affected".

[dorpauli]

My Nextcloudpi instance was updated today to version 19.0.2. and this issue came up. It's exactly the same behavior as @biva described. And I also think this issue is critical!

The comment of @acsfer made me realize, that now you can configure the max trashbin-size. Because this is a new feature, I think the default setting of 50% available space is corrupt.

So I just tried to change it to a max of 1G global tashbin-size with sudo -u www-data php .../nextcloud/occ trash:size 1G and it worked. With the manually setting I can avoid the issue.

[biva]

@MorrisJobke @ChristophWurst @icewind1991 this issue is really critical and causes data loss. Wouldn't it this time to stop the release of 19.0.2 and 18.0.8?

[danielb42]

Well, if losing the contents of your trash bin is a critical data loss for you, I strongly suggest you rethink your work flows.

[SimplyCorbett]

cc @rullzer 19.0.2 should be pulled.

[SimplyCorbett]

Well, if losing the contents of your trash bin is a _critical_ data loss for you, I strongly suggest you rethink your work flows.

Let me accidentally delete that 10 year old photo of my family. Whoops! Let me restore it from trashbin... gone. Cron already ran.

I'm smart enough to have backups but most people don't.

[danielb42]

@Dragon988 I'm not saying this shouldn't be fixed with urgency. But calling for suspending update distribution because the litterbox gets cleaned a bit too often seems a bit excessive.

[SimplyCorbett]

@Dragon988 I'm not saying this shouldn't be fixed with urgency. But calling for suspending update distribution because the litterbox gets cleaned a bit too often seems a bit excessive.

Yeah, maybe if it's for personal use. Remember that nextcloud is used by organizations around the world. Some of those organizations may have a legal responsibility to retain documents for x amount of time. Those organizations might have trashbin setup in such a way to automatically delete those objects after that amount of time.

tl;dr you're downplaying this severely.

[eppfel]

thanks @kesselb

Not sure if related but as you are trying to fix the way it determines the trashbin size it might.

I set the system trashbin size with occ but still received this one: remove \"DSC_0236.jpg\" (1519434B) to meet the limit of trash bin size (50% of available quota)

{"reqId":"8PAK9tmDlTas2BcXP4Dj","level":1,"time":"2020-09-01T12:14:37+00:00","remoteAddr":"","user":"--","app":"files_trashbin","method":"","url":"--","message":"remove \"DSC_0236.jpg\" (1519434B) to meet the limit of trash bin size (50% of available quota)","userAgent":"--","version":"19.0.2.2","id":"5f4e5a7d3720c"}

And checking trashbin size, when it was not set, gives a parse error:

$ occ trashbin:size [username]
Failed to parse input size

[kesselb]

$ occ trashbin:size --user=[username]

[biva]

@rullzer thanks for having solved that, but I'm quite frustrated to not see any detail (how to fix it for current users who have experienced the bug?) and to not have any message from the nextcloud team showing sympathy after this critical bug which led to data losses :(

[ChristophWurst]

https://patch-diff.githubusercontent.com/raw/nextcloud/server/pull/22528.patch is the patch.

You can restore the files from your backup.

Thoughts and prayers.

[biva]

@ChristophWurst is it irony? If yes, do you really think it's a nice way to consider contributors who submit issues?

[ChristophWurst]

Of course it is.

But on a serious note, you really should have a backup. And then it's easy to get back anything that was deleted by accident, bot by user fault as well as by software fault.

We don't provide everyone with a step-by-step guide on how to fix their instance. We're already 100% transparent on the code changes, so it's not rocket science to follow the github references and see what was changed to close the ticket. And the fix will go into the very next maintenance release. Anyone who keeps their system up to date will get the patch ASAP.

Also, this is the (technical) issue tracker of our software and not the support forum.

[biva]

So you treat with irony a contributor who took time to properly submit an important issue and helped you to solve this issue in a constructive way. Thank you, much appreciated.

I thought that the objective of Nextcloud was to better integrate the community when you forked from owncloud. As a community member, I'm hurt by your mocking.

[ChristophWurst]

What were you expecting as a response from

to not have any message from the nextcloud team showing sympathy after this critical bug

? We build this software together as a community. Would you expect anyone to apologize for a bug in the free and open source software? Do we need to use git blame to find the author of the bug?

If I did offend you in some way then I'm sorry. This was not my intention.

I really appreciate your bug reports.

[jknockaert]

And the fix will go into the very next maintenance release. Anyone who keeps their system up to date will get the patch ASAP.

Is there an updated release schedule? The one at https://github.com/nextcloud/server/wiki/Maintenance-and-Release-Schedule got a bit out of sync...

[ChristophWurst]

Is there an updated release schedule? The one at https://github.com/nextcloud/server/wiki/Maintenance-and-Release-Schedule got a bit out of sync...

Oh, it should not. But I'll ask the release team :)

[jospoortvliet]

And the fix will go into the very next maintenance release. Anyone who keeps their system up to date will get the patch ASAP.

Is there an updated release schedule? The one at https://github.com/nextcloud/server/wiki/Maintenance-and-Release-Schedule got a bit out of sync...

what is out of sync? At first glance it seems correct...

[biva]

Would you expect anyone to apologize for a bug in the free and open source software? Do we need to use git blame to find the author of the bug?

I expected exactly what I wrote: sympathy. Where did I ask for apologies? I just expected a kind word, something like: "thanks for this bug report, we are working on it and we'll try to fix it". We are human, I just expected understanding between people.

[jknockaert]

what is out of sync? At first glance it seems correct...

Both current and next release are specified as 19.0.2/18.0.8. So there seems to have been a change of plan which hasn't been fully translated to that page...
Bugs like this one may motivate a point release rather than wait another month as that page seems to suggest now.

[jospoortvliet]

I expected exactly what I wrote: sympathy. Where did I ask for apologies? I just expected a kind word, something like: "thanks for this bug report, we are working on it and we'll try to fix it". We are human, I just expected understanding between people.

I get what you mean, and it isn't unreasonable to ask for. Unfortunately, for one, for most of the devs they handle dozens of bug reports a day, so that leaves a bit less room for niceties. And there is, even more unfortunately, a lot of abuse thrown at them by some users which probably numbs us down even more.

I hope you managed to recover the data you lost and, well, the bug fix will be in the upcoming release, next month - so it shouldn't happen again.

[jospoortvliet]

what is out of sync? At first glance it seems correct...

Both current and next release are specified as 19.0.2/18.0.8. So there seems to have been a change of plan which hasn't been fully translated to that page...
Bugs like this one may motivate a point release rather than wait another month as that page seems to suggest now.

ah, the version numbers of the upcoming releases were wrong. I fixed them ;-)

[biva]

FYI, I tried to apply the patch, but it doesn't work. I'm probably doing something wrong and I started a discussion in https://help.nextcloud.com/t/solve-the-trashbin-issue-integrating-a-patch-doesnt-work/91337

[SimplyCorbett]

@ChristophWurst Just to confirm, 19.0.2 is going to remain mainline and available for everyone with this critical bug and won't be patched until 19.0.3 is released on 10-08-2020?

I follow github and watch things play out but most people using nextcloud don't do that. If 19.0.2 and 18.0.8 is going to remain available with the bug as is then can we at least get a warning out on the nextcloud news?

[rullzer]

We are planning a release next week. Butsince there have been many changes already on the stable branches we will first have the RCs today. Please do test them.

[jospoortvliet]

@ChristophWurst Just to confirm, 19.0.2 is going to remain mainline and available for everyone with this critical bug and won't be patched until 19.0.3 is released on 10-08-2020?

I follow github and watch things play out but most people using nextcloud don't do that. If 19.0.2 and 18.0.8 is going to remain available with the bug as is then can we at least get a warning out on the nextcloud news?

As @rullzer said we're going to do an extra bugfix release to get this out sooner, we agree a month takes too long. The build isn't done yet, there is another issue we want to get in but it is very stubbornly resisting a fix :cry: