Handbrake: Destination filesystem almost full

[frapell]

Description of the problem

I was encoding a video to an external pendrive directly, and right at the middle of it, the process paused because the filesystem was getting full while it still had 10GB free (Full encode was 1.5GB, so there was plenty of space left)

I see a closed issue about this same issue here gh-853

HandBrake version (e.g., 1.0.0)

1.2.1

Operating system and version (e.g., Ubuntu 18.04 LTS, macOS 10.14 Mojave, Windows 10 1809)

Ubuntu bionic 18.04

Error message text or screenshot

HandBrake Activity Log (see https://handbrake.fr/docs/en/latest/help/activity-log.html)

I don't have one since I restarted Handbrake and seems the old log is deleted when a new one is created :(

[sr55]

I'm not sure what your asking here?

The app is designed to pause things when the storage volume starts getting low on space. 10GB is a reasonable default limit to do so.

The limit is configurable in preferences so you can adjust based on your use case. For example, if it's a boot drive, you'd want a considerable amount of free space to cover for things like hibernation, swap etc. (Assuming your not booting from USB, you could set to a lower limit, or just turn it off)

[frapell]

@sr55 Well, I am not a regular user of the application, let me explain you why I created this ticket and why I consider it a bug:

• I needed to copy a video file to a pendrive
• Noticed the file was bigger than my free space
• Downloaded the latest version of handbrake and installed it
• Started to re-encode my video so it would fit my free space
• Whole process would take an estimate of 30 minutes
• Went to do something else, instead of staring at my computer for 30 minutes
• Came back 45 minutes later, to find the process was stopped at 20% with the error shown
• Had to wait 20 more minutes for the process to complete

The final result was a 1.5GB file, so issuing a warning at 10GB doesn't sound reasonable... Even less so in a 16GB pendrive.

Again, this is as an end user experience... A warning like this does sound useful, but I think the defaults need some tweaking... Maybe have different defaults depending on the quality of the encoding? or maybe have a lower, more reasonable default (maybe 1 or 2 GB) and let the user increase it if need be?

FWIW, I don't think an application like this should take into consideration swap or hibernation needs, becase it is so different from system to system that you will never get it right, and you end up getting in the way of some users (like my case).

Anyway, if you don't consider this a bug, feel free to close it

Cheers!

[bradleysepos]

That certainly is frustrating, and not the experience we intend of course. On the other hand, this feature has helped many users avoid system instability due to a full system disk; we know this because reports have become nil and now we only occasionally see a complaint about the protection feature.

I do think we need to re-evaluate the 10 GB limit. Perhaps something tighter like 2 GB would be enough. Additionally, if possible, we might want to look into an option to have the limit only apply to the boot disk.

Anyway, thanks for sharing your experience. For now, adjust the setting in preference to your needs and we'll discuss how we can reduce friction.

[bradleysepos]

Marking as an enhancement. @jstebbins @galad87 @sr55 can we agree on a tighter constraint, maybe only 2 GB (likely enough to preserve system stability), and is it possible to add a checkbox to apply the constraint only to a boot volume?

[sr55]

I'd argue against lowering it. By rights it should be no less than system ram +2 GB. So most commonly that would be 10GB, 18GB or 34GB for the vast majority.

10GB is a reasonable compromise as it'll catch the majority use case. 2GB will cause a lot of folks to get caught out again which would defeat the purpose.

As for boot drive, it's not quite that simple. You'd have to go looking at the system to see how/where it's configured to store it's various assets. It's not necessarily the boot drive.

[bradleysepos]

Does Windows not keep a hibernation file static on disk? macOS does.

[sr55]

I believe so, but it's not always fully allocated it seems.

[frapell]

I am not a regular user of this software, and if you used to have a lot of reports of people running out of space, and this implementation has helped turn all those reports down, I would then agree that it is positive and maybe you shouldn't change it.

Maybe (And I am just thinking out loud) an enhancement would be to display a big flashy warning at a high threshold (say 10GB), but do not stop the process until a more critical value is met (say 2GB)

Also, adding information in the warning on how to tweak or even disable this feature would be useful for non-regular users (I knew about it being a configurable feature after @sr55 first reply to this thread)

[bradleysepos]

Ah. Perhaps an interface like this would make sense, at least on Windows:

---

Disk space warning level:

(o) 10 GB (System RAM + 2 GB, Default)
(  ) Custom level:
      [            ] GB

[x] Apply disk space warning level only to system disk

_100 MB minimum warning level applies to all disks_

---

This would at least make it more clear what is happening and why, while allowing users encoding to external/non-system media to avoid the entire problem.

[bradleysepos]

For reference, since the macOS VM/hibernation file is always present, we don't have to worry about this at all. The preference looks like this and 2 GB seems sane:

The wording "Pause queue..." seems more indicative of what is going to happen as well.

[sr55]

I suspect the Mac GUI, like the Windows GUI (probably) isn't doing active monitoring. I honestly don't recall. I think it only checks on start. It would be better if it was active in my opinion but it was quicker to implement as-is

[sr55]

Confirmed, Windows UI only paused at the start of a job.

[bradleysepos]

A high/low threshold UI could be something like this:

---

Disk space warning levels:

[x] Warn when disk space is below:
    (o) 10 GB (System RAM + 2 GB)
    (  ) Custom level:
          [            ] GB

[x] Pause queue when disk space is below:
    (o) 2 GB
    (  ) Custom level:
          [            ] GB

[x] Levels apply to system disk only

_100 MB minimum warning level applies to all disks_

---

[bradleysepos]

Here's another idea, in the layout of the current Windows UI:

---

[x] Pause queue if system/hibernation disk space is lower than (GB):     [              ]
      _System RAM (8 GB) + 2 GB recommended_

[x] Pause queue if non-system disk space is lower than (GB):                  [              ]
      _2 GB recommended_

---

Currently, the two relevant lines in the preferences are disconnected by an unrelated preference (libdvdnav) between them.

This idea combines the checkbox (enable) and textbox (amount) into a single line, adds recommended levels (ideally in a smaller font size just below the label), and separates the behavior for system and non-system disks.

[bradleysepos]

We were also talking on IRC about how to warn the user. I'd prefer a status message to a modal dialog. Instead of "Queue paused" or similar when manually paused, perhaps something like "Queue paused due to low disk space" when automatically paused due to exceeding one of the thresholds.

A system notification would seem to be in order as well.

[jstebbins]

Just for reference, I started a queue full of stuff a week ago then went on a trip. When I got back (a week later), my queue was half done, paused, and this dialog was waiting for me. I moved the finished encodes to their final destinations to free up space and resumed the queue. It saved me from having diagnose why a bunch of encodes failed.

In this case, the disk being written to was not by system partition, so the system would have remained stable. But it would have filled my user partition which could have resulted in some mysterious behaviour that I would not have initially understood till diagnosing the encode failures. I.e. I may not have been able to open the activity log since editors usually create temp file backups of the edited file.

I agree that the default limit could be substantially lower. On linux 1GB is more than enough. When I set it initially, I was thinking in terms of encoding to HDD where sizes are massive and 10GB is 0.0025% of the total space. Percentage-wise, it seemed like a good number ;)

[jstebbins]

FYI, preferences for this feature on linux looks like:

[sr55]

I don't really see value in spending time adding complexity to deal with drive types. Folks have all kinds of wonderful setups and thinking about all the scenarios that could come up just isn't a good use of time.

Inst read, I'd rather just keep it simple.

I've matched the Mac UI's 2GB default (and I've added Active monitoring of the destination drive like the Linux GUI so that it's far less likely to run the drive to 0)

Since the Linux UI already has active monitoring, I'd say set it to 2 and call it a day.

It'll still trip folks up from time to time, but that's exactly why the preference is there.

[jstebbins]

I've reduced the default to 2GB on linux to match the other UIs 1e6078b

[sr55]

Fair enough. Think we call it a day then. Mac is already lower :)