Go-ipfs: Never fill the disk

[Stebalien]

We should stop writing to the datastore once we get down to some minimum amount of remaining space left. Unfortunately, this'll be tricky due to platform differences but go-ipfs really stops behaving well when the disk fills up.

[ghost]

In many cases it actually refuses to start if the disk is full, so you don't get to run GC, and must reclaim space in areas outside the ipfs repo.

That might be a good additional goal: don't require disk writes just to start.

[Stebalien]

That might be a good additional goal: don't require disk writes just to start.

Well, we can't write our API file in that case. Assuming we can do some disk IO, we can mostly start but won't be able to do a GC (badger, e.g., needs to be able to write to delete).

Really, we just need to avoid running out of space. If something else ends up chewing up all the disk space, there's not much we can do (everything else will grind to a halt as well). However, we can avoid doing that ourselves.

[bonedaddy]

what effect does the IPFS config directive for the upper bound of datastore disk usage % have on this? Shouldn't capping the utilization % stop this from happening? Sorry if any of the questions have obvious answers, been diving deep down the IPFS rabbit-hole and just want to understand more about how it works.

[Stebalien]

what effect does the IPFS config directive for the upper bound of datastore disk usage % have on this?

We don't currently have one. We have a max absolute size config option but we don't really enforce it (we just use it to decide when to GC if GC is enabled).

Shouldn't capping the utilization % stop this from happening?

That's basically what this issue is. We need to refuse to write to a disk that doesn't have any space left.

Nit: We'd probably want to reserve 1% up to 1GiB (so we don't, e.g., reserve 10GiB on a 1TiB drive.