Go-ipfs: Handle the case where GC is disabled but a repo limit is configured
When a repo size limit is configured but gc is disabled (i.e., no --enable-gc flag has been passed), we should either (or both?):
- Warn on start.
- Refuse to write to the datastore once the size has been exceeded.
Stebalien
All 7 comments
Warn on start
:+1: This would have to be something like if Datastore.MaxStorage != defaultMaxStorage I guess
Refuse to write to the datastore once the size has been exceeded.
This would probably lead to situations where your node is running fine but has exceeded MaxStorage, you update go-ipfs or restart for whatever reason, and suddenly it refuses to start. We could combine this with MaxStorage usage mandating --enable-gc though, i.e. it'd run GC right away in that case. (And use the warning on start as a migration path.)
ghost
on 19 Jun 2018
馃憤
ability to quietly exceed maxStorage is definitely not what I would expect -- warning on start would be a good start
parkan
on 19 Jun 2018
I like both options. I think you could have multiple levels of warnings:
1) warning when datastore usage is at 90%
2) non fatal error when datastore usage is at 95%
3) fatal error, preventing startup when datastore usage is at 99%
@lgierth makes a good point about preventing writting to the data store once the size is exceeded, however I would be more concerned about what disastrous potentials attempting to write to a datastore that you have exceeded limits for. What if by attempting to write to a datastore whose limits you passed you overwrite already existing data, or fill up the remaining space too much such that an application which needs that space may not run properly?
In the (my) ideal world, the node would fail to write to the datastore when it fills up, and entering a sort of "locked" state, where you can serve content but are prevented from writting new data, and only GC'ing data.
bonedaddy
on 23 Jun 2018
Hey @Stebalien , Did you have finished this issue?If not , I want to do for this. Because i want to learn more about gc through this issue.
kjzz
on 19 Sep 2018
Go ahead.
Stebalien
on 24 Sep 2018
@Stebalien Get it .Thx,I am doing this now.
kjzz
on 24 Sep 2018
Hey @Stebalien, i have create a pr about this issue , please help me review it?
kjzz
on 13 Nov 2018
Related issues
djdv
路
3Comments
emelleme
路
3Comments
magik6k
路
3Comments
Mikaela
路
3Comments
whyrusleeping
路
4Comments
Most helpful comment
I like both options. I think you could have multiple levels of warnings:
1) warning when datastore usage is at 90%
2) non fatal error when datastore usage is at 95%
3) fatal error, preventing startup when datastore usage is at 99%
@lgierth makes a good point about preventing writting to the data store once the size is exceeded, however I would be more concerned about what disastrous potentials attempting to write to a datastore that you have exceeded limits for. What if by attempting to write to a datastore whose limits you passed you overwrite already existing data, or fill up the remaining space too much such that an application which needs that space may not run properly?
In the (my) ideal world, the node would fail to write to the datastore when it fills up, and entering a sort of "locked" state, where you can serve content but are prevented from writting new data, and only GC'ing data.