Duplicati: Suggest better values for dblock-size and block size

[jdahl]

The wizard could in it's final step have a analyzer of the data and suggest better values for dblock-size and blockssize. For example if it notice most are small compressible files it would use current default. But if it find out most are non-compressible large files like images and specially videos it could suggest other more suitable defaults. Like a dblock size of 1Gb and blocksize of 100Mb for example.
Without this the database could become too large when used with terabytes of data and the user wont understand why.

[mauro-miatello]

what is the difference between volume size and dblock ?
I've blocksize of 100Mb, volume of 1GB and default dblock of 50MB

[jdahl]

As far as I know so is "volume size" the pretty name for the "dblock-size" advanced configuration

[vnau]

To make such a suggestion source files should be scanned during backup configuration process. This could take a while especially in case of lots of small files. In this case a user have to wait until wizard completes scanning.
Another difficulty is that optimal volume size significantly depend on selected backend, connection quality and bandwidth.

[jdahl]

It shouldn't be a requirement but a suggestion to the user, specially if the backup is say above 100Gb.

Would be nice if known limits to volume sizes where taken into account for each specific backend. That way the recommendation could go up to that size but not above it. The user should still be allowed to overrule it if they know better.

[kenkendk]

As @vnau mentions, it is difficult to balance this.
I have written some of the considerations here:
https://www.duplicati.com/articles/Choosing-Sizes/

I have considered allowing the block-size to be dynamic, such that files that grow can gradually increase the block size to keep the overhead down. Such a change is non-trivial, so I have not been able to pursue it.

Also, as @vnau mentions, scanning the disk to figure out the size of the backup can take some time. This needs special care, because the user could close the browser window while counting the files, and then the counter will keep running (can be fixed).

Despite all this, I think we should do it anyway, starting with some basic stuff like:

size > 1GB => volume-size=100mb, block-size=200kb
size > 5GB => volume-size=100mb, block-size=300kb
size > 10GB => volume-size=200mb, block-size=500kb
...