Tuf: Delta Support

[HulaHoopWhonix]

Please add support for binary delta updates to reduce bandwidth requirements for server admins who can't afford large running expenses.

[vladimir-v-diaz]

We do plan to reduce the bandwidth requirements by minimizing the size of metadata and using delta compression. In fact, we just submitted (last night) a paper that covers your exact request. We found that the minimized metadata + delta compression / encoding / compression incurred about 4% the overhead of an average package each month (a monthlong trace of package requests for a popular community repository).

The reference implementation has so far been updated with the changes that minimize metadata (only store version information of available metadata on the repository) and will soon have an option to allow administrators to use delta compression / encoding (for now, only compression of metadata is optionally supported)

[vladimir-v-diaz]

Delta updates for binary / target files should be doable, as well.

[vladimir-v-diaz]

I should add that it is only my opinion that binary delta compression can be supported, but I'd need to go over it with the rest of the team. I'll get back to you once we've had that discussion.

[vladimir-v-diaz]

Hi @HulaHoopWhonix

Another straightforward way to reduce bandwidth requirements (apart from the minimized metadata I have outlined above) is to treat binary deltas as target files that are specified in TUF metadata. In other words, binary deltas can be regarded as any other target file that is secured with TUF. This approach is supported today.

Binary deltas can also be generated by your preferred diff utility and patched once the binary delta has been downloaded via TUF and transferred to the software update system.

[HulaHoopWhonix]

OK Thanks.

[probonopd]

Have you considered zsync?

[azubieta]

Hi! Any update on this?

[probonopd]

Also check https://github.com/AppImage/zsync2

[trishankkarthik]

For my experiments on Mercury (a feature of TUF that reduces B/W costs used to prevent rollback attacks), I found that bz2 works well enough in practice for metadata of intermediate size (not too small, not too large). I compressed deltas of JSON there.

[MVrachev]

Do we think this idea is worth exploring two years later @trishankatdatadog or we want to close that issue?

[trishankatdatadog]

Do we think this idea is worth exploring two years later @trishankatdatadog or we want to close that issue?

Probably not. No one's asked for it yet. When they do, we will revisit. Thanks!