Zstd: Is it splittable?

[robert3005]

Can I partition the compressed file and read it in chunks or does the compression format encode whole file

[Cyan4973]

The format allows multiple frames to be appended.
They will be regenerated in the same order, as if it was a single content.

So yes, it makes it possible to cut files into chunks of any size.

[terrelln]

You can break the original file up into chunks, compress each chunk independently into a Zstandard frame (using the zstd CLI), and concatenate each frame. This will still be valid in the eyes of the Zstandard format. You will then be able to break the compressed file up into the component frame and decompress each frame independently.

The only problem is that Zstandard doesn't encode the size of each frame, only the size of each block. So to find where the next frame starts you will have to do one of two things:

1. Seek through the frame a block at a time until you find the next frame.
2. Encode some extra information into the compressed data to indicate the size of each frame. You can use a Skippable Frame, as defined by the Zstandard format to do this.

[Abdulmalik13]

Cod

[Cyan4973]

Issue merged into #395