Zstd: The speed of zstd at default compression level(3) is only 0.43 of snappy. Is this phenomenon normal?

[wangxiaoq]

Hi, I have tested lz4, snappy and zstd algorithms on my server, and there is a problem confused me.

I use the default compression level(3) of zstd, the compression speed is almost the same as snappy,
but the decompression speed of zstd is only 0.43 of snappy. Is this phenomenon normnal?

My test environment is as follows:
Server OS: Redhat Enterprise Linux Server release 6.3
CPU: Intel(R) Xeon(R) CPU [email protected]
zstd version: v1.4.5
snappy version: 1.1.3
Test Data: From hbase table without any compression algorithm.

Could you give me some advise?
Thanks.

[felixhandte]

Hi @wangxiaoq,

Compression algorithms are always balancing compression ratio against compression and decompression speed. By default, Zstd works harder to achieve a higher compression ratio than Snappy. So it decompresses a little slower. If you care more about decompression speed than ratio, you could try a "negative" compression level (e.g., --fast=1), which disables the entropy coding of the literals. Alternatively, give LZ4 a try.

Hope that helps.

[felixhandte]

@wangxiaoq, I'm going to close this based on inactivity. If you have any follow-up questions, please feel free to re-open.