Zstd: Window size

[AKJ7]

Hello,

i was reading through the documentation and it seems that you have been able to compress integers above 1024 into a single byte as mentioned under the window_size section. The documentation shows a sample code on how you decompress the byte into the original value, but the compression wasn't shown. Could you point me to the file in which the integer encoding was made? I have actually tried to reverse engineer the decompression of the byte, but i can't fit the mantissa into the 3 bits, that are reserved for it without losing some integers.

Thanks.

[bimbashrestha]

Hey @AKJ7. Thanks for posting the issue. Do you mind linking which documentation you're referring to?:)

[terrelln]

Zstd currently only writes power-of-two window sizes. See the code we use to write the byte.
https://github.com/facebook/zstd/blob/ff6350c098300ee3050dc6a6cdc0f48032755e84/lib/compress/zstd_compress.c#L2633

[AKJ7]

@bimbashrestha, i meant this one
Edit: Section: Window_size

[terrelln]

@AKJ7 this format cannot represent all integers, since there are only 256 possible values. To find out what integers it can encode, you could simply write the decoding function, and "decode" every possible input 0, 1, ..., 255:

#!/usr/bin/env python3

for byte in range(0, 256):
  exponent = byte >> 3
  mantissa = byte & 7
  window_log = 10 + exponent
  window_base = 1 << window_log
  window_add = (window_base // 8) * mantissa
  window_size = window_base + window_add
  print(f"{byte} -> {window_size}")

Outputs:

0 -> 1024
1 -> 1152
2 -> 1280
3 -> 1408
4 -> 1536
...

[AKJ7]

@terrelln, i see. I thought the documentation meant that you could somehow encode every number from 1024 to 4 Tetra into a byte. Thanks.