Zstd: List command (displayInfo)

Created on 19 Jul 2017  路  8Comments  路  Source: facebook/zstd

There are two issues with the list command:

  1. It only accepts skippable frame magic 0x184D2A50 and reports "not compressed with zstd" for 0x184D2A51 to 0x184D2A5F.
  2. Displaying the sizes in MB doesn't make much sense for small files (< 1 MB).

Both issues are easy to fix.

All 8 comments

Good points @ib !
cc @paulcruz74

There are more issues:

  • If I add two skippable frames (one before a Zstandard frames, one after it) I get:
Skippable  Non-Skippable  Compressed  Uncompressed  Ratio  Check  Filename
        1              0     0.00 MB       0.00 MB  0.000   None  test1.zst

Shouldn't it be: 2 Skippable, 1 Non-Skippable? Where is the ratio?

  • If I add one skippable frame after a Zstandard frame I get:
There was an error reading skippable frame sizeAn error occurred with getting file info
Skippable  Non-Skippable  Compressed  Uncompressed  Ratio  Check  Filename
        0              1     0.00 MB       0.00 MB  0.120  XXH64  test2.zst

The (plain) ratio is usually (i.e. by all other compressors I know) defined as compressedSize/decompressedSize, not the other way around.

EDIT: I have to revise my remark a bit. The ratio currently shown with the list command (decompressedSize/compressedSize) follows the definition of data compression ratio. Compressors usually show the reciprocal value, but lrzip, for example, does the same as zstd.
It would have been nice, perhaps, to have the same output here which is displayed at the end of a compression, but most important seems to not break compatibility. It would be extremely annoying if different versions of zstd would display different things with the list command. Programs utilizing zstd and parsing this information must be able to rely on a "stable information".
The sizes have a measure, though. So they can and should show their values also in KB or B in order to not display only zeros.

Patches attached for all bugs that are mentioned here.

... and on top of the previous four patches a fifth one for output beautification.

@Cyan4973, are you going to apply these?

Yes,
I'm a bit concentrated on LZ4 right now,
but when I come back to zstd, I'll take care of your patches

In latest dev branch update, I've updated the --list command based on your recommendations and patches.

While doing it, I've spotted another potential error, which still needs to be fixed.
It probably has no impact currently, because it's related to a type of block (rle) which is not generated by existing encoder.

Was this page helpful?
0 / 5 - 0 ratings