Zstd: Could zstd be compatible with zlib?

[JamesShenmj]

When we use zlib to compress a file, the format is .zip, which can not be decompressed by zstd.
Because of this(forward compatible), the programs which use zlib have little will to instead by zstd.
Will zstd start to support it?
Thanks!

[Cyan4973]

It's an opened question. That's technically possible, we would need to make it an objective.

[gyscos]

As a note, .zip is more than just a compression format, but rather a full archive format.
Supporting zlib would more likely allow to uncompress .gz files.

[Cyan4973]

That's a very good point @gyscos .
zlib, gz and zip are 3 different (though related) formats.
zlib is not a file format, so I guess it's not the point.
I agree that .gz has a higher priority.

[JamesShenmj]

Yes, I agree with gyscos and Cyan.
zlib is a compress/decompress algorithm.
.gzip is a file format, and use zlib to compress/decompress.
.zip is a full archive format, may use zlib or other algorithm.

@Cyan4973 :I see zlibWrapper in zstd 1.1.0, does it begin to be considered ?
Thank you all.

[Cyan4973]

zlibWrapper works pretty well so far @JamesShenmj, you can use it.
It can decode a mixture of zstd and zlib compressed messages automatically.

[JamesShenmj]

@Cyan4973: zlibWrapper is included in zstd 1.0.0, but how should I use it?
I compile and build the simple_decompression.c, but the binary simple_decompression cannot decompress .gz format.
Maybe I have some mistakes.
Thanks!

[JamesShenmj]

@Cyan4973 May I have your email address?

[Cyan4973]

The zlibWrapper replaces the zlib API (zlib.h).
As stated above, it's a bit different from .gz files.
Basically, .gz file format encapsulates zlib streams into its own overlay.

We don't have .gz support yet.
If you can wait a bit, we will probably have it in a not-so-long future release.

[JamesShenmj]

@Cyan4973 :Thank you for the reply. Please tell me if the new release which support .gz format.
And, how long will I wait, one month, two months, or more longer?
Thanks!

[Cyan4973]

Let's have a soft target for Christmas ...

[JamesShenmj]

@Cyan4973 :Thanks for reply!

[lorenzblass]

A very interesting question. I am on the lookout for a compressor that is faster than zlib but whose output can be decoded by a browser that understands "Content-Encoding: gzip", so Zstandard would be a great option for me (and others, I'm sure) if it met this requirement.

[dracular1983]

@Cyan4973 I tried to compress a gz file using zstd.
I modify the "gz-files" in zlib to support zstd ,such as gzwrite.c ,gzopen.c . And I replace the functions, such as "deflate" ,"deflate_init"，“inflate” in the files.
But finally i got a compressed file with uncorrect format.
is this a wrong way ?

[inikep]

You don't need zstd to compress a gz file because it's already done in zlib. In this thread we are talking about decompression of gz files with zstd. If you are trying to put a zstd stream into a gz file it will not work as there is no decompressor that recognizes this format.

[Cyan4973]

Latest release features __gzstd__, a combined gzip + zstd decoder.

You could test this version and see if it correctly decodes *.gz files.

Note that *.gz are not *.zip files. These are 2 different formats.

[JamesShenmj]

@Cyan4973 Thanks for the great work, I 'll test it in the coming week.
Thank you!

[JamesShenmj]

@Cyan4973 By the way, There are 39 releases in github, there should be many stable releases, How can we know which of them are stable?
Thanks!

[Cyan4973]

All releases are supposed to be "stable".
Prefer downloading the latest one : https://github.com/facebook/zstd/releases/latest