Etcher: Add zstd support

[sedlund]

Please add support for zstd compressed images.

[lurch]

Also previously discussed in #711

[thundron]

Thank you for your request. As @lurch pointed out it was previously discussed about a year ago, so we can at least see if the situation got better wrt dealing with 7-zip

[lurch]

@sedlund Out of curiosity, are you aware of disk images that are being distributed in zstd format?

[sedlund]

@lurch I haven't seen anything offered as of yet - I suppose its a bit chicken and egg now.

I use it for creating SD backup images compressed on the device and then streamed over the network to its final destination. I'm sure you've seen the benchmarks, Zstd is far and away faster in compressing on small devices,

7zip itself doesn't have Zstd support. There is a fork that includes it:
https://github.com/mcmilk/7-Zip-zstd/
There are also some good benchmark graphs showing the advantages, although using an older Zstd 1.2 - it has received many large performance increases since then.

I'm not sure why you'd need 7zip to implement Zstd though. I would think it would be implemented similarly to how lzma was in Etcher.

[lurch]

I would think it would be implemented similarly to how lzma was in Etcher.

A quick search finds https://www.npmjs.com/package/zstd which was last published 4 years ago and https://www.npmjs.com/package/node-zstandard which was last published 3 years ago :slightly_frowning_face:
More chicken & egg? :chicken: :egg:

[sedlund]

@lurch I guess a lot of iteration is not required when you only need to implement 2 methods to hook to an external lib - compress and decompress eh? 😃

[lurch]

Sorry, I was indirectly referring to your "using an older Zstd 1.2 - it has received many large performance increases since then" comment - have these "performance increases" been added before or after these node packages were last published? :man_shrugging:

[jakogut]

I've been looking into this because of the speed of ZSTD's decompression, combined with its good compression ratio, which I think offers a uniquely useful algorithm for compressing disk images and the like.

It looks like the main blocker is the age of existing ZSTD node modules, the lack of a module that implements a node Transform stream, and the small Venn diagram of people who know Node and C++ well enough to make a native module implementing this interface.

Fortunately, compiling libzstd to WebAssembly is pretty straightforward with Emscripten, at which point we can just call exported methods directly from JavaScript. I'm no JS guru, but I know my way around Emscripten, and this is an interesting project to me, so I guess we'll see where it goes.