Stockfish: [NNUE] compress network files

[nguyenpham]

The current default network nn-c157e0a5755b.nnue is 21MB, if we compress it, the size is about 10MB, thus 10MB can be saved or the rate is 50%. That is a very good saving in long term, for downloading and storing.

I suggest SF NNUE:

• can run with both non-compressed and compressed network files
• uses zip libraries/utilities (by zlib) since it is more popular and zip/unzip the whole file (unzip totally in memory) for being simple (not in handicraft way, say, divide data, compress, save them by chunks...)
• uses file extension .nnue.zip for compressed network files

[vondele]

allowing for compressed files would add more dependencies to the build... not a good idea IMO.

[nguyenpham]

NN files themselves are dependencies already, we have to deal with them anyway

[mstembera]

Having some dependencies doesn't mean we should add more.

[noobpwnftw]

You can implement a very simple Huffman or LZ codec which is not too long in terms of LOC, self contained and no dependencies needed.

[nguyenpham]

All depends on how we implement it. If we add a 3rdparty library source code (not binary), 1 line or few files are the same. You may consider them (additional codes) are dependencies or not. At the moment, we have a few 3rdparty library source code already: syzygy, some for NN.

[sf-x]

Please don't support internal compression, it prevents using actually good, external compression.

[gonzalezjo]

There is no need. Networks can be shared as .zip files, or through websites like GitHub, which already transmit content using transparent, HTTP-based compression.

[syzygy1]

Buy a bigger hard drive if you need to save 11Mb :-)

[nguyenpham]

Buy a bigger hard drive if you need to save 11Mb :-)

It is not a good suggestion: if Syzygy EGTB wouldn't be compressed or had a worse ratio, it was hard to be that success ;)

[syzygy1]

But nothing is stopping you from compressing the network files. It is just the one you are currently using that needs to be uncompressed.

For TBs this is quite different.

[vondele]

I'll close this for the reasons mentioned.