Zstd: Questions about ZSTD_compressBound() speed optimization
I have two questions about ZSTD_compressBound() speed optimization, hope to get your answer.
1, It seems ZSTD_compressBound() speed optimization only works for ZSTD_e_end end directive (in both single-threaded and multi-threaded compression), is this right?
2, Is it possible to support ZSTD_e_continue / ZSTD_e_flush mode in the future?
Thanks
animalize
All 2 comments
1, It seems ZSTD_compressBound() speed optimization only works for ZSTD_e_end end directive (in both single-threaded and multi-threaded compression), is this right?
That is correct.
2, Is it possible to support ZSTD_e_continue / ZSTD_e_flush mode in the future?
No, it is not possible. The optimization works because when you pass ZSTD_e_end with output size >= ZSTD_compressBound() zstd knows that it has all of the input, and is guaranteed to have enough space to finish the compression.
The optimization avoids copying the input into zstd's internal buffer. When ZSTD_e_continue or ZSTD_e_flush are passed, zstd must copy the input into an internal buffer, because it needs to continue to reference it in future calls to ZSTD_compressStream2().
terrelln
on 24 Sep 2020
I see, thank you for your answer.
animalize
on 24 Sep 2020
Related issues
escalade
路
3Comments
AbdulrahmanAltabba
路
3Comments
TheSil
路
3Comments
scherepanov
路
3Comments
pjebs
路
3Comments
Most helpful comment
That is correct.
No, it is not possible. The optimization works because when you pass
ZSTD_e_endwith output size >=ZSTD_compressBound()zstd knows that it has all of the input, and is guaranteed to have enough space to finish the compression.The optimization avoids copying the input into zstd's internal buffer. When
ZSTD_e_continueorZSTD_e_flushare passed, zstd must copy the input into an internal buffer, because it needs to continue to reference it in future calls toZSTD_compressStream2().