Adios2: SZ compression of 3D single precision data

[tleger26]

I’m trying to use SZ through ADIOS2 to compress some single precision data and have run into an odd issue. When passing the data in 3D matrix form, the decompressed data for some portions of the domain are significantly (orders of magnitude) different from the original data being compressed, regardless of the error bound mode or level specified. If I convert the data into a vector (1D array) first and then pass it into ADIOS2, then the SZ compression works as expected. I’ve also tried using ZFP 0.5.5 and it works as expected for data in both 3D matrix and vector form. I’ve tried version 2.4.0 and 2.5.0 of ADIOS2, along with SZ 2.1.5.3, 2.1.6.0, and 2.1.7.0, all with the same results. Has anyone else tried and/or experienced this behaviour?

[williamfgc]

@tleger26 thanks for bringing this up. Just need a piece of info, is this something that happens outside of ADIOS2 as well? Meaning, can you reproduce this if your data interacts directly with SZ? I want to make sure this is an ADIOS2 issue, not SZ's. If it is, can you provide more info on how you're compressing the data? We don't have much coverage with SZ, as it largely depends on the data itself. Thanks!

[tleger26]

Ok, I'll create a little program to do that and let you know the results. Just thought I'd ask if anyone else had tried/noticed this behaviour (i.e. possibly an already known bug) before doing so.

[williamfgc]

@tleger26 cool stuff, your question is valid and thanks for taking the time to look into this. Looking forward to your findings.

[tleger26]

OK, I wrote a little program to use SZ directly, and long story short it appears the issue lies with SZ. I tried both the Fortran and C interfaces and got the same results. Vectors work fine, but the same data in matrix form (both 2D and 3D) results in significantly larger error beyond that specified by the error mode and bound.

[williamfgc]

@tleger26 thank you very much for speding the time narrowing down this issue. Please let me know if you move forward with opening an issue with SZ, you can always refer to this issue on their page. Thanks!

[chuckatkins]

I'm going to go ahead and close this since it's an issue outside of ADIOS in SZ itself.