Adios2: Dims of size_t potentially problematic in 32-bit systems

[ax3l]

As we discussed last Monday, I think that the definition of adios2::Dims as of

using Dims = std::vector<size_t>;

might be potentially a bit problematic. The problem I am describing here applies to clusters of 32bit systems, such as Raspberry PI clusters, older Nvidia Tegra systems, among others. (Been there with PIConGPU, done that.)

adios2::Dims are used, inter alia, for (global) shapes, counts and starts (/offsets) in user-control. Since these metadata attributes are somewhat loose in ADIOS, user-level concepts like a moving simulation window can at some point overflow the range of size_t for e.g. the start. Similar problems can occur when writing the 1D shape of dataframes/tabular data, such as global particle arrays.

It is therefore probably safer to define this to a fixed integer type, such as

#include <vector>
#include <cstdint>

// ...
using Dims = std::vector< std::uint64_t >;

[williamfgc]

There are lot more issues with 32-bit systems we'd need to tackle (starting with array indexing and memory limitations). The decision was to not dedicate resources and stick with what the STL does in memory which is all size_t based anyways, e.g. vector::size() . The only issue I see to trying to read large datasets from 64-bit systems on 32-bit platforms. Not sure if it's a priority, but I'd would be happy to help anyone willing to maintain and support this.

[ax3l]

The last thing you mention is a third case where this is problematic, yep.

Even if we cannot support internal data structures yet, since they are based on vector::size() as you said, we can at least already fix the adios2::Dims which allows meta-data such as offsets to go very large over time (case: moving simulation window with only partial filling).

I am just thinking: the earlier we adopt the user-facing part of this, the better.

[williamfgc]

Are there 32-bit production systems where we can test at scale?

I am just thinking: the earlier we adopt the user-facing part of this, the better.

Just trying to answer, the better for which production users, systems, codes, etc. in 32-bit.

offsets to go very large over time

Those large offsets can't be used as indices in 32-bit systems, including our backbone library the STL, which defeats the purpose of making a real investment.

[ax3l]

Testing is a good question, do you still have a tiny Titan instance around? That's seriously one of the platforms we should support for teaching.
There are some cloud services though, mythic-beasts, AWS, ...

Those large offsets can't be used as indices in 32-bit systems...

Aren't they just meta-data in ADIOS as well for individual process groups? Offsets would be stored and diff-ed for visualization/processing purposes anyway.

[williamfgc]

Aren't they just meta-data in ADIOS as well for individual process groups?

We need to use (payload) offsets in calls to POSIX read, STL buffer and map access, both of which are size_t based, so a large enough offset from uint64_t dims can't even be used internally. My point is that there is little to no gain to not stick with the standard size_t, especially since the core business of adios has always been I/O performance at scale which translates to 64-bit production environments. The community is welcome to use adios2 in 32-bit systems, but we don't dedicate resources for testing and maintenance at the moment (it's a hard sell in 2019).

[ax3l]

I see. Then let's close this, it's not too urgent and I just wanted to document the use case. I agree with your rationale.