Zig: Data Oriented Design?

[dzmitry-lahoda]

Any special features to allow to use Zig in data oriented stuff?

In short, as I understand this,

If I have struct A with Foo, and B with Bar.
Than if I have C which references A and B then C has Foo and Bar in code (but still references other memory allocated for A and B).
From code perspective, C may hold-have Foo and Bar directly, or reference A and B. Code to get Foo and Bar from C is same in any case.

Like Data-Oriented Demo: SOA, composition or via C++ templates.

I am asking it because Zig is very good, at least looks like that. And build in DOD seems suits it.

[dzmitry-lahoda]

Could I clarify meaning of milestone?

1. Probably Zig will not be introduced with features which prevent clean and nice integration of DOD?

Or,

1. What ever route Zig will go, will look after 1.0+ if DOD can be efficiently (from coding perspective and interaction with other features) integrated into Zig? And if not than do it manually as in other languages (Rust, C#, C++, ..)

[Sahnvour]

I'm also very interested in this as I believe DOD fits perfectly into Zig's philosophy. I have approached this subject in my comment near the end of #383 where @andrewrk showed real interest for.

[andrewrk]

Hi @dzmitry-lahoda, I put this issue as 1.1.0 because it is a broad discussion topic with no actionable content. Are you intending to discuss the Struct of Arrays use case in particular? Are you proposing a specific feature or asking about the existence of a specific feature?

[daurnimator]

Is this a dupe of #1214 ?

[dzmitry-lahoda]

1. #1214 is similar partially, but not exact, at least by first message in thread. As per linked Unnamed-Fields I cannot treat several array heap references same as single array of embedded structs. Also Unnamed-Fields are must for DOD, but not DOD. DOD needs to treat pointers to arrays or heap references as if these are embedded into structs. Similar coding to Unnamed-Fields, but data layout is totally different.
2. #1214 does not mention Data Oriented Design and it is hard to find that it is partially relates.
3. DOD may require special handling of arrays on language level. Let have struct A with i16 and struct B with f32. Then we can have 2 options. First, struct C with A and B and Array of 1000 elements with C. Second, Array of 1000 elements of A and Array of 1000 elements with B. Running loops or other way interacting with first and second should be same or very similar in code.

[dzmitry-lahoda]

https://github.com/ziglang/zig/issues/383#issuecomment-427791824 seems very relevant.

@andrewrk i have look into zig documentation for dod, soa, aos, and found nothing relevant. than i searched dod in issues, but forget to search for soa and aos (at least no issues with such title).

I have asked about awareness and possible future existence of soa/aos/dod like stuff, so I can start learn zig now with chances it will go the road i like.

Seems there are couple of stories relevant, so issue may be closed.

Documentation contains packing and data alignment items (as these are expected for C). These are "part of dod" and seems covered. Thanks.

[JesseRMeyer]

The choice of SoA and AoS can have a considerable impact on assembly output, as well in many cases, overall performance improvements and developer joy as SoA is naturally suited to SIMD.

Consider the following example: https://ispc.godbolt.org/z/OW_WmK from the following discussion: https://twitter.com/pbrubaker/status/1127142640556732417. Arseny has an interesting counter (let the programmer have intrinsic access for optimal performance). JAI at one point had a #SOA compiler directive to force the layout of the struct while preserving access syntax which alleviates the general tire of the a[i].y -> a.y[I] transform when making data layout changes, but lately Jon expressed a different way of handling this but hasn't elaborated any further.

[daurnimator]

On SoA vs AoS, #2255 seems relevant.

[dzmitry-lahoda]

Example

There is Bosque language interesting features:

3.1 Type System
3.2 Expressions (except Collection Pipelining)
3.3 Statements
4.1 (Im)mutable State (part which is not about immutability)

So it has kind of struct inheritance with flattening constructors into one. It uses named updates of data to allow backward compatible code changes of adding new fields. It has some kind of convenience of updating data in bulks and shortcuts for handling option type.

Why these are DOD related?

Such features will allow me to type less and make changes faster and have more reuse with no lost of performance when I try to work with data (like move field from one struct to another, or do updates).

Why these may be related to Zig?

I guess Zig has no story for struct polymorphism fully defined, so Bosque example may help.

Others

I guess these features can be imitated in C++, may be Rust has such stuff either.

I guess Bosque will give APL like features to masses (this may be not related to Zig).