Zig: SIMD vector type syntax: [|N|]T

[andrewrk]

Currently we have @Vector for this, however, see #5207 and #6209.

Array syntax is [N]T. This is a proposal for SIMD vector syntax to be [|N|]T instead of @Vector(N, T). For example, a vector of four 32-bit integers would be [|4|]i32.

The main motivation for this would be that the compiler needs to be able to talk about primitive types in type names and in compile errors. Without syntax for this primitive type, in order to do this the compiler would introduce a dependency on the std lib such as std.meta.Vector(4, i32) which is verbose and can make compile errors and types more difficult to read at a glance, or it would have to do something like @Type(.{.Vector = .{.len = 4, .child = i32}}) which is even more verbose, making people wonder whether simd vectors really are first-class types in zig after all.

I chose | because it is already associated with bitwise operations, and because it looks OK when symmetrically positioned against the [ and ].

Related:

• Add SIMD Support #903

[tadeokondrak]

The main motivation for this would be that the compiler needs to be able to talk about primitive types in type names and in compile errors.

Note @typeInfo(@typeInfo(@TypeOf(%s)).Fn.return_type.?).ErrorUnion.error_set is already
used for inferred error sets.

I think syntax like v4f32 or f32x4 is easier to read and much better for the common case of non-pointer vectors. @Type/std.meta.Vector is available for any others.

[LemonBoy]

I think syntax like v4f32 or f32x4 is easier to read and much better for the common case of non-pointer vectors.

I like the v syntax, it feels a natural extension of the usual scalar type syntax.
On top of that we should offer a set of "sane" combinations of N and T in std.simd that make sense for the hardware and avoid people running face-first into a (performance) wall.

LLVM covers your ass when working on non-canonical (where T is not i/u{1,8,16,32,64,128}) but every single load/store/op done on such vectors is slow AF since the hardware has no native support for such wonky-sized vectors and the generated code scalarizes, performs the requested operation, masks off the unwanted bits and then re-packs the vector.

[MasterQ32]

I also think we should add a native syntax, although i prefer TxN, so f32x4 is more readable imho, as it still conveys type information first (it's a f32, but 4 of them). But i can understand why v4f32 is preferrable, as it follows the zig array decl: [4]f32 and v4f32 are similar.

another option would be <4>f32, but this would introduce ambiguities

[michal-z]

I think that f32x4 is more readable and easier to write.

[Snektron]

The problem with v4xf32 and f32x4 though is that the compiler still needs to generate calls to std.meta.Vector (or a long chain of builtin calls like with error set returns).

[MasterQ32]

After some discussion with @Snektron we came up with another idea, utilizing already existing features and make people remember less syntax (assimg @Vector(4, f32)):

• [|4|]*f32
• *f32x4
• v4*f32
• <4> *i32
• 4 ** *i32
• *i32 ** 4

Just use the ** operator for comptime repetition to also lift types from scalar to vector type:

T ** N == @Vector(N, T) == std.meta.Vector(N, T)

[LemonBoy]

The problem with v4xf32 and f32x4 though is that the compiler still needs to generate calls to std.meta.Vector (or a long chain of builtin calls like with error set returns).

Why? Once that syntax is adopted for all the Vector types the compiler is free to use that syntax as well.
std.meta.Vector returns v4f32 (or f32v4) vectors with this proposal.

[Snektron]

Why? Once that syntax is adopted for all the Vector types the compiler is free to use that syntax as well.
std.meta.Vector returns v4f32 (or f32v4) vectors with this proposal.

This syntax doesn't allow for vectors of pointers, or vectors of some aliased type.
I probably should have clarified in my original comment, sorry about that.

[LemonBoy]

This syntax doesn't allow for vectors of pointers, or vectors of some aliased type.

I always forget of the vectors of pointers, thanks for reminding me.

[EleanorNB]

I very strongly disapprove of **. Firstly, on values that's an array operator, so it's easily confused; secondly, a SIMD multiplier would then be the only postfix type modifier, and we'd have C-style spiral precedence. Packed-native format is also terrible, because it only covers a subset of valid use cases, so it doesn't actually eliminate any human memory overhead.

A strictly regular type modifier is a necessity in my eyes, and since we don't have a modifier application operator (and we absolutely should not ever add one), another variation on bracket syntax seems like the best option. Andrew's original proposal fits that, as well as being "augmented" enough that it's clear something else is going on.

[Rocknest]

I dont like any of the proposals, bars, x'es, stars don't look good and also confusing, one looks like an array and others like an identifier. I'm fine with status quo, whenever i use vectors i make aliases. If we really really need syntax for vectors i guess this is the least confusing, since it similar to const array ptr:
[4]vec i32

[SpexGuy]

I know we removed it but personally I think @Vector(N, T) is clearer than any of these.

[zzyxyzz]

[4]vec i32 looks nice. Though maybe it should be [4]simd i32? To more explicitly signal that it is an array-like object that is meant specifically for SIMD processing.

I know this is a bit off topic, but "vector" is such an overloaded term in computing, and usually little to do with the original mathematical term to boot.

[LemonBoy]

[N]vec T is inconsistent with the array syntax, here vec applies to the whole thing while other modifiers such as const affect the type. If you want to stretch this syntax you could use something like [N]lane T that makes sense from the simd point of view.

[zzyxyzz]

@LemonBoy, could you clarify? I was thinking of [N]vec as an atomic modifier just like const, [N:0] or anything else.

[zzyxyzz]

Some more syntax variants on a slightly more complex example:

[w][h][4]vec f32

[w][h][4v]f32

[w][h][4 simd]f32

[w][h][|4|]f32

[w][h]@Vector(4, f32)

All of the bracket-based variants have the disadvantage that they only make sense on the inner-most array (you can't really have [w][|4|][h]f32), which is a bit inconsistent. In light of that, I would agree with @SpexGuy that the old @Vector syntax is still best in many cases.

[LemonBoy]

@LemonBoy, could you clarify? I was thinking of [N]vec as an atomic modifier just like const, [N:0] or anything else.

[N]const T is a N-element array of const T values, the whole array is transitively constant too.
*const T is a pointer to a constant value.
Following this logic [N]vec T is a N-element array of vector T (??), hence my suggestion to use the term lane as [N]lane T means a bundle of N lanes of width equal to the one of T.

[zzyxyzz]

Yeah, I guess the associativity is backwards in this case :smile:

[Snektron]

Following this reasoning the modifier could simply be placed right of the array: simd [N]T

[zzyxyzz]

[4x]T anyone?

[kyle-github]

Delurking for a minute.

Is there any projected impact on Zig's use of SIMD vectors from things like Arm's SVE? The examples I have seen of what compilers can do to automatically vectorize normal arrays using tools like SVE and RISC-V's V extension are quite impressive.

[zzyxyzz]

Interesting question. From my superficial understanding of ARM-SVE, it represents a very significant departure from the SIMD paradigm. It is designed to operate directly on large arrays with runtime-known length, rather than manually partitioned fixed-sized chunks. In particular, array length does not need to be a multiple of the native vector size, thanks to the ability to load and operate on incomplete vectors. SVE also relies heavily on a separate bank of predicate registers that don't have a direct counterpart in traditional SIMD.

My cautious conclusion would be that SVE is not urgently relevant to the present bikeshedding session, since we are discussing syntax sugar for a fixed-width SIMD data type. It should also be kept in mind that the availability of SVE-supporting commodity hardware is still pretty much zero (I'm not going to count the Fujitsu A64FX), so introducing special syntax for it may be premature. All in all, it would probably be best to extract this question into a separate issue.

[kyle-github]

Availability, yeah, that is an issue today, but probably not within a year or so. Even availability of Arm servers has gone from zero to lots with AWS being so cheap for Graviton instances. Arm is clearly pushing (we'll see what NVidia does) SVE/Helium everywhere in their next generations of cores. Everything is going to have some form of VLA (variable length array) support.

It was precisely these facts that made me wonder a bit if Zig was skating to where the puck is today and not where it will be in a few years:

• VLA extensions toss out a lot (most?) of the pain of dealing with SIMD.
• Arm is used in supercomputers, cloud instances (AWS), and will be in mainstream laptops shortly (Apple), as well as the usual area of phones where it has more than 99% market share.
• RISC-V is not mainstream yet, but gaining surprising wins and there is only one accepted SIMD extension and that is VLA too.

Where is x86 in this? No idea but with Arm now entering into the Supercomputer 500 list due to SVE... There are so many, many advantages to VLA support.

But I agree that this is a different discussion point. Sorry for the diversion! I am really excited about VLA support in CPUs because of the ability to write code once that just works across a large range of hardware and it means far less support for Intel's idiotic market segmentation by ISA version (try to figure out which AVX512 instructions are supported on which processor!).

I'll go back to lurking :smiley:

[EleanorNB]

@andrewrk There is an ambiguity in the proposed syntax: if the length is the result of a bitwise or, the lexer will need to look ahead to know that the pipe does not pair with a close bracket. We don't have this problem with captures because they can only be alphanumeric, but integers can be arbitrary expressions. We could potentially make use of the unused #, $ sigils, but that would be ugly.

Re: VLA, I think our fixed-length paradigm can be adapted, if we relax the requirement of corresponding strictly to hardware SIMD, like we already do with integers. So, we have a vector corresponding to the size of our problem, which we can make as big as we like, and then the compiler is free to split it up into appropriately-sized chunks. Lane predication could be handled by vectors of bool and overloading of index syntax.

[Snektron]

There is an ambiguity in the proposed syntax: if the length is the result of a bitwise or, the lexer will need to look ahead to know that the pipe does not pair with a close bracket. We don't have this problem with captures because they can only be alphanumeric, but integers can be arbitrary expressions. We could potentially make use of the unused #, $ sigils, but that would be ugly.

Adding a separate token for [| and |] would solve that.