Simd: Alignment of `loadAxB` and `load_splat`

[abrown]

We have been going back and forth about this in e-mail chains and PRs: what are the allowed values for the WAT align modifier when using the SIMD loadAxB (load extend) and load_splat instructions?

The SIMD document doesn't clearly specify the valid values and we should remedy this. The current spec does for existing instructions and reading loadN would lead me to assume the following for loadAxB:

• we ignore the destination size (128 bits) and focus exclusively on the source size of the data--the value to be loaded
• we need to get the byte-width of the value to be loaded (e.g. for Load8x8: 8x8/8 = 64/8 = 8)
• we need to satisfy 2^memarg.align <= [the byte width] (e.g. for Load8x8: memarg.align is in [0, 1, 2, 3])
• this means that in WAT, align can have the values of 2^memarg.align (e.g. for Load8x8: 2^0 = 1, 2^1 = 2, 2^2 = 4, and 2^3 = 8)

Is this understanding correct? I think it intuitively makes sense to have alignment of 1, 2, 4, and 8 bytes for 64-bit values (e.g. Load8x8) because those are the log_2 ways of aligning the bytes; similarly, 128-bit values would have alignments of 1, 2, 4, 8, and 16 bytes. But I want to make this more explicit in the spec or correct my understanding before I go much further. I would also like to confirm my assumption that the Wasm binary encoding will use memarg.align and that the Wasm textual encoding will use 2^memarg.align.

[penzn]

This is a very good point. Normally alignment should follow the source size - as it is supposed to match the store that originally produced the values - but there is no "partial size" SIMD store to match it. It would still make sense to follow the source size though, even if store is a full SIMD store aligned on 16 bytes, taking parts of the value would be aligned on respective size (4 byte lane would be aligned on 4 bytes, 8 byte chunk for load-and-extend would be aligned on 8 bytes and so on).

[tlively]

I don't remember where the previous conversation was, but I believe @penzn suggested using the alignment of the individual source lane value as the natural alignment, so for example i8x16.load8x8_s would have a natural alignment of 2^0=1. Following the rest of the spec, I would not expect overaligned loads to validate, so that would be the only possible alignment for i8x16.load8x8_s. This is what I implemented in Binaryen.

[tlively]

But alignment hints are supposed to be useful for engines to optimize performance. Do any hardware implementations of these operations have alignment preferences?

[penzn]

I think we should treat size of the entire source as natural alignment (8 bytes for "2x" load-extend).

But alignment hints are supposed to be useful for engines to optimize performance. Do any hardware implementations of these operations have alignment preferences?

That is a very good point, I did not want to mention it to avoid confusion 😄 I am not sure it is possible to effectively leverage Wasm alignment (in general, not just SIMD) at this point. When JS or Wasm code sets something in module's memory, it has no way to map integer index into the array to "hardware" alignment of the destination. Filed WebAssembly/design#1319 for this.

[dtig]

The Wasm binary encoding will use memory.align, and the textual encoding will use 2^memory.align. We had a brief discussion about this in a previous PR #107, but it looks like that got merged without us resolving what that should look like for memory operations.

Intuitively though, I think it makes sense to focus on the source values for alignment for both of these sets of operations. On x64, I don't think these have any specific alignment requirements, except when alignment checking in enabled. When this is enabled, then the underlying instructions (PMOVSXBW, PMOVSXWD, PMOVSXDQ) all expect 64-bit alignment. On ARM platforms, I don't think there are any specific alignment requirements that affect performance.

I would lean towards the alignment hints adhering closer to hardware requirements, instead of aligning to individual source lane values.

[abrown]

I would lean towards the alignment hints adhering closer to hardware requirements

@dtig, does that mean that you would want Load8x8 to only have memarg.align values of [0, 3] and not [0, 1, 2, 3]?

Also, I posted some concerns about memory alignment in @penzn's issue, https://github.com/WebAssembly/design/issues/1319.

[zeux]

Just so that I understand this better - is it correct that by default, none of the memory loads in SIMD imply a specific alignment, and thus this only pertains to the valid values of explicitly specified alignment hint? Do existing toolchains emit those?

(worried about this because for my codec, I definitely need unaligned loads in several places to be supported, so want to make sure we don't restrict the spec too significantly in that regard)

[tlively]

Incorrect alignment hints may have performance penalties, but will never produce a crash or incorrect results. Using Emscripten's intrinsics, LLVM knows that the natural alignment of a v128_t is 16, so if you load or store to a v128_t*, the alignment hint on the corresponding v128.load or v128.store will specify alignment of 16. However, the wasm_v128_load and wasm_v128_store intrinsic functions go out of their way to avoid assuming any alignment, so they lower to instructions with an alignment hint of 1 byte.

[zeux]

Incorrect alignment hints may have performance penalties, but will never produce a crash or incorrect results.

Interesting - is this useful for code generation?

Looking at meshoptimizer use cases, the only times where I have to use load_splat, it has the alignment of the element of splat (e.g. load_splat of 64-bit integer uses 8-byte alignment), so as long as 128b loads are unaligned I'm content :)

[dtig]

@abrown, I think it makes sense for these to be aligned with source values, i.e. the maximum alignment should be 64-byte aligned. So for 64-bit alignment, memarg.align values of [0, 1, 2, 3] will not be problematic because these are alignment hints for the engines to use if they choose to. A value greater than 3, though will be a validation failure.

For general usage though, it makes most sense to emit an alignment hint that is in-line with what the hardware would expect when alignment checking is enabled. (This is not enforced by the Spec.)

@zeux, it's probably useful in some cases, but V8 currently doesn't use the alignment hints for SIMD code generation.

[zeux]

@dtig Right I asked because I expected the semantics to be along the lines of "if alignment at runtime is not compatible with the one declared, behavior is undefined".

If the implementations are required to maintain correct behavior, I'm struggling to imagine under what circumstances the codegen can be improved by being given an alignment hint. About the only case I can think of is that if an architecture doesn't support unaligned loads, loads can come with a branch that validates the alignment dynamically; but this seems pretty expensive regardless of the presence of the hint so I'd imagine that an architecture like that would be not-worse always emitting unaligned loads (e.g. on some architectures this would be a few instructions that load two 16b vectors and combine the results).

[tlively]

@zeux see https://github.com/WebAssembly/design/issues/1319#issuecomment-567822478 for an example.

[dtig]

@zeux It's not necessarily a declared alignment though, by design it is meant to be an alignment hint to indicate what the natural alignment of a memory operation should be.

Aside from the example you mentioned, even if the architecture supports unaligned loads, aligned memory accesses are usually faster than unaligned memory accesses. The branch that would decide between unaligned and aligned accesses wouldn't have to be that expensive because the alignment hint is known ahead of time, and it would be a constant so it would be a mod operation on the effective address of the memory operation. It is strictly more expensive than a regular memory access, but it's not obvious to me that this would always be more expensive than unaligned accesses.

Also, for this statement "if alignment at runtime is not compatible with the one declared, behavior is undefined", strictly incompatible values are the values in which the alignment hint is too large for the bit-width of the operation. If the data alignment hint is indicating smaller bit-widths, it is possible for the engine to break this down to smaller memory accesses to be compatible with the alignment hint. I don't know of an engine that does this, but only using this as an example for these still being compatible with different alignment hints so the behavior need not be undefined.

[zeux]

it's not obvious to me that this would always be more expensive than unaligned accesses.

Right, it depends. I had PowerPC in mind where unaligned loads require 4 vector instructions with no branches (ld, ld, lvsl, perm). Of course the branch here would be perfectly predictable even on in-order CPUs so perhaps and + beq + vector load is still a touch faster.

[abrown]

I think I have the answer to my original question and I updated wasmparser accordingly. Should we update this repo's documentation to make the alignment hint more clear and if so, where and how?

I think this conversation has raised an important issue about alignment hints that I will attempt to summarize here but we can continue to discuss in https://github.com/WebAssembly/design/issues/1319:

• @tlively: "alignment hints are supposed to be useful for engines to optimize performance"
• @penzn: "I am not sure it is possible to effectively leverage Wasm alignment (in general, not just SIMD) at this point"
• @zeux: "if an architecture doesn't support unaligned loads, loads can come with a branch that validates the alignment dynamically; but this seems pretty expensive regardless of the presence of the hint so I'd imagine that an architecture like that would be not-worse always emitting unaligned loads"
• @dtig "[the alignment hint] would be a constant so it would be a mod operation on the effective address of the memory operation. It is strictly more expensive than a regular memory access, but it's not obvious to me that this would always be more expensive than unaligned accesses."
• in essence, the alignment hint, designed for improving performance, requires extra work at runtime and we are not completely sure if/how this extra work will actually be a performance improvement
• another approach (mentioned elsewhere) instead of the branch approach is to allow the aligned access to trap on an unaligned memory access and somehow recover from this (which sounds rather difficult on x86 but perhaps someone can show me how it's easier than I think?)

I'm closing this issue but let me know if I pulled your comments above out of context and I will edit.

[lars-t-hansen]

A couple of stray comments:

https://github.com/WebAssembly/design/issues/1319#issuecomment-567822478 points to a case where SpiderMonkey currently uses byte loads to avoid trap overhead of unaligned word accesses on some platforms; this is an example of effectively using the alignment hint, if, as https://github.com/WebAssembly/design/issues/1319#issuecomment-571790065 points out, the compiler usually gets the alignment hint right and a trap would be very expensive.

Regarding using a trap for fixing up an unaligned FP access fault, this was somewhat hairy on ARM-32 Android, though mostly because the Android and Linux kernel headers do not always provide the necessary declarations for getting at register values from the trap handler and there's some variations across multiple OS versions. I've no idea about x86; I would assume this problem would be OS-dependent more than anything.

On ARM-32 we're going to remove the trap handling logic again and use unaligned-access instructions everywhere. If it turns out that those are much more expensive than aligned-access instructions then it seems worthwhile to experiment with a runtime test, as @dtig suggests.

[lemaitre]

I would like to highlight that on recent-ish x86 platforms, the unaligned load instruction has no overhead: meaning that if the address is actually aligned, then it is as fast as with an aligned load instruction.
As ARMv8 has a single instruction handling both aligned and unaligned, the situation here is the same.
In fact, at least since Haswell, Intel platforms are able to load unaligned addresses without overhead (except on page boundary).

So in practice, the compiler could always generate byte-align accesses and still produce optimal performance.

Also, the hint should not have any overhead when it is fulfilled: meaning that the detection of the actual misalignment should not be done in other way than traps (or hardware implementation).
Otherwise, we could easily encounter cases where specify a larger alignment (say words versus byte) actually slows down the code, even if the hint is always fulfilled.
This situation should be avoided.