Simd: Memory load / store intrinsics do not mention alignment requirements

[gnzlbg]

Are these instructions aligned or unaligned load and stores ?

That is, can I load a v32x4 from a pointer to some 32-bit value aligned to say a 4 byte boundary ? Or do these need to be aligned to a boundary with the alignment of a v128, let's say 16-byte boundary ?

[penzn]

They allow for optional alignment argument, that is at least how WABT sees it. I am not sure why that is missing from the spec. @tlively, @aretmr, do you have more context?

[tlively]

@dschuff, I was actually wondering about this as well when I was writing tests for loads and stores. iirc, I just copied the other load and store tests which never had interesting alignments. Do we support alignments in LLVM?

[arunetm]

I suggest memory alignment and the actual instructions generated (movups, movaps) to be kept as VM implementation detail not exposed to the programmer. In this case there is no need to have it in the spec.

Performance of unaligned mem access is platform dependent and there are perf penalities archs older than sandybridge, also this can cause higher register pressure. The WASM spec choose not to specify how linear memory maps to virtual mem. Hence VM is in a better position to decide how to handle the mem operations as opposed to the programmer imo.

[gnzlbg]

People often want to read e.g. 4 contiguously stored f32s from memory into an v128. I guess that a VM could track, for every pointer, the size and alignment of the allocation, and use that information when performing vector loads. Doing an "alignment" check at run-time, to decide whether to perform an aligned or unaligned load, is not worth it on any architecture, so I have my doubts that any VM will be efficiently able to perform these optimizations, or even want to incur the overhead of tracking alignment as well.

Particularly because "alignment" knowledge often acts at a distance. For example, whether memory is aligned or not is often known by programmers writing higher-level language code: e.g. one allocates v128s, casts a v128 pointer to an f32 pointer, and "know" that the memory referenced by that pointer allows v128 aligned loads. The actual loads and store can happen in code that's far away, where the VM only sees that an f32 pointer is casted to a pointer to v128 and that's it.

[sunfishcode]

The alignment field is there, in the memarg immediate field in the load and store instructions in the proposal. As with the scalar loads and stores, this alignment field is a hint, and not a guarantee -- all wasm loads and stores always work when unaligned, with any hint value, but may go slow if the hint value is greater than the actual runtime address.

[gnzlbg]

@sunfishcode how does a code generator (e.g. like cranelift) use this information to generate aligned / unaligned vector loads ? EDIT: this is not obvious to me if the alignment is only a hint, and not a guarantee. Arguably in many targets unaligned loads are as fast as aligned loads though..

[lemaitre]

The alignment field is there, in the memarg immediate field in the load and store instructions in the proposal. As with the scalar loads and stores, this alignment field is a _hint_, and not a guarantee -- all wasm loads and stores always work when unaligned, with any hint value, but may go slow if the hint value is greater than the actual runtime address.

If the alignment is a hint and not a guarantee, how can you expect any speed up if the hint is a correct?
You would either need to generate the slow path code (unaligned access), or have a condition to take the fastest considering the hint was actually true.
The former gives no benefit compared to no hint at all, and the latter would be actually slower.

However, if it is a guarantee (meaning if it is not fulfilled, anything can happen: just working or crashing are examples), then you could generate the fastest access considering the required alignment with absolutely no overhead.

If you're not sure about the alignment, just specify the minimum alignment possible: 1.
Any extra information about the minimal alignment of your data should only make things better, not worse.

[binji]

You would either need to generate the slow path code (unaligned access), or have a condition to take the fastest considering the hint was actually true.

The original rationale is described here: https://github.com/WebAssembly/design/blob/73cb0e6e379e473071533f14437e1516dd7e94c8/Semantics.md#alignment

Alignment affects performance as follows:

• Aligned accesses with at least natural alignment are fast.
• Aligned accesses with less than natural alignment may be somewhat slower (think: implementation makes multiple accesses, either in software or in hardware).
• Misaligned access of any kind may be massively slower (think: implementation takes a signal and fixes things up).

[gnzlbg]

@binji that rationale does not really answer the question.

If the alignment is a hint, and the hint is incorrect (the memory address passed is not a multiple of the alignment), and the code generator issues an aligned store to generate a fast path where it should have issued an unaligned store, then the behavior of the program is pretty much undefined (i hope that on most targets you would get a trap).

If the code generator needs to issue a run-time check (which an optimizing code generator might be able to avoid in some cases, but in general, AFAICT, any address can be passed to a load, so it has to be checked at run-time), to verify whether the alignment hint is correct or not, then nowadays the code generator might as well always do unaligned memory accesses which will often be faster than a check + an aligned memory access.

But there is probably a way for code generators to make use of this hint, I just cannot see what that is yet.

[sunfishcode]

@gnzlbg There are two main implementation strategies, and which one is best depends on the hardware and the VM:

• Just use unaligned instructions always (eg. movups). This works great on hardware where unaligned instructions are fast when given aligned instructionsaddresses at runtime. Essentially, the hardware is doing the runtime alignment check for us. Many popular hardware architectures do this today, and it seems to be the trend. Or:
• Use aligned instructions, and have a VM that catches signals and fixes things up in the case where an address isn't actually aligned.

The main use for the hint is: If a hint specifies a lower alignment, we can split the access into multiple accesses and then shuffle the bits into a single value. This can be much faster than taking a signal when using the second strategy above, or when unaligned accesses are slow on the underlying platform (some platforms take a signal and handle it in the OS).

[lemaitre]

@sunfishcode What you describe with signal catching should not be considered valid implementation for a "hint" alignment.
For most people, when you specify a hint, and the hint is not correct, you should not get any slower than without (or at least not by much).
But trapping is very expensive and should not be considered a normal run path.

What you describe is more a guaranteed alignment, but runtime failure of this alignment is somehow recoverable (but really expensive).
Also, the use-case for the hint is also covered if the specified alignment is guaranteed and not a hint.

The only interesting bit with the hint is if you could patch the assembly code after a signal to handle a lower alignment without overhead for the rest of the execution.

[gnzlbg]

@sunfish that makes sense. I'll send a PR to amend the spec (the binary format document appears correct here with respect to memarg).

---

@lemaitre code doing v128 load and stores has to specify an alignment when doing the load and stores. This alignment will often be the natural alignment for v128, which is correct if the memory was allocated for v128 since it is then properly aligned. When the program allocates, say f32s in memory, at a lower alignment, and users want to load from this memory into v128, they can just pass a lower alignment value, in which case the code generator just generates unaligned loads, which are relatively fast (otherwise the user can do unaligned loads until the first alignment boundary, and continue to do aligned loads from them onwards).

The only problematic case is when the user does an v128 load using its natural alignment, from an address that is not properly aligned. WASM has no undefined behavior, so the code has to still work properly somehow, even if it means making it slower.

In practice, I doubt this will be an issue. Users don't write WASM directly. They will use C++, Rust, or some other higher level language. In Rust, users can perform aligned or unaligned loads, we just pass different values for memarg depending on the load type. In Rust, however, aligned loads on unaligned addresses are undefined behavior, so they just can't happen. By default, debug builds check that the address is properly aligned, and release builds can optionally check as well. As long as the Rust code is minimally tested, the generated wasm can use aligned or unaligned loads properly, without really hitting the problematic situation on production. If they hit it, even if the WASM happens to work, the Rust program has undefined behavior anyways, and I'll imagine that the C++ programs will have undefined behavior as well. This means, that LLVM, or whatever optimizer is used, will probably already have optimized the WASM under the assumption that the memory is properly aligned if the user has performed an unaligned load. If it isn't, all bets are off anyways, and the generated WASM after optimizations is unlikely to be meaningful.

[lemaitre]

@gnzlbg I understand all that. But what I meant is: if there is a huge penalty when the hint is not respected, then it should not be called an hint.
The fact that you can recover from misaligned access is fine. But this should really not be viewed as a normal execution path.

As I just said, I would be fine with this if the first misaligned access would patch the generated code to handle a lower alignment for future accesses.
But I don't know if this is something planned, or even desirable for the WASM VM.

Also, no human will write WASM by hand, but some will write using intrinsics (if they are provided at some point).
And the question about alignment will arise again for intrinsics: the compiler might lose information the developer knows.

[gnzlbg]

Turns out that the spec already has load / stores taking memarg. It just wasn't clear to me from reading it that memarg was the alignment, duh, I've updated #34 to make it clear that memarg is an immediate mode argument, and referenced memarg to the scalar part of the wasm spec.

[sunfishcode]

"Hint" is the word the wasm spec uses. In that context it's jargon for "semantically irrelevant". I can see how it can be misleading though.

However, the approach of taking a signal and fixing up the value is valid, and it's even mentioned in the design documents @binji quoted above: "Misaligned access of any kind may be massively slower (think: implementation takes a signal and fixes things up)."

The strategy you mention of patching the code dynamically would be valid too, though I haven't heard of any engines doing that. To my knowledge, the majority of wasm code today is generated by compilers that have relatively reliable information about alignment.