Reference (SAE):

vAmiga 馃檲:

This morning, vAmiga looked like this:

I don't know why. Yesterday, I worked on serial transmission and sound only. Somehow, it cured itself over night 馃槼.
The remaining error was due to the handling of registers BPL1MOD and BPL2MOD. Their contents has to be treated as signed 16-bit values, not unsigned 16-bit. With uint16_t, no sign extension takes place when adding to larger values (such as the 24-bit DMA pointers). Hence, the bitplane DMA pointers never decreased, even if BPL1MOD or BPL2MOD were negative.

Hopefully, this fixes other issues as well.
A big thank you goes to @keirf. With his amazing AmigaTestKit and his easy-to-read source code I was able to find the error in no time.
Most helpful comment
This morning, vAmiga looked like this:
I don't know why. Yesterday, I worked on serial transmission and sound only. Somehow, it cured itself over night 馃槼.
The remaining error was due to the handling of registers BPL1MOD and BPL2MOD. Their contents has to be treated as signed 16-bit values, not unsigned 16-bit. With uint16_t, no sign extension takes place when adding to larger values (such as the 24-bit DMA pointers). Hence, the bitplane DMA pointers never decreased, even if BPL1MOD or BPL2MOD were negative.
Hopefully, this fixes other issues as well.
A big thank you goes to @keirf. With his amazing AmigaTestKit and his easy-to-read source code I was able to find the error in no time.