Virtualc64: Emulamer's Errata Demo does not work in part n. 3

[Alessandro1970]

Emulamer's Errata Demo does not work in test n. 3 (press space to advance), you should see an image and instead there is a distortion.
I read that it could be an "Illegal opcode tweak" but I do not know if it's the same problem that occurs in VirtualC64.
errata.prg.zip

VirtualC64 (part 3):

Vice (part 3):

[puleyo]

I tried this demo, the third step, but also I cannot see the image. The result is the same: the green screen. The first and the second demo work fine.

[rossimariolee]

Also I can not see the last image of this demo and I see a green rectangle made up of strange characters like in the picture above

[Alessandro1970]

This bug had already been evidently met by the programmers of the HOSX emulator:

HOXS 11 March 2012 v1.0.7.4:

1) The application is marked as "DPI Aware". All Windows GUI elements are DPI aware and will render
smooth clear text when used with Windows 7's "make text bigger" option. Only the windowed mode C64
bitmapped rendering remains non DPI aware as with previous versions.
2) Added a work in progress breakpoint listview with context menu options "Show Address" and "Delete
All Breakpoints"
3) Illegal opcode tweak for demo 'Errata' by Emulamer.
4) CRASHFIX: Fixed stack corruption crash when reading G64 files with custom speed zones.
5) CRASHFIX: Fixed application lock up bug while drag scrolling the disassembly window.

[dirkwhoffmann]

We are on the wrong path here. The errata demo uses illegal instructions in the first part which is working well in VirtualC64.

In the failing picture drawing demo, no illegal instruction seems to be used. (If a snapshot is created during picture drawing and this snapshot is loaded into an emulator without illegal instructions registered, it does not jam the CPU. Hence, no illegal instruction is executed in this demo part.)
Some other special, yet unsupported VICII feature must be used in this demo part.

[Alessandro1970]

As we read in INTRO part 3 this demo has the same problem, it should show a green image ...

ruined_art.prg.zip

[Alessandro1970]

Even the last Emulamer demo called "The Conclusion" has problems: it shows the image of the tree but does not show the characters at the bottom of the page.
Conclusion.zip

VirtualC64:

VICE (SC):

[puleyo]

I've tried all three demos:
Conclusion.zip - ruined_art.prg.zip - errata.prg.zip
there must be some trick that prevents VirtualC64 from displaying images properly, because the emulator doesn't stop working.
I assume that for all three demos the problem is the same, I'd like to know exactly the cause of this bug or trick that is ....

[Alessandro1970]

I do not know if it can be some help but I found that if you rename the original file "errata.prg" in "errata.zip", without doing anything else, you get a real file zip compressed with this inside:

errata.zip

[mortinus]

Good morning, any news about the solution of this problem?

[dirkwhoffmann]

Not yet.

Currently, I'm restructuring the VICII code which is pretty time intensive. The latest compatibility improvements came with a performance hit and I'm trying to get up to the old speed by optimizing the overall architecture. As a side effect, I'm trying to make the code easier to understand and therefore easier to maintain.

[mortinus]

Ruined Art run only on vice, strange message on screen ?!

[mortinus]

...nothing special in starion on vice

starion.zip

[UgoCaneFifone]

This group of demo emulamers is a problem ...but I'd like to see them run on VirtualC64 !!!
Some "undocumented" CPU instructions that were once thought to have real random behavior have a well-defined behavior when the BA line rises just before the last instruction cycle, someone told me.

[dirkwhoffmann]

Didn't anybody report that some demo only runs in VICE when glue logic is set to "Custom IC"? I don't find any reference to that any more, but it's a smoking gun ...

[Alessandro1970]

There are too many conflicting news on the web.
David Horrocks managed to make the Ruined Art demo work in his HOXS and the only clue is as follows:
" VIC phase 2 fetch fix for Ruined Art demo "
Then I found this:
" SpriteEnableX fixed
Thinking about Emulamer, I just had to fix the "SpriteEnable"-Stuff (Sprites can have DMA turned on, but still they can be inactive). Vice 2.3 (normal Version) fails at some of those tests, we (and Vice SC) pass all now :)
_(Sidenote: Ruined Art still not working...)_ "

1) Sprite enable tests (1-5): https://sourceforge.net/p/vice-emu/code/HEAD/tree/testprogs/VICII/spriteenable/
2) Instead about the "VIC phase 2 fetch fix", which would solve the problem as David Horrocks wrote, I can not find the test at the moment...

[Alessandro1970]

In addition to " VIC phase 2 fetch fix for Ruined Art demo ", David Horrocks confirmed to me by e-mail (about Emulamer Demos) what had written above Ugo that I had previously written to him (copy and paste to UGO/whatsapp) while I was on vacation and I had a lot of problems connecting to write here.

This the complete answer:
_"One or more of the “undocumented” CPU instructions that were once thought to have random behaviour actual have a well defined behaviour in the case where the BA line goes high just before the last instruction cycle. I can not remember exactly which instructions needed to be to corrected but I believe the VICE test suite (not the Lorenz CPU test suite) does now test for the correct behaviour of these CPU instructions."_

[dirkwhoffmann]

I don't expect the problem being due to illegal opcode emulation. The VICE tests are passed.

I think the problem is related to bank switching. The memory bank VICII sees is controlled by two CIA bits. When these bits change, the bank switches. However, there is some delay between switching the bits and the bank. Also, there is an interesting effect when a C64 with "Custom IC" glue logic is used. E.g., if the bits change from 01 -> 10 or from 10 -> 01, the 1 bits show up first. Hence, VICII sees memory bank 3 for one cycle. I'm pretty sure that this is the trick used by errata to get the correct graphics data. There must also be some effect when switching between banks 2 and 3 though, because this is what errata does when drawing the picture.

[Alessandro1970]

I tried the VICEsc, this morning, one way and another but the result does not seem to change

[Alessandro1970]

In the screenshot I sent you see the difference between Custom and Discrete.
Custom hangs and does not reveal the inscription "THE END".
You have to wait for the end to check when hangs ...
There is however instability

[Alessandro1970]

The ERRATA demo at the 3° step writes: This is the first trick in Ruined Arts...

So Ruined Arts has that so called: "" VIC phase 2 fetch fix for Ruined Art demo ""
If Horrocks wrote the true the solution for these two demos is the finding related to _VIC phase 2 fetch fix_

[dirkwhoffmann]

"VIC phase 2 fetch". Indeed, VICE has two fetch methods:

inline static uint8_t fetch_phi1(int addr) 

inline static uint8_t fetch_phi2(int addr)

Graphics data is fetched in phase 1 and character data in phase 2. Sprite data is fetched in phase 1 or phase 2, depending on their DMA cycle.

Right now, VirtualC64 doesn't care about the fetch phase. Hot lead ...

[mortinus]

Is the problem an "Illegal opcode tweak" ?

[dirkwhoffmann]

Illegal opcodes are not the problem (in the failing demo part, no illegal opcodes are used). It's a timing issue (race conditions between the CPU writing a value into an I/O register and VICII picking up the value).

[dirkwhoffmann]

Just run Errata on VICE and compared the results of fetch_phi1(int addr) and fetch_phi2(int addr). Both functions always return the same value. Hence, the problem must be related to something else.

[mortinus]

...this demo tortures us! if could we find a solution ?

[dirkwhoffmann]

VICII performs several memory accesses. c-acceses, g-accesses, s-accesses and i-accesses (idle). c-accesses seem to be OK in VirtualC64. Next, I'll compare how g-accesses are handled in VICE ...

[dirkwhoffmann]

I'm getting a little bit closer. I've tracked down where VC64 and VICE differ when running this demo and came to that line:

.C:1fa1  4D 1E D0    EOR $D01E      - A:1C X:1C Y:00 SP:f6 ..-..I..   15864248
.C:1fa4  29 FC       AND #$FC       - A:02 X:1C Y:00 SP:f6 ..-..I..   15864252

At the time when the EOR command is executed, VC64 reads 0 from $D01E, in contrast to VICE which reads a value greater 0.
$D01E is the sprite-sprite collision register, so it seems like the demo uses the timing of collision detection to make yet-not-so-decent emulators stumble.

When I detected this, I got back to the sprite test cases and found out that VirtualC64 indeed fails on test case sprite-sprite.prg. Hence, I'll postpose working on the errata bug until the sprite-sprite.prg test case is fixed. With some luck, this might also fix the errata demo bug.

[dirkwhoffmann]

It seems that the demo is performing some tricky sprite collisions in the boundary area (around cycles 59, 60) at exactly the same time as the shift registers of the sprite are loaded with the three data bytes fetched in the DMA cycles. Therefore, several time constraints must be taken into account.

I understand that some demo programmers are eager to write programs that are able to trick emulators and see this as a kind of competition. Since I really don't like cat and mouse games, the Emulamer team can get the trophy. Due to my limited time resources, I will continue to work on the VICE test cases, which seems more fruitful to me.

[dirkwhoffmann]

With a new day comes a new chance. By doing some standard code cleanup, I happened to see that the current implementation of VirtualC64 disables sprite collision detection outside the visible area even if the sprite sequencer is running. And indeed, we're coming closer ...

The result looks like if the picture we want to see is bitwise ORed with the random looking pattern.

[mortinus]

:+1:

[dirkwhoffmann]

Finally ...

There is still an issue with the glue logic stuff. Regardless of selecting "Custom IC" or "Discrete", a CPU jam occurs at the end of the demo.

[Alessandro1970]

It also happens with the VICESC, I have tried repeatedly both with the PC version and with the MAC version and once the CPUJAM happens. Both with CUSTOM and DISCRETE, there is no certainty that it closes without error

[Alessandro1970]

Even the brand new GTK3VICE-3.2 (SC version) goes into cpujam in both DISCRETE and CUSTOM, just try several times to run the demo

[dirkwhoffmann]

"Both with CUSTOM and DISCRETE, there is no certainty that it closes without error"

OK, good to know. So it seems like I can close this issue 😃.

I am still working on cleaning up the code, because some of the recent changes slow down emulation. I.e., the sprite sequencer has to be executed in each and every cycle and not only in the cycles that belong to the visible screen.

[Alessandro1970]

Great result, for me this issue can be closed, given your excellent result.
Thank you

Ps. Does Ruined Arts work in you new version ?

[dirkwhoffmann]

I haven't checked the Ruined Arts demo, yet. To relax, I decided to do some theoretical investigations and to learn something new about color models (which is not my field of expertise). As ever, things get complicated when looking at them in more detail.

[mortinus]

:+1: :champagne:

[Alessandro1970]

Unfortunately, in recent reviews something has changed the results of the third part of this demo. Now a horizontal line appears too.
Is it possible to fix it again ?

[dirkwhoffmann]

Current findings:

Demo got broken with checkin d66c4e0e8b4cff4e01cfa818ee4da9520ca69c6c

Prviously, I had this in beginRasterline():

// Prepare sprite pixel shift register
for (unsigned i = 0; i < 8; i++) {
    spriteSr[i].remaining_bits = -1;
    spriteSr[i].colBits = 0;
}

To get VICE test spritscan.prg to pass, I had to remove line

  spriteSr[i].remaining_bits = -1;

But this causes the the green line to appear in the Errata demo.

Needs further investigation ...

[dirkwhoffmann]

Hard to see what's going wrong here. We need to rerun all sprite related VICE tests first, to see if something else got broken. The Donkey Kong Jr. bug could also be related to sprites ...

[rossimariolee]

Why don't you try to add the " spriteSr[i].remaining_bits = -1;" line again in the version alpha4/5 ?

[dirkwhoffmann]

Adding

spriteSr[i].remaining_bits = -1;

is wrong as shown by VICE test spritscan.prg. There is another bug which was shielded before by resetting the remaining bits to -1.

[dirkwhoffmann]

Background: VICII has a shift register for each sprite containing the pixel data. Once activated, the existing VICII implementation runs the shift register for 24 cycles (because there are 24 pixels in a sprite). It seems that this is wrong and the shift register stops running when it's value matches 0 (this may take less than 24 cycles, but it may also take more than 24 when the register is loaded while running). If I read the VICE code correctly (not so easy), it seems like VICE is doing the same thing. From a computer engineering point of view, it's hard for me to image that this is really happening in a real VICII. If I had to design this chip, I would run it until the next load event happens, because this would require the least amount of glue logic. Anyway, I implemented it that way and it seems to fix this issue, as well as issue #474 (it doesn't fix #473 though):

http://www.dirkwhoffmann.de/virtualc64/VirtualC64_3.2alpha9.zip

It'll be great if someone could confirm this and could run some other tests. Because a change to the VICII code is like an operation on the heart 😷, our "patient" may have suffered other damage in another place 🤒.

[dirkwhoffmann]

Alpha 10 most likely fixes #473 as well:

http://www.dirkwhoffmann.de/virtualc64/VirtualC64_3.2alpha10.zip

[Alessandro1970]

All emulamer's demos work fine.
Great job!

[UgoCaneFifone]

It's ok for me too