Ppsspp: Warriors Orochi 1 and 2 disappearing enemy officer.

[idespar]

I'm currently using the Android version in my Xiaomi Redmi5. The game works fine except this one thing, enemy officers will disappear, not only graphical since I can't hit them in-game when the bug occurs. I've searched for few hours and got nothing.

I've tried :

• Installing both of the Europe and USA version of Warriors Orochi 2.
• Changing the setting combo.

But I've seen several videos on YouTube using the 0.9.9.1 version PPSSPP and the problem doesn't seem to occur. So I downgraded to that version, is there any other fix?

[unknownbrackets]

Does it not happen in v0.9.9.1? Can you try a few more versions in between?

Does this happen on other platforms, such as Windows?

-[Unknown]

[newbietwo]

Having same problem on Android device (Xiaomi Redmi 4A).
I've tried on pc and it's works just fine.

Actually it is possible to make the disappearing enemies to reappear. You can simply run away from them until his/her mark on map separated with yours, then go back to them. It's a bit tiring tho.

[hrydgard]

If you go into Settings/Tools/Developer Tools and change CPU Core from JIT to IR Interpreter or Interpreter, does it happen then? (you will see reduced performance with these)

[unknownbrackets]

Also, it'd be great to know when this stopped working, even if it's a major version. For example, if it worked in 1.0.1 but not 1.1, that would help narrow it down.

-[Unknown]

[newbietwo]

I try to change the CPU core to Interpreter. Haven't found any disappearing enemies so far. But, this performance reduction is the way much tiring rather than disappearing enemies.

[unknownbrackets]

We're not suggesting that as a fix - just a way to determine where the bug is.

So sounds like this is a bug in arm64jit, most likely.

-[Unknown]

[Dummah]

Hello! Same problem here. PPSSPP 1.4.2 version. Sometimes, even my character disapear, and common enemies. Leaving the area with a horse works, and doesn't took a lot of time. But sometimes, after 2 or 3 hits, enemies disapear again. I tried interpreter and doesn't seem to happen in this Mode. Sorry for my English.

[Saramagrean]

I try v1.0.1-237-g2c51f06(20150403) It's work, character enemies isn't disappear.

Maybe, this problem is start between v1.0.1-256-gf4dfd49(20150406) - v1.0.1-341-g8afab3f(20150406)

[unknownbrackets]

https://github.com/hrydgard/ppsspp/compare/f4dfd49...8afab3f

That's when we introduced the arm64 jit (#7652.) So basically it has never worked on the arm64 jit, which makes it hard to tell what part of it is working incorrectly. That's helpful at least, though, to know that it's not some bug introduced later.

-[Unknown]

[Exiasa]

so.....still not fix :''( ?

[k5nn]

After testing it appears something between 0.9.8 and 1.1 broke

[hrydgard]

Yeah, as commented previously it's pretty clearly a nasty ARM64 JIT bug...

[akelapahu]

Pls fix this, really wanted to play the game...

[hrydgard]

Can someone who sees this try to follow this guide so we can find which area of the JIT the bug is in?

https://github.com/hrydgard/ppsspp/wiki/How-to-find-JIT-bugs

Thanks!

[Dummah]

Sorry, can't find JIT Debug tools, but I enter logging channels on Developer tools and disable everything. The bug is still present. Enable interpreter and reload a save game with the bug and it's still there. Sorry for my English.

[hrydgard]

@Dummah , sorry to get the JIT debug tools you need to install the latest APK from https://buildbot.orphis.net/ppsspp/ (Get the first link you see in the Android column). If you can do the experiment, that will help a lot narrowing down the possible causes of this bug.

[Dummah]

Hi again. I tried the game with Dynarec (JIT) enabled in Developers Tools, but everything disable in JIT Debug Tools. First reload the savegame with the bug. The bug was there. The game works at 3fps with everything disable. I didn't know if the save game keeps the bug, so I started the game from zero. Around 10 or 15 minutes (2 or 3 fps) a boss disapeared again. Walking away and returning solves the problem, but the bug persist with everything disable (all the box checked).

[hrydgard]

Hi Dummah,

That's a bit surprising, but an interesting result. Just to be sure, after every checkbox change did you restart the game? (you can restart + reload save state, but just reload state doesn't reset everything completely).

[unknownbrackets]

And if the answer to that is yes - to confirm, this does not occur:

1. On Desktop (using jit or not.)
2. On Android, using IR Interpreter (instead of jit.)
3. On Android, using Interpeter (likely slowest option.)

Just want to make sure. There aren't many things that would occur with all of the jit features disabled but would not occur on IR Interpreter, so if that's true it might really help narrow it down.

-[Unknown]

[unknownbrackets]

See #11179 - it sounds like this issue is different though, because it doesn't happen in Interpreter. Knowing if it happens with IR Interpreter would help.

-[Unknown]

[RyanBram]

I have tested the first Warriors Orochi game by disabling all of the following JIT features, except one.

• [x] ALU
• [x] ALU_IMM
• [x] ALU_BIT
• [x] MULDIV
• [x] FPU
• [x] FPU_COMP
• [x] FPU_XFER
• [x] VFPU_VEC
• [ ] VFPU_MTX
• [x] VFPU_COMP
• [x] VFPU_XFER
• [x] LSU
• [x] LSU_UNALIGNED
• [x] LSU_FPU
• [X] LSU_VFPU
• [x] SIMD
• [x] Block Linking
• [x] Pointerify
• [x] Static regalloc
• [x] Cached Pointers

Based on my test VFPU_MTX seems the one that causing the bug, although VFPU_COMP seems also contributing to the bug, but I am not sure.

[unknownbrackets]

For arm64jit, that means:

• vmmov
  
  
  
  • Ignores prefixes, but unlikely...
  
  
  • Overlap disabled by mtx equal
  
  
  • Looks identical to arm32.
  
  
• vmmul
  
  
  
  • DISABLES on unknown prefix only, but unlikely...
  
  
  • Overlap disabled by mtx equal.
  
  
  • Looks identical to arm32.
  
  
• vtfm
  
  
  
  • DISABLES on unknown prefix only, but unlikely...
  
  
  • Overlap handled by always using temp regs.
  
  
  • Looks identical to arm32.
  
  

Seems like this is similar to #11179, and probably works with IR Interpreter (but slow.)

I double checked the FMADD encoding, so unless it's a hazard with using Ra == Rd, I'm not sure. But it seems like gcc makes the same choice we do:
https://github.com/gcc-mirror/gcc/blob/c695b63de87707fa0eebb6076ba7999bc5c557e4/gcc/config/aarch64/aarch64.md#L5832

Maybe it's some rounding mode issue, not being respected by certain ops?

-[Unknown]

[hrydgard]

Well, we know that the PSP performs dot products in hardware (and thus matrix multiplications, which are repeated dot products) non-standard-accurately, probably using an align-and-add method which we do not emulate correctly. But generally matrix math in games just isn't very sensitive (except for reproducing replays...), and if it would only be due to that, the interpreter should also fail. So I do believe we have some logical bug... but it's indeed also mysterious why it would differ from ARM32.

[unknownbrackets]

Maybe it's some issue with the reg cache and disabling matrix ops makes us never build up enough float regs to consistently trigger it...

-[Unknown]

[hrydgard]

That could be identified by adding a further checkbox to flush the regcache after every instruction.

(maybe could even trace further by adding options to flush only after matrix instructions, or similar.. dunno. Anyway, did a pull request.)

[hrydgard]

@RyanBram So yeah, please get a new build and try the two new options, together with and without also disabling VFPU_MTX.

[Exiasa]

So....is it fix :3? or i need to wait for 1.9.0version :3?

[unknownbrackets]

The latest git build here:
https://buildbot.orphis.net/ppsspp/

Has a new debugging option we can use to get closer to figuring out the bug. The current option will make PPSSPP slower, but if it makes the problem go away it'll tell us more about the cause.

It's not fixed yet, but we're getting closer.

-[Unknown]

[RyanBram]

@hrydgard

My test results when disabling:

TEST 1

• [x] VFPU_MTX
• [x] GPR Regalloc across instructions
• [x] FPR Regalloc across instructions
  Result: Bug gone

TEST 2

• [x] VFPU_MTX
• [ ] GPR Regalloc across instructions
• [ ] FPR Regalloc across instructions
  Result: Bug gone

TEST 3

• [ ] VFPU_MTX
• [ ] GPR Regalloc across instructions
• [ ] FPR Regalloc across instructions
  Result: Bug still exist

TEST 4

• [ ] VFPU_MTX
• [x] GPR Regalloc across instructions
• [x] FPR Regalloc across instructions
  Result: Bug still exist

@Exiasa The bug is on progress being fixed, but if you really curious, you can try latest build and just disable VFPU_MTX in developer menu. It has little impact in performance, but at least you can already play the game correctly and maybe also helping test the latest patch to ensure the bug has gone in the new version.

[Exiasa]

disable VFPU_MTX and other? wanna check or not :'3

[unknownbrackets]

That's very helpful. So it's either not a regalloc bug, or it's a regalloc bug that happens entirely in the instruction (seems unlikely.)

I tried mucking with it, but still don't see issues in those ops. I'll send a pull with some tweaks though, in case changes have some effect...

-[Unknown]

[unknownbrackets]

It may be worth trying it again (it's only necessary to try it with all disable flags off), but I'm not expecting it to be fixed. I made some optimizations / correctness changes, but it's unlikely it was hitting any of those cases.

-[Unknown]

[hrydgard]

There's still the remote possibility of some ARM64 emitter bug... but it would be pretty weird, they're fairly well exercised by now...

[RyanBram]

Just want to let you know, in stable version the bug only make the enemy characters disappear, but since I tried v1.8.0-252-g47eb7d4a5 the ally characters also disappear.

If I remember correctly the bug only makes the enemy characters disappear, not ally characters.

@Exiasa To play correctly, just go to Settings -> Tools -> Developer Tools -> JIT Debug Tools and check the VFPU_MTX to disable it.

Leave other features unchecked.

[hrydgard]

That's odd. 252 is before unknown's latest changes so that difference must be something different ,and could be interesting. Could you try a few intermediate builds between master and 252 to try to narrow that down?

And you're sure it's not kinda random if the allies appear or not?

[RyanBram]

I did playtest again with same build to ensure I saw it correctly. And yes after trying it again the character ally also disappear randomly like the enemy characters. I try by not doing any action moves to ensure that the enemies don't disappear because they are dying from my attack. And yeah, they are disappear randomly without I am doing any action.

I dont't have much knowledge about game programming, but it seems with so many characters on screen, the game tries to hide characters that don't appear on screen to clear memory usage. But somehow the game falsely hide the characters that still appear on screen. I saw some enemy characters that approach my characters from bottom screen will suddenly appear.

[hrydgard]

And you're sure that doesn't happen the same on stable (1.8.0)? Any chance you can do what I asked and try a few intermediate builds and see when it starts happening?

[RyanBram]

I tried the 1.8.0 from Playstore and the ally characters also disappear. Maybe because I only focus in attacking the enemies, I didn't notice if the allies also disappear.

So it means, even in 1.8.0 stable build, the ally characters also randomly disappear as the enemies.

Sorry for inaccurate report.

[hrydgard]

Okay, good, then at least we're not tracking two issues here. Thank you for making the extra testing effort.

[hrydgard]

Not sure what the next testing step might be. Thinking of splitting up VFPU_MTX into its three operations so we can test them one by one... getting a bit desperate here though, the code looks good and I don't understand why this would happen only on ARM64.

[RyanBram]

For everyone who read this and want to see Warriors Orochi getting fixed, I want to let you know how I reproduce the bug.

1. Play Warriors Orochi Story Mode using SW team, then choose SW Chapter 1 Battle of Jing Province

2. Go to the enemy fortress in the center of map and try to beat the commander there. See if the enemy characters disappear randomly or not.

3. If you can beat commander in the center fortress, go to the east fortress to help Guan Ping from being ambushed. This is where you'll find so many enemies and allies in one screen. Find out if enemies and allies randomly disappear or not.

4. Usually step 3 is enough. But if somehow you can manage to go so far, please try to finish this Chapter 1 by beating Cao Pi. See if the enemies or Cao Pi himself will randomly disappear or not.

If you manage to find the bug like me, please try to go to Settings -> Tools -> Developer Tools -> JIT Debug Tools and check the VFPU_MTX to disable it. Leave another options unchecked and try repeat step 1 until 4 above. See if the bug has gone or not.

I hope that with more people test this bug, the report will be more accurate to avoid my own mistake if try it alone.

For @hrydgard and @unknownbrackets , I have free time because of school holiday, so I can test any fix that you made. But if finding the bug is so hard, maybe the temporary solution is just by automatically disabling VFPU_MTX for Warriors Orochi in future build although this is not an ideal solution.

[Exiasa]

well maybe i just need to play WO game on PPSSPP 1.0.0 version...no Alie/Enemy Char missing there

[hrydgard]

@Exiasa this does not affect the PC version. Only ARM64 based devices. And only versions from after the JIT for ARM64 was introduced.

[RyanBram]

@Exiasa
Why you need to bother to go to 1.0.0 just to play Warriors Orochi correctly? By doing that you will missing most bug fixes for other games. Are you sure you want to use PPSSPP just for playing Warriors Orochi on Android?

I will still use the recent 1.8.0 build because:

1. It is so much stable and responsive
2. It has many fixes for other games
3. It already has solution for playing Warriors Orochi 1 and 2 correctly by disabling VFPU_MTX (you can set this feature only for Warriors Orochi 1 and 2, while other game can use general settings)

So, if you just want play PSP games: Version 1.8.0.is better choice than 1.0.0 on Android, and...

... if you want the developers to fix the bug, you can help them by testing the 1.8.0 and future builds.

Playing and bug testing, and you can have both advantages by always try the latest build.

Just my 2 cents.

[Exiasa]

:o wah~ well ok

[unknownbrackets]

So it sounds like there has been no change, even after v1.8.0-256-gf687371cd, right?

Yeah, at this point I feel like an arm emitter bug is most likely. We can break up MTX into three separate settings, it'd also help #11179 if it's not the same bug. I feel like vtfm is the most likely.

I also think if we did DISABLE on vtfm and vmmul, it probably wouldn't be that awful for perf. I really doubt vmmov is broken.

-[Unknown]

[hrydgard]

I'm leaving disabling the ops as a very last resort. That pull request I just added does the split so that testing can continue and at least tell us which op is at fault...

[hrydgard]

Please re-test once that build is up, check the new options VFPU_MTX_VTFM, VFPU_MTX_VMMUL, VFPU_MTX_VMMOV one at a time and see which one fixes it. VFPU_MTX_VMSCL can be ignored.

[RyanBram]

PPSSPP v1.8.0-258-g9450c1358

Test Results:

TEST 1

• [x] VFPU_MTX_VTFM
• [ ] VFPU_MTX_VMMUL
• [ ] VFPU_MTX_VMMOV
  Result: Bug still exists

TEST 2

• [ ] VFPU_MTX_VTFM
• [x] VFPU_MTX_VMMUL
• [ ] VFPU_MTX_VMMOV
  Result: Bug gone

TEST 3

• [ ] VFPU_MTX_VTFM
• [ ] VFPU_MTX_VMMUL
• [x] VFPU_MTX_VMMOV
  Result: Bug still exists

So it seems you can check VFPU_MTX_VMMUL to find the bug.

[hrydgard]

Thank you for your testing @RyanBram !

[hrydgard]

VMMUL is the one that allocates the most registers, so that increases the possibility for a regalloc bug due to how many registers it allocates internally... but a very similar allocator is used on ARM 32-bit which is not affected, so that still kind of points to an emitter bug.

[hrydgard]

New experiment committed, hopefully this is the one... (reduce precision)

[hrydgard]

In the latest build, this should now be fixed. Can you confirm @RyanBram ?

[RyanBram]

I have tested the last build without disabling any JIT options. The result is the annoying bug when enemy officers suddenly disappear when I was attacking him is gone. But I can still see other enemies randomly disappear. It is a bit different with previous bug. Previously, most enemies or allies will randomly disappear even though I don't do any action. Now in PPSSPP latest build, the enemies that I still attacking or enemies in the center of screen don't randomly disappear, but enemies in the corner of screen that looks not important still randomly appear and disappear. I am not sure it is the expected behaviour in real PSP or it is by design to free up memory. In the current condition, I can still finish a chapter without problem, because I can defeat enemy officers without interrupted with random disappearance even though other enemies onscreen that not engaged in battle with my character still randomly appear and disappear.

I hope somebody else can test it and reproduce my experience, because I am afraid that this is only my subjective personal perception as examining many enemies in one screen that enter and exit the screen in a fast battle scene is little bit tricky.

I will try it again with stable version to confirm about this.

[hrydgard]

Okay. It may still be a bit iffy, but what I really want to know if it's the same with or without the option now.

I know we still don't implement vmmul 100% like the real thing, which may affect the enemies disappearing/reappearing too much... but the important thing right now is that there should be no remaining bug unique to ARM64.

Hopefully the game is at least playable enough.

[hrydgard]

It's very likely that any remaining issues here will be fixed once we've got the issues discussed in #12082 under control.

[HeriKuen]

Wariors Orochi 2 is one of my favorite games but why is there a problem with ppsspp with a 64-bit structure and runs smoothly on 32-bit.

[RyanBram]

Which 64 bit version do you mean?
If it is for x86 system, such as Windows, Linux, or MacOSX, I don't see the problem.

If it is 64 bit ARM, then the problem is because there are little problem in 64bit ARM JIT in PPSSPP. Unlike most of other component, JIT cannot just being ported from one platform to another, it needs to be re-written that makes sometimes there are problem found in one platform, but not others.

The recent build of PPSSPP seems fixes all of the issues.

[HeriKuen]

Oh, I mean, ARM 64. A similar thing happened to Tekken 6, sometimes the player legs vibrate themselves.

[unknownbrackets]

Do the far away characters not disappear using 32-bit ARM but do in 64-bit ARM? Or is the behavior the same?

Note: they should not disappear entirely in the latest git build or in v1.9.4, per the latest comments.

-[Unknown]

[Panderner]

same happens on latest build? but the video where the enemy officers disappears?

[hrydgard]

It's unclear how similar the current behavior is to the real psp. Nothing should have changed after RyanBrams's post above.

[Panderner]

idk. Can we post the video where the enemy officers disappears

[Panderner]

No feedback? i can't produce this issue on latest build.