Reicast-emulator: Fix Shenmue 1/2 crashes [$150]

[twinaphex]

The goal of this bounty is to get the game-breaking crashes fixed that happen in both Shenmue 1 and 2.

'Several major issues in self-modifying code detection (Shenmue 2).'

Talking specifically about Shenmue 1 right now -

Both dynamic recompiler and generic recompiler crash at several parts of the game. With dynamic recompiler, the game crashes directly after the intro when you get out of bed and the loading screen gets shown. With generic recompiler, the game crashes later on when you leave the house.

Apparently these are regressions since earlier versions of nullDC did not have these crashes - Shenmue Dojo is still having to go back to older nullDC versions in order to have this game working crash-free.

http://www.shenmuedojo.net/forum/viewtopic.php?t=46995

Apparently this version did not have these issues. Not sure if this is a solid lead on uncovering the regression and fixing it in dynarec.

Anyway, I am adding a $100 to this bounty.

EDIT: Might want to look into the Bountysource integration, this is the URL of the bountysource bounty -

https://www.bountysource.com/issues/59486961-shenmue-1-2-crashes

---

Did you help close this issue? Go claim the $150 bounty on Bountysource.

[rcaridade145]

One thing i believe we are doing wrong on the mmu side (https://github.com/fail0verflow/switch-linux/blob/master/arch/sh/include/cpu-sh4/cpu/mmu_context.h for reference) is not marking the dynarec as dirty when MMUCR AT states changes. @skmp you're more familiar with the codebase, did i miss something?

[AbandonedCart]

@rcaridade145 Why not give it a try? If it works, you could pack it up as a PR and cash out ;)

[rcaridade145]

@LoungeKatt I'm taking baby steps :) First thing i did was updating xbyak and testing some games reveal no problems. Need to test Shemnue.

[rcaridade145]

@twinaphex @LoungeKatt I updated xbyak on my fork of libretro-reicast and tested it also on the upstream project (Makefile unchanged)

libwebsockets test server - (C) Copyright 2010-2013 Andy Green <[email protected]> - licensed under LGPL2.1
Initial logging level 7
Library version: 1.3 c11b847
IPV6 not compiled in
libev support not compiled in
 static allocation: 4488 + (16 x 1024 fds) = 20872 bytes
 canonical_hostname = rui-desktop
 per-conn mem: 184 + 1554 headers + protocol rx buf
 Listening on port 5678
Malformed entry on config - ignoring @ 41(ElfFile)
Malformed entry on config - ignoring @ 44(ta.HashCheckFile)
Malformed entry on config - ignoring @ 45(ta.HashLogFile)
Found 10 matching FB configs.
Chosen visual ID = 0x24
Loaded /home/rui/.local/share/reicast/data/dc_bios.bin as bootrom

Loaded /home/rui/.local/share/reicast/data/dc_flash.bin as nvram

Using Recompiler
Auto-selected audio backend "alsa" (Advanced Linux Sound Architecture).
Initializing audio backend "alsa" (Advanced Linux Sound Architecture)...
recSh4 Init
Freeing fpcb

     CodeCache addr: 0x55d41723e000 | from: 0x55d41723e000 | addr here: 0x55d41723c2a0
GL context version: 3.1

Session 1 has 1 track(s)
Saving  Track:  1  Type: Audio/2352  Pregap: 150  Size: 300     LBA: 0       
Track position: 352800

Session 2 has 1 track(s)
Saving  Track:  2  Type: Mode2/2336  Pregap: 150  Size: 340368  LBA: 11700   
Track position: 1408800
gdrom: Opened image "/home/rui/Jogos/Dreamcast/Shenmue_USA_DC-KALISTO/KAL-SMU1.CDI"
Initialising empty vmu...
Sh4 Reset
libevdev.so: cannot open shared object file: No such file or directory
WARNING: libevdev is not available. You'll not be able to use button names instead of numeric codes in your controller mappings!

evdev: Trying to open device at '/dev/input/event0'
evdev: open: Permission denied
evdev: Controller 2 disabled by config.
evdev: Controller 3 disabled by config.
evdev: Controller 4 disabled by config.
Legacy Joystick input disabled by config.

[twinaphex]

Very cool, you haven't been able to experience a crash yet when venturing further?

[rcaridade145]

Need to play a bit more :)

[rcaridade145]

@twinaphex @LoungeKatt @skmp Well played a bit more. Promised to help an old lady and gave tofu to a kitten :+1:

[twinaphex]

Awesome. Just out of interest, does this fix the crash on the second stage of Marvel Vs Capcom 2 too? Would be interesting if this fixes more games.

[rcaridade145]

@twinaphex i can't test that game. I have it in cdi but it seems reicast does not like this version. I'd be nice if someone could test a 32 bit build to see if it fixes some issues old version had with xbyak32

[twinaphex]

OK, is this the version I should be checking out?

https://github.com/rcaridade145/reicast-emulator/commit/48d4d5bd7aacfbbea67c0fd9c7c042f8423a549a

Or have there been alterations made since that fix the Shenmue 1/2 crashes? I tried checking out that latest commit but I can still get the crashes in Shenmue 1 when trying to walk outside the dojo.

I was trying to see if my copy of Marvel Vs Capcom 2 would work with this version, but it seems that further alterations are needed before this can have any effect.

Anyway, even if the bugs with Marvel 2 are still there, no big deal, this bounty was only for getting Shenmue 1/2 to run without all the crashes, so as long as that part is fulfilled, I have no issue with considering this issue finished. I am just merely curious.

[twinaphex]

https://github.com/reicast/reicast-emulator/issues/107

Can you try Shenmue 2 really quickly too?

For instance, try to replicate the crash this guy experienced and see if that no longer happens -

"Shenmue 2 in game after intro video and having photo taken then after arm Wrestle with Mexican man walk towards town and game FC and returns to path directory. I had to disable dyranec initially to get the game to load then once game loaded I Saved re enabled dyranec to speed up game..I thought I'd try a load screen just before the Mexican arm wrestling man without disabling the dyranec and it didn't crash??!! Hope this makes sense"

[rcaridade145]

@twinaphex No that is my latest version. Basically Ubuntu 18.04 reicast upstream with soft renderer with those updates.

[twinaphex]

Hmm, the crashes still happen for me when I backport that to libretro reicast.

Can you try with these VMUs maybe in libretro-reicast?

https://anonfile.com/xbN6Qee5b3/vmus.zip

Try the save in Shenmue 1. This should be for the Shenmue 1 PAL Version (GDI).

I'm testing this on Windows 10, 64-bit build of RetroArch and 64bit Reicast core.

[twinaphex]

This is the error I still get -

https://hastebin.com/ofovomuhun.md

[rcaridade145]

@twinaphex I'll try it later today. The compiler is MSVC?

[twinaphex]

No, mingw.

[AbandonedCart]

@rcaridade145 Nice work so far. This has been an issue for a long time.

[rcaridade145]

Tried today libretro with my changes and got the crashes. On Shemnue it crashes when loading Ryu's room. Tried to merge changes from reicast makefile but to no avail. Generic recompiler due to ifdef also seems to not be used even if it is set to 1 on the makefile. The quest continues..At least it works on reicast upstream.

[twinaphex]

Generic recompiler can be switched to at runtime by going to Quick Menu -> Options and changing CPU from 'dynamic_recompiler' to 'generic_recompiler'.

Even with generic recompiler, Shenmue 1 will crash, but only later on when you try to leave the house after the intro. With dynamic_recompiler it happens a lot sooner and it happens at the first loading screen directly after the intro.

The big change between libretro and 'reicast upstream' is that the libretro core gets compiled as a dynamic library with -fPIC enabled. It has to because it's a dynamic library. Apart from that, if there are still discrepancies in the code between the two that we could change back in Reicast libretro, I can certainly do so.

I can only consider the issue fixed unfortunately when it works in libretro Reicast as well.

[twinaphex]

I'm trying to rebase most of the code in core/hw/sh4, core/hw/holly and core/hw/aica back to upstream code in hopes it will fix something. They were more or less inconsequential changes anyway that likely didn't result in any real performance improvements anyway, or just done for code syntactical reasons.

[twinaphex]

How can we actually find out what Reicast crashes on since it hijacks the exception handler? How do you debug something like this without being able to use gdb, or is there a way to bypass this exception handling bypassing?

[twinaphex]

This is the part it crashes at -

Unimplemented opcode: 00004E49 next_pc: 0C4223ED pr: 0C4223E3 msg: Unknown opcode
[libretro INFO] Fatal error : iNimp reached

Does this ring any bells?

[skmp]

o/ @rcaridade145. I'm late to the party, it's been a busy week. How are you trying to replicate this issue?

Afair, this is some SMC-related mess, at least for Shenmue and just updating xbyak shouldn't fix it. These issues are also known to be hard to replicate. @MrPsyMan do you have some time test around this?

IIRC Shenmue (and a few other games) do SMC without clearing the cache at all, and that is the issue.

Does it work with unstable opts turned off?

https://github.com/reicast/reicast-emulator/blob/master/core/hw/sh4/dyna/driver.cpp#L126 should force smc validation for all blocks

[skmp]

Also, does this happen on arm and x86 or just on arm?

x86 smc detection is much more complete than arm smc detection

[rcaridade145]

@skmp I justed got the master as zip updated xbyak and on shell/linux did make. I did not change any settings. Then i tried to replicate this on libretro. Is that - unstable opts - true by default on reicast x64 for linux?

[rcaridade145]

@twinaphex On one of the many tests i did yesterday night it crashed @ 0x8C00XXXX . I believe it was when i set GENERAL_COMPILER:=0. One thing that striked me was different was the use of -fsingle-precision-constant on more archs on reicast itself than on libretro so i "ported" the changes manually (not on my branch yet) so i could have a more consistent set of flags being used between both.

[twinaphex]

Also, does this happen on arm and x86 or just on arm?

We are testing the x64 dynarec here and the output target is an x86 x64 build.

Also, in reference to this -

settings.dynarec.unstable_opt

I believe we always set this to false right now in the libretro port.

[rcaridade145]

@skmp @twinaphex this is what i'm getting

Get MemPtr unsupported area : addr=0x0
Get MemPtr unsupported area : addr=0x18
Sh4: i-cache invalidation 00000016
@@  ngen_ResetBlocks()
Freeing fpcb
recSh4:Dynarec Cache clear at 00000016
Get MemPtr unsupported area : addr=0x80000080
Get MemPtr unsupported area : addr=0x800000A8
Get MemPtr unsupported area : addr=0x800000BE
Get MemPtr unsupported area : addr=0x800000C6
@@  ngen_ResetBlocks()
Freeing fpcb
recSh4:Dynarec Cache clear at 8C0000DE
Get MemPtr unsupported area : addr=0xC5D89ED3
iNimp 0D63
terminate called after throwing an instance of 'SH4ThrownException'

Is that section not the bios? Wonder why the unsupported warning

[twinaphex]

@skmp @rcaridade145 Is this related?

https://github.com/reicast/reicast-emulator/blob/master/core/linux/common.cpp#L91

Does this need to be implemented for x64 in order to support SMC?

[skmp]

Does libretro have https://github.com/reicast/reicast-emulator/commit/e512b95445ed2f3509e7afc01b7664c3042e6376?

I don’t know if the x64 backend actually implement the dochecks flag

[skmp]

Yeah, https://github.com/reicast/reicast-emulator/blob/master/core/rec-x64/rec_x64.cpp#L160 Does not implement the force_checks option. That’s the SMC validation logic for the fallback path. See how the x86 does it, it’s a trivial implementation.

Also not sure if the generic jit implements it. It is less sloppy than the x64 jit, but still it got written over two sleepless nights so .... here be dragans

[twinaphex]

Well, if we can correctly assume that Marvel Vs. Capcom 2 and Shenmue 1's crashes are due to the same issue (SMC), switching to the generic JIT fixes the crash/hang on the second stage in Marvel Vs. Capcom 2 when it happens. However, it cannot fix all the crashes in Shenmue 1. You can get in further with generic JIT vs. the real dynarec, but crashes can and still will happen at later parts.

[twinaphex]

https://github.com/reicast/reicast-emulator/blob/master/core/rec-x86/rec_x86_driver.cpp#L324

It's not clear to me what I should backport to x64 based on this, this is the only part (apart from the profiler part, but I think we can easily ignore that for now) where force_checks is used.

[skmp]

Rec-cpp also has no smc implementation, but the cache is small so the issue is far less visible

[skmp]

Yeh, just port the DoChecks function and it should be alright

[twinaphex]

You mean CheckBlock? There is no DoChecks function.

Or do you mean DoCheck from sh4/dyna/driver.cpp?

Either way, I think I best leave this to either you or @rcaridade145. I don't understand this code well enough to do that.

[skmp]

Yes. w/e it is named nowadays.

On Sun, 17 Jun 2018 at 08:09, Twinaphex [e-mail] wrote:

You mean CheckBlock? There is no DoChecks function.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reicast/reicast-emulator/issues/1167#issuecomment-397857268,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYAMpObH3xLcezmnHowfZexCUoqCZDAks5t9fKhgaJpZM4UhvOn
.

>

~skmp

[rcaridade145]

@skmp but that does not explain why i can run on amd64 on reicast master and not on libretro

[skmp]

How do you build reicast? I think the makefile defaults to 32 bits?

On Sun, 17 Jun 2018 at 15:15, rcaridade145 [e-mail] wrote:

@skmp https://github.com/skmp but that does not explain why i can run
on amd64 on reicast master and not on libretro

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reicast/reicast-emulator/issues/1167#issuecomment-397878425,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYAMg7Jg7b0EodfeWxi0VSzzuNNpFl_ks5t9laPgaJpZM4UhvOn
.

>

~skmp

[rcaridade145]

@skmp

rui@rui-desktop:~/Transferências/reicast-emulator-master/shell/linux$ readelf -a reicast.elf | more Cabeçalho ELF: Magia: 7f 45 4c 46 02 01 01 03 00 00 00 00 00 00 00 00 Classe: ELF64 Dados: complemento 2, little endian Versão: 1 (current) OS/ABI: UNIX - GNU Versão ABI: 0 Tipo: DYN (ficheiro de objecto partilhado) Máquina: Advanced Micro Devices X86-64 Versão: 0x1 Endereço do ponto de entrada: 0xf0d0 Início dos cabeçalhos do programa: 64 (bytes no ficheiro) Início dos cabeçalhos de secção: 30824208 (bytes no ficheiro) Bandeiras: 0x0 Tamanho deste cabeçalho: 64 (bytes) Tamanho dos cabeçalhos do programa:56 (bytes) Nº de cabeçalhos do programa: 9 Tamanho dos cabeçalhos de secção: 64 (bytes) Nº dos cabeçalhos de secção: 38 Índice de tabela de cadeias da secção: 37

[twinaphex]

Does a similar ifdef need to be added too for 64bit?

https://github.com/reicast/reicast-emulator/blob/master/core/windows/winmain.cpp#L138

[rcaridade145]

@skmp I don't think the issue is related to SMC. As on my previous comment the compiler is the 64 bit. If the issue happened to be on SMC from what i understood both the generic and x64 would crash at the same point and they do not. If the upstream project runs ok, my opinion is that it should be down to compiler flags or some issue with delay slots (in terms of rec-cpp). Because rec-x64 dos issue a invalidate block at that stage but at the wrong address.
And on the loop as far as i could tell nothing happens if at the end of the 448 cycles you end up on a delay slot.

@twinaphex i would not add it.

[skmp]

No, they would not. Rec cpp has 8k active blocks, x64 has 128k or so. It is more complicated than you think.

You know, you can trust me a bit, i’ve only written 95% of the core code and
been debugging this codebase for 15 years now.

It is not compiler flags or something similar. I broke those game on
purpose by removing SMC to make things faster.

On Mon, 18 Jun 2018 at 12:26, rcaridade145 [e-mail] wrote:

@skmp https://github.com/skmp I don't think the issue is related to
SMC. As on my previous comment the compiler is the 64 bit. If the issue
happened to be on SMC from what i understood both the generic and x64 would
crash at the same point and they do not. If the upstream project runs ok,
my opinion is that it should be down to compiler flags or some issue with
delay slots (in terms of rec-cpp). Because rec-x64 dos issue a invalidate
block at that stage but at the wrong address.
And on the loop as far as i could tell nothing happens if at the end of
the 448 cycles you end up on a delay slot.

@twinaphex https://github.com/twinaphex i would not add it.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reicast/reicast-emulator/issues/1167#issuecomment-398010568,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYAMiH_HIr9jdcjLO1SPAskxIqNlhBLks5t94BJgaJpZM4UhvOn
.

>

~skmp

[rcaridade145]

@skmp I did not meant to be rude in any way. If i did that i'm sorry.
And do i have a lot of respect for the work that has been done. I'm just trying to understand why reicast ok and libretro nok. When i mentioned flags i was referring to libretro. Reicast uses f-single-precision float libretro does not.

[skmp]

No offense taken. single precision floats have never caused a hard core
crash, so that is extremely unlikely. Shenmue broke before I switched to
single precision. We already have an smc workaround for shenmue, even for
the psp version.

On Mon, 18 Jun 2018 at 13:04, rcaridade145 [e-mail] wrote:

@skmp https://github.com/skmp I did not meant to be rude in any way. If
i did that i'm sorry.
And do i have a lot of respect for the work that has been done. I'm just
trying to understand why reicast ok and libretro nok. When i mentioned
flags i was referring to libretro. Reicast uses f-single-precision float
libretro does not.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reicast/reicast-emulator/issues/1167#issuecomment-398018783,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYAMsiLKvtugJXR0yocICvA1FL8d2XHks5t94k7gaJpZM4UhvOn
.

>

~skmp

[twinaphex]

Hopefully we can reach the consensus now that what @skmp suggested earlier (honoring the force_check bits in the Compile function of the x64 dynarec) should be honored in order to fix this issue. Let's move past the whole red herring of 'I didn't have to do this for my binary', point is that it should be done anyway and that it cannot possibly work right now reliably without at least adding this in. You can use the 32bit x86 version of DoChecks as a base for this, the emit functions just have to be converted to the xbyak equivalents and you might need to ask @skmp about which parameters you need to feed to the DoChecks function.

But this is basically it. I think after adding this, the crashes in Marvel Vs. Capcom 2, Dead or Alive 2 LE and Shenmue will all be fixed in one swoop. So it is well worth it. If I could do it myself I would right now. So hopefully you can finally carry this over the finishing line now @rcaridade145 .

[rcaridade145]

@skmp Thanks for your input. One more question. You say that rec-cpp has 8k cache. If a block inside that cache becomes invalidated and the block size is X what must happen if the new block size is Y and X!=Y.

@twinaphex :)

[skmp]

Blocks are variable length and not reused. A cache reset happens when we
run out of block descriptors

On Mon, 18 Jun 2018 at 16:05, rcaridade145 [e-mail] wrote:

@skmp https://github.com/skmp Thanks for your input. One more question.
You say that rec-cpp has 8k cache. If a block inside that cache becomes
invalidated and the block size is X what must happen if the new block size
is Y and X!=Y.

@twinaphex https://github.com/twinaphex :)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reicast/reicast-emulator/issues/1167#issuecomment-398065721,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYAMo48O6tANJp9GQURpkQkfjmuYKjFks5t97O3gaJpZM4UhvOn
.

>

~skmp

[AbandonedCart]

I know I'm a bit late to the party, but if it hasn't been implemented yet...

https://github.com/reicast/reicast-emulator/blob/master/core/rec-x86/rec_x86_driver.cpp#L270
https://github.com/reicast/reicast-emulator/blob/master/core/rec-x86/rec_x86_driver.cpp#L324
https://github.com/reicast/reicast-emulator/blob/master/core/rec-x86/rec_x86_driver.cpp#L347

CheckBlock and force_checks (not including the function variable "declaration")

[twinaphex]

Yeah, we know now which function needs to be backported to the x64 driver in order to fix this, but it seems nobody has been able to do it yet. You could give it a try too.

[AbandonedCart]

@twinaphex Wasn't sure, since the last thing you mentioned was only the one-liner. I have enough on my plate without getting directly involved in this one, too. I am sure you guys can handle it, but figured whatever info I could offer with a quick glance was better than nothing.

[rcaridade145]

I'm not sure that implementing that per backend is the best way to go. @skmp Would very much like your input here. Would you consider appropriate an approach like https://github.com/mupen64plus/mupen64plus-core/blob/master/src/device/r4300/new_dynarec/new_dynarec.c with its memory_map.

Reasoning would be implement a sort of view over the ram and if the block got invalidated by a write to a section of memory you would directly call rdv_CompilePC which is what in the end the x86-32 core will end up (right?).

[twinaphex]

@rcaridade145 I think it would be more prudent to just implement this for x64 (DoChecks) and then later on look at your memory map idea in a separate issue. I really think it's a bad idea to try to fix this issue AND introduce an entirely new model to the dynarec both at the same time instead of going for the low hanging fruit here. What this probably will result in is adding even more potential complications to what ought to be a simple and straightforward fix. Best to split that up into a separate issue and tackle one issue at a time.

My main fear is that instead of trying to solve the dynarec crashes in the most straightforward manner as suggested here before, by going in this new direction instead we will get bogged down in lots of little issues that will go wrong with the memory map model at first and which will require lots of growing pain issues to resolve, then we will be even further from home here. I'd suggest just implementing the DoChecks function already in x64, then move from there once we have actually fixed the issues first. Let's not overcomplicate this further and move step by step.

[twinaphex]

I will add another $50 to the bounty if we can implement this DoCheck function in the most straightforward manner possible, and if that will fix the issues. Let me know if you agree to this @rcaridade145, if so I will get it done.

Just to clear up any confusion, I am not against the memory map ideas proposed. I just would like to see the main issues fixed first before we move onto something more ambitious.

[rcaridade145]

@twinaphex gotcha.

[twinaphex]

OK, let me get that done as soon as possible.

[skmp]

Yes, using pages for protection is the best long term approach. Manual
checking is needed for some blocks cus they store both data and code
nearby.

nullDC implemented a hybrid approach with some bonus optimizations on top
like immification of memory constant pools.

Implementing the DoChecks is the first step for accuracy, and then we can
figure out how to optimize from there.

On Wed, 20 Jun 2018 at 16:08, Twinaphex [e-mail] wrote:

OK, let me get that done as soon as possible.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reicast/reicast-emulator/issues/1167#issuecomment-398762704,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYAMrOouNsO43R6P5dKQZutHX4HUWm1ks5t-ldggaJpZM4UhvOn
.

>

~skmp

[skmp]

Also please note, there are a few platforms that have no mmu - eg, html5.

On Wed, 20 Jun 2018 at 16:49, skmp [e-mail] wrote:

Yes, using pages for protection is the best long term approach. Manual
checking is needed for some blocks cus they store both data and code
nearby.

nullDC implemented a hybrid approach with some bonus optimizations on top
like immification of memory constant pools.

Implementing the DoChecks is the first step for accuracy, and then we can
figure out how to optimize from there.

On Wed, 20 Jun 2018 at 16:08, Twinaphex [e-mail] wrote:

OK, let me get that done as soon as possible.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reicast/reicast-emulator/issues/1167#issuecomment-398762704,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYAMrOouNsO43R6P5dKQZutHX4HUWm1ks5t-ldggaJpZM4UhvOn
.

>

~skmp

>

~skmp

[rcaridade145]

@skmp another probably stupid question but

`ngen_blockcheckfail:
call rdv_BlockCheckFail
jmp eax

#

.globl ngen_blockcheckfail2
.type ngen_blockcheckfail2, @function

ngen_blockcheckfail2:
int 3
call rdv_BlockCheckFail
jmp eax`

CheckBlock(block,force_checks?x86_ptr_imm(ngen_blockcheckfail):x86_ptr_imm(ngen_blockcheckfail2));
Both on fail and success they both call rdv_BlockCheckFail. Why is that?

[AbandonedCart]

https://github.com/reicast/reicast-emulator/blob/master/core/hw/sh4/dyna/ngen.h#L68 ?
(Read: is it just being used as another way to invalidate the block?)

[skmp]

because i wanted to get crashes on unexpectedly changed code so i‘d notice
it and add blocks to the blacklist

On Thu, 21 Jun 2018 at 03:00, TwistedUmbrella [e-mail]
wrote:

>

https://github.com/reicast/reicast-emulator/blob/master/core/hw/sh4/dyna/ngen.h#L68
?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reicast/reicast-emulator/issues/1167#issuecomment-398943511,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYAMgswUa111ijjjVuJy7yAKditkaL2ks5t-vAcgaJpZM4UhvOn
.

>

~skmp

[twinaphex]

Added another $50 to the bounty guys as promised.

[rcaridade145]

@skmp but in the end both will invalidate the code if force checks is 1 no?

[skmp]

No, it is a bit complicated. But this is a debug feature that is special
for x86. When the flag is false, you should not do any checks (see how the
arm backend does it).

On Thu, 21 Jun 2018 at 10:01, rcaridade145 [e-mail] wrote:

@skmp https://github.com/skmp but in the end both will invalidate the
code if force checks is 1 no?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reicast/reicast-emulator/issues/1167#issuecomment-399012388,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYAMr1CbBnOPz9YdDg38VN8xCeF9lBhks5t-1LxgaJpZM4UhvOn
.

>

~skmp

[skmp]

So, is anyone actively working on this? Could get it done tomorrow but I'd rather someone else do it since there is a bounty on it

[rcaridade145]

@skmp I have been studying it and trying to learn the best way to implement it. Looking at other projects and trying to learn.
In my opinion for rec-x64 and rec-cpp would be something like https://github.com/devmiyax/yabause/tree/master/yabause/src/sh2_dynarec_devmiyax .. If you can implement it ,please do :)

[twinaphex]

Yeah, I told @skmp too that I would be fine with the bounty still going towards you for your efforts if you prefer that, or if he prefers it goes to him and he then reallocates it to other bounties/issues that need it, I am fine with that as well. Maybe he could reallocate the money to go towards the Vulkan bounty then, or whatever other big issue needs it. Completely in his hands. Let me know what you guys think.

I only care about seeing the issue finally resolved, and I'm sure others would wholeheartedly agree there. I am fine with whatever you guys suggest. I also thank both of you for your interest and efforts so far.

Hope we see an implementation for this soon. Will be great to finally have this resolved.

[rcaridade145]

@twinaphex @skmp At the end of the day i only care about the product (also an end user :) ) . So if goes to the person doing the implementation totally ok. If you decide to reallocate that for another project i'm ok also. Like you said "I only care about seeing the issue finally resolved".

@skmp One thing i noticed especially on libretro reicast is that when -flto at the end of the build on Linux x64 i get

Warning: ignoring changed section attributes for .text

https://gcc.gnu.org/ml/gcc-help/2010-09/msg00086.html

Probably defined here
https://github.com/reicast/reicast-emulator/blob/master/core/hw/sh4/dyna/driver.cpp

`#if !defined(_WIN64)
u8 SH4_TCB[CODE_SIZE+4096]

if HOST_OS == OS_WINDOWS || FEAT_SHREC != DYNAREC_JIT

;

elif HOST_OS == OS_LINUX

__attribute__((section(".text")));

elif HOST_OS==OS_DARWIN

__attribute__((section("__TEXT,.text")));

else

#error SH4_TCB ALLOC

endif

endif

`

If i remember something else i'll post it here.

[AbandonedCart]

Why not split it? Half to @rcaridade145 for his investment and half recycled into other issues?

Is there some method for allowing that?

[skmp]

@rcaridade145 if you want to work on it, I'm not in a particular hurry. We do want to push r7 out during #1150 so if it's not done by then I'll do it. Meanwhile, feel free to poke me at irc if you have any questions :)

[rcaridade145]

@skmp I think its better for everyone that if could get it done by tomorrow you should :) It is better for everyone than waiting on my learning / implementation/ etc :) Like i said first of all is getting the issue resolved :)

[twinaphex]

@skmp I think its better for everyone that if could get it done by tomorrow you should :)

I agree with @rcaridade145 completely.

[rcaridade145]

@skmp @twinaphex Even because with that implemented i'm not sure if Libretro won't need more work. I can after all play the game on reicast standalone like the screenshots before show.

[rcaridade145]

Btw @skmp you sent an e-mail. Not sure if had that info :)

Edit: zip not allowed :|

[twinaphex]

I'm not sure if @skmp remembers too but there was another bounty I talked with him about before (libretro-specific) dedicated towards getting the x86/ARM dynarecs working in libretro. . I seem to remember he said if it wasn't done before a certain timeframe (around end of June or something) he would look at it. It is already a higher bounty so if preferable we could have this bounty already successfully completed before that conference happens and that other bounty could be looked at meanwhile.

It was this one -
https://github.com/libretro/reicast-emulator/issues/12

[rcaridade145]

@twinaphex Have to commit my work on that by today but if you look at the Makefile on reicast and the one Libretro, on terms of x86, there are lots of -m32 missing.

[twinaphex]

Yeah, I see right now there is a guy in that thread who has a dynamic library for reicast 32bit. I will have to find somebody with a 32bit Linux machine (or set one up myself) in order to test.

[rcaridade145]

@twinaphex Let's hope he can't get any picture due to 32 bit drivers lacking :)

[twinaphex]

BTW, when you tested Shenmue and you reported that it worked on your end, did you use a CDI rip or the GD-ROM original? I hear from @Shoegazer it can make a difference. I am using the GD-ROM original when testing.

[rcaridade145]

@twinaphex cdi rip

[twinaphex]

OK, that could have been why you were not able to reproduce the crashes then. Try it again with the GD-ROM versions.

[rcaridade145]

@twinaphex Ok. Still have to fix some issues caused by including libretro headers that i've identified last night. INLINE , MAX , MIN are being redefined because they are also present on https://github.com/reicast/reicast-emulator/blob/master/core/types.h#L452 and for Linux that is wrong http://jijithchandran.blogspot.com/2013/12/force-inline-functions-in-cgcc.html

[twinaphex]

Things compile just fine for me on both Linux and Windows.

[AbandonedCart]

@rcaridade145 If you need to verify the compile, you can always submit it as a PR (and tag it don't merge) to use the automated builds.

[rcaridade145]

@twinaphex @LoungeKatt It compiles but the macros are redefined. If you try build it with clang for instance it is very much verbose about that.

[twinaphex]

I'll take a look at it.

[twinaphex]

https://github.com/reicast/reicast-emulator/pull/1194

My initial tests -

Shenmue 1/2 works now with x64 dynarec.
Marvel Vs Capcom 2 no longer hangs at the second/third stage with x64 dynarec. Previously you needed rec_cpp in order to get this to work.
Heavy Metal Geomatrix no longer crashes after character select with x64 dynarec. Previously you needed rec_cpp in order to get this to work.

DOA2 LE still crashes after selecting a character.
@skmp wants to still finish implementing this for rec_cpp before we can consider this one closed.

[twinaphex]

@skmp For some reason the crash at startup with Shenmue 1 (New Game, Ryo gets out of boot, loading screen happens and a diary is about to be shown) still happens on Linux x64, but Windows x64 is fine. Not sure why that is.

More details here -

https://github.com/libretro/reicast-emulator/issues/55

EDIT: OSX has the same dynarec crashes/bugs still too. So OSX/Linux x64 is still broken, Windows x64 is fixed.

[Shoegzer]

Confirmed here too, also with Marvel Vs. Capcom 2 - still having crashes with these games. Linux x64. Thanks for any help tracking this down @skmp.

[zimudec]

Retroarch version of windows with core reicast OIT with dynamic recompiler active (dynarec) with wine in linux, works perfect, without errors, fast and without crashes

[skmp]

I had the windows register hardcoded in the codegen, updated the PR with a fix. Can you test again please?

[twinaphex]

Will there be a patch for the generic recompiler too?

Anyway, @skmp, since you closed the issue, in order to collect the bounty, you now have to go to the bountysource URL for this issue, click on 'Claim Solve Bounty'. From there, the people that posted the bounty money (in this case, me) can then approve of the solution and then the transfer is complete.

[skmp]

Huh, Looks like I closed it by accident.

I guess I'll

• [x] Create a follow up issue for the rec-cpp side

[skmp]

(Follow up ticket created)