Cxbx-reloaded: Convert vertex shader at drawing time

[PatrickvL]

Some titles create a vertex shader even before calling CreateDevice. To support those in HLE, vertex shaders need to be converted only at drawing time, much like how LLE does it.

• Kung Fu Chaos (retail version is LTCG, but demo is non-LTCG contained the Xbox Exhibition Vol.2.)
• NHL 2004
• Sega GT 2002

See https://github.com/Cxbx-Reloaded/Cxbx-Reloaded/pull/1271

[gandalfthewhite19890404]

NHL 2004 has such behaviour.

[jeltaqq]

Kung Fu Chaos has too. The retail version is LTCG, but demo is non-LTCG contained the Xbox Exhibition Vol.2.

[LukeUsher]

I had an idea for this that would work to solve the problem, without a massive refactor that converting at drawing time would involve.

1. Update our D3DDevice_CreateVertexShader patch to do the following:
   1a. Call the xbox CreateVertexShader function, and (if successful), save the handle it returned for later (if it failed, simply return the error code and do nothing)
   1b. Insert an entry into a data structure, where the key = the Xbox vertex shader handle, and the value = a struct containing all the data that was passed to CreateVertexShader (a copy of the declaration, function, flags, etc)

2. Update our MapXboxVertexShaderHandleToCxbxVertexShader function, so that it does the following:
   2a. Check if the shader handle is in the data structure we created above, if no, log a warning/error.
   2b. Check if the shader already has a host conversion, if yes, use that.
   2c. Convert the shader to a host shader, using the conversion code removed from D3DDevice_CreateVertexShader (the parameters we saved in the data structure earlier can be used to generate it)

Using this approach, we'll still hook CreateVertexShader to catch the data that we need to create the host shader, the Xbox shader handle will be returned to the running title, to keep that happy, and shader execution will be deferred to the point that Cxbx-Reloaded tries to use it.

[PatrickvL]

It's a massive hack, but this approach would work. It's a pity the entire HLE approach is so hacky. Vertex shader don't fill their slots correctly for example. A better approach would be to defer all conversions upto a draw call happens. We'd need to be able to pick up all state from their respective D3D structs (using symbol detection XREF's), and all push buffer commands that do get pushed should be handled correctly for HLE too. But once we're going that way, it'd be simpler to do everything via the FIFO puller anyway. All that remains would be to speed up MMIO (either via non-trivial backpatching or CPU emulation).

[LukeUsher]

One thing I noticed, is that the Xbox CreateVertexShader function converts the pDeclaration field into a list of NV2A pushbuffer commands, which get inserted in one of the Vertex Shader structs member fields.

With a bit of code refactoring, we could completely get rid of the need to cache data in the CreateVertexShader function, or even patch it at all, by parsing this data in the struct!

[bass3]

Unreal Championship 2: The liandri conflict also has issues with vertex shader.

[0x4680] WARN: Applying screen space vertex shader patching hack!
LOG_TEST_CASE: Pushbuffer COMMAND_TYPE_CALL
In EmuExecutePushBufferRaw (c:\projects\cxbx-reloaded\src\cxbxkrnl\emud3d8\pushbuffer.cpp line 640)LOG_TEST_CASE: Last pushbuffer instruction exceeds END of Data
In EmuExecutePushBufferRaw (c:\projects\cxbx-reloaded\src\cxbxkrnl\emud3d8\pushbuffer.cpp line 594)LOG_TEST_CASE: Pushbuffer COMMAND_TYPE_CALL while another call was active!
In EmuExecutePushBufferRaw (c:\projects\cxbx-reloaded\src\cxbxkrnl\emud3d8\pushbuffer.cpp line 635)[0x4680] WARN: Applying screen space vertex shader patching hack!
LOG_TEST_CASE: Pushbuffer COMMAND_TYPE unknown
In EmuExecutePushBufferRaw (c:\projects\cxbx-reloaded\src\cxbxkrnl\emud3d8\pushbuffer.cpp line 648)LOG_TEST_CASE: Pushbuffer COMMAND_FLAGS_RETURN with additional bits?!
In EmuExecutePushBufferRaw (c:\projects\cxbx-reloaded\src\cxbxkrnl\emud3d8\pushbuffer.cpp line 679)LOG_TEST_CASE: Pushbuffer COMMAND_INSTRUCTION unknown
In EmuExecutePushBufferRaw (c:\projects\cxbx-reloaded\src\cxbxkrnl\emud3d8\pushbuffer.cpp line 666)[0x4680] WARN: Locking host Texture failed!
[0x4680] WARN: Locking host Surface failed!
[0x4680] WARN: Applying screen space vertex shader patching hack!
[0x4680] WARN: Xbox Surface Format 12 will be converted to ARGB
[0x4680] MAIN: Received Exception (Code := 0xC0000005)

[LukeUsher]

WARN: Applying screen space vertex shader patching hack!
Is not a problem, it's expected behavior, as Xbox vertex shaders include screen space transformations but DirectX does those for us, if we don't patch them away, it happens twice.

[PatrickvL]

Work on this is happening in https://github.com/PatrickvL/Cxbx-Reloaded/tree/Vsh_unpatching