Cxbx-reloaded: Prevent duplicate OOVPA's

[PatrickvL]

Many OOVPA's are duplicates of one another.

For example, IDirect3DDevice8_SetRenderState_StencilFail_1_0_4361 is identical to IDirect3DDevice8_SetRenderState_StencilFail_1_0_4134.

In this issue, find and remove all duplicates.
Always remove the newer version (since the older one apparently hasn't changed since).

If possible, create an alias for the duplicate, and use that alias in the OOVPATable for that version (and all versions above that, until a more recent OOVPA declaration is encountered).

Ideally, once this is done, an OOVPATable should solely contain OOVPA names for it's own version.

Alternatively, the HLE scanning code could be changed to first find functions using the exact version, but use increasingly older version OOVPATable's for undetected functions.

[LukeUsher]

I agree with all of this, removing all duplicates is useful and adding aliases would ensure that the OOVPA tables reference the correct versions.

I like the idea of the HLE scanning code iterating backwards through older XDK revision, however I think that could cause us issues with false positives. We should stick with aliases.

[jarupxx]

I'll help you find and remove all duplicates.

[PatrickvL]

Since this could involve lots of manual labour, it'd be best if duplicates are detected by a function, which could not only detect duplicates but also nearly identical OOVPA's

[PatrickvL]

A comparison should loop over all HLE data entries, and all OOVPA's in there. For each OOVPA compare against all others (so this becomes a exponential algorithm). Comparing two OOVPA's should ignore small/large and count differences, but must count number of unique offsets on both sides, the number of identical offset plus value pairs and the number of identical offset but differing value pairs.

[PatrickvL]

When the number of differing values, and the number of unique pairs on both sides are low (less than 3 for example), these OOVPA's are nearly identical. If these three counts are all zero, the compared OOVPA's are completely identical.

[PatrickvL]

Prior to comparing two OOVPA's, a checks could be done on each main loop item, like: Is it marked large but only contains small offsets? Are all offsets incremental? Are there no duplicate offsets? These checks must never fail, else they are faulty OOVPA's.

[PatrickvL]

Since all offsets are incremental, comparing two OOVPA's is a linear algorithm, much like a merge sort (without the sorting part)

[jarupxx]

I think I understand. I think that once this task is completed I will be able to detect the function correctly. I also found a issue with #98, so I think it is a good idea to move to .inl.

[PatrickvL]

The compare function could be aborted when a threshold count is reached, to speed up the comparisons. But then again, speed won't be too important if this check is only executed when requested by the user.

[jarupxx]

I created a program to delete duplicate OOVPA's. Excel vba.
(Offset, Value) -Pair is checked and deleted. Replace OOVPATable.
There are some bugs, but we will release it as it is

Known bug
We will fail to delete Get 2 DSurfaceDesc, 5849
Those replacement will fail
CDirectSoundBuffer_SetAllParameters, 5558 -> DirectSound_CDirectSoundBuffer_SetAllParameters, 4627
OOVPA_TABLE_XREF (DirectSound_CMcpxBuffer_Pause, 5788), into DSound.1.0.5788.inl.

https://github.com/jarupxx/cxbx_log/blob/master/Del_duplicate_OOVPAs.xlsm

[PatrickvL]

Nice. I'll have a look later. I hope it doesn't really delete OOVPA's, but just replaces each duplicate with an alias to the original, so no information is lost.

[PatrickvL]

Duplicates that are registered with the same xref and patch are not really a problem. But duplicates that are associated with another xref and/or patch should be reported and fixed by adding unique offset value pairs. Otherwise they could be mixed up with the other duplicate (s)

[PatrickvL]

@jarupxx Can you verify if this is issue now completely fixed? If not, please add a summary to this issue, with the names of duplicates still present.

[jarupxx]

This issue is not fixed. I have not completed the investigation of this issue and I have no documents.

[jarupxx]

I was completed the Integrated duplicate OOVPA.