Gliden64: VI fixes

[gonetz]

This is inspired by #1977 and https://github.com/gonetz/GLideN64/issues/1977#issuecomment-452258006
Cite from angrylions commit message for revision 87:

rdp: new tests revealed the shocking truth: the VI aggresssively and regardlessly of any data alignment specificities cuts off either 7 or 8 pixels closest to h_start and h_end, so that nothing ever appears on the screen unless (h_end - h_start) is 16 pixels or greater (this removes marginal garbage graphics on the far right in Mario Tennis, Turok 2 Seeds of Evil intro and probably other games);

I built angrylions software LLE plugin from the latest sources on SourceForge. Indeed, all games have black boarders from left and right. I compared picture in Indiana rendered by angrylions plugin with image from my N64 console. I don't have video capture devices to take screen shot, but as far as I can see, picture from N64 is the same as emulated one.

This fix "removes marginal garbage graphics on the far right". GLideN64 also suffers from this problem, so I ported angrylions fix to GLideN64.
Please help me to test it.
The fix is in vi_fixes branch.
Test build for Project64:
https://drive.google.com/file/d/1k17xP-yrATGO251ynuFDUEEjTGY2HDdS/view?usp=sharing
https://drive.google.com/file/d/1K0uaUdvkrcwGXhWc0tU2zeTIpCJfk42L/view?usp=sharing
Build of angrylions rev 112:
https://drive.google.com/file/d/1OxbngLjW6jaBLncM59uiwWgYxypFJJu3/view?usp=sharing

How to test:

1. Configure GLideN64:
2. Select windowed resolution 1024x768
3. Select N64-style 3 point filtering
4. Select Original N64 resolution
5. Select LLE mode in emulator: Options->Settings->Plugin uncheck Graphics HLE
6. Run any game
7. Run another instance of Project64, select "My little plugin" (it is angrylion's plugin) as graphics plugin and run the same game.
8. Both plugins should render nearly the same picture, with the same black boarders and the same graphics inside these boarders.

[theboy181]

@legendofdragoon had done something else in AL experimental plugin. Instead of extending the black bars, he swapped the right hand image to the left side of the screen, maximizing the entire image, while correcting the issue.

His fix didn't remove it, it improved it. Could you possible address it in the same way?

I could be wrong here, so I hope he responds. I also think that disabling AA filters in the ROM can fix this issue too.

Would be awesome if GLideN64 had emulated VI filters. :)

[LegendOfDragoon]

I didn't do anything special for this, to be honest.

I also think that disabling AA filters in the ROM can fix this issue too.

I don't think so, but i'd be very intrigued if it did fix this issue.. The problem is that in certain games, the RDP isn't always drawing to the first and last 8 pixels of each row, which is why you're seeing garbage pixels.

[gonetz]

The problem is that in certain games, the RDP isn't always drawing to the first and last 8 pixels of each row

Yes, the problem is that left and right edges of RDP image are not filled during rendering in some games, thus we see garbage here. I guess it is because the games devs knew that VI will cut image anyway, so they saved some computation resources.

One note: 8 pixels cut not from RDP image, but from output image. For example, NTSC image size is 640x480 interlaced. VI cut 8 pixels from that image, that is visible part of each line is [8, 472]. If game's internal resolution is 320x240, each pixel of RDP image scaled to two pixels on screen, thus visible part of RDP image row is [4, 316]. Many games, for example Wave Race, add black boarders wider than these 4 pixels, thus this fix does not affect them at all.

[gonetz]

I fixed several issues in VI emulation code. Link to test build updated.

[oddMLan]

GLideN64 Latest WIP with VI fixes (85062921)

GLideN64 8e621210

All default settings, as you can see there is some slight blurring introduced with the VI fixes, while the previous version without them looks crisp... is this intended? 🙁

[theboy181]

VI filters are working in HLE? I love it, but what is actually going on?
Do the games that show SEEMS look good with this ?

[theboy181]

Could this be exploited to help level seems?

[theboy181]

@oddMLan we should try to see if turning off AA filters works in HLE now. :)

[oddMLan]

@theboy181 I'm pretty sure VI filters are not implemented, it's just some weird resizing taking place blurring the picture. Like it's one pixel off or something.

[theboy181]

looks like just changing
takes away the AA effect. :(

[LegendOfDragoon]

Imo, that smoothing looks better for super mario :smile: .

[gonetz]

@oddMLan
< is this intended?
Not exactly. Video Interface maps RDP image to TV screen. VI emulation part does the same: maps FBO texture to OpenGL main buffer. For SM64, RDP image is 320x240 while output image is 640x480. Thus, some scaling is used. Plugin tries to create FBO texture of the same size as size of output buffer to avoid scaling and get crisp image. It is not always possible because VI may add black borders to output image and image in FBO texture must be scaled to fit the place among the borders.
This fix emulates "VI cuts off either 7 or 8 pixels closest to h_start and h_end". If angrylion's implementation is correct, VI just do not show first 8 and last 8 pixels of mapped image, drawing them black. Since each pixel in RDP image line corresponds to two pixel in output image (320 => 640) VI will draw first 8 pixels black, then RDP line [4, 316] mapped to [8, 632], then again 8 black pixels. Plugin needs to calculate these offsets for input and output images, taking in account the scale with which original RDP image is mapped to screen and scale between VI image size and user window size. I guess some rounding errors during all these mappings lead to difference between size of input image and output one, which leads to use of bilinear filtering and thus blurring.

[olivieryuyu]

Ken Griffey Jr.'s Slugfest intro shows nearly the gfx but they are flickering now.

[oddMLan]

The unintended blurriness is fixed \o/