Gliden64: Fix bilinear filtering

[theboy181]

OK looks a lot better. Are the sprites also a bug? or something different?

[theboy181]

Something else is wrong. The output of the pixels here are smaller than the others. and you can see that its happening throughout all games.

[theboy181]

Shouldn't all the pixels be square?

[gonetz]

Shouldn't all the pixels be square?

In native res and without VI post-processing - yes, they should, but there is no such mode in GLideN64.

[theboy181]

Can this issue be fixed for X2+ resolutions, and VI post-processing? How are other emulators able to double the resolution without issue?

[gizmo98]

Just a note about 2D elements which consist of more then one tile and the use of higher resolutions. I'm not an expert but i think N64 splits big textures in tiles because texture memory is limited. Most of the time tile borders are visible (those ugly lines in textures) because texture filter cannot sample from the surrounding tiles. A red texel at coord 1.0 from tile0 and a white texel at coord 0.0 from tile1 will not be interpolated to rose. If texture clamp is enabled You will see a red or white pixel at the tile border.

If you know all tiles you could reconstruct the original texture and sample from this one. But this is no easy task.

[theboy181]

Probably big part of the issue and even know native res was mentioned as not having this issue, is simply does.

[gizmo98]

@theboy181 n64 programmers had some tricks like overlapping textures or cut off wrong colored texture borders with SHRINKSIZE to reduce such issues. I'm not sure if some of the present issues are not there on real hardware. Don't know how the n64 hardware should sample from adjacent tiles to prevent borderline issues.

[gizmo98]

I have found a thread about tiles and seams:
https://gamedev.stackexchange.com/questions/74420/seamless-tilemap-rendering-borderless-adjacent-images

[theboy181]

I wondered if it wasn’t locked to intergerscaling. This thought to impliment

[weinerschnitzel]

I think it would be useful for comparisons and also nice to have if we could disable VI

[standard-two-simplex]

I have found a thread about tiles and seams

The N64 programming manual encourages game programmers to include a 1px border when tiling bigger textures that should be bilinear filtered.

http://level42.ca/projects/ultra64/Documentation/man/pro-man/pro14/index14.5.html

See also the link in that page about filtering. If I recall correctly, the texture coordinates aren't wholly correct, which creates unexpected results. See the Mario Kart in-race portraits. The portraits are point sampled while the glow is bilinear filtered. And they don't match, even if they are rendered similarly.

[gonetz]

I put Stefan's 'Fix black and violet outlines around transparent textures #1939 ' into fix_texture_filtering branch: https://github.com/gonetz/GLideN64/tree/fix_texture_filtering

Test build for Project64:
https://drive.google.com/file/d/175wrz9JF8BcV92wMVBaaPIedfA1fFbei/view?usp=sharing

Disable shader cache!

[gonetz]

Fix with use of premultiplied alpha color values works great where it applicable.
However, the idea to smooth alpha for filtered textures is questionable. Stefan's comment: "improves kirby64 and zelda mm text". Lets look at Zelda MM text:

The only difference is number of rupies. Does it look better with alpha smoothing? It is matter of taste. I don't like it. The situation is worse when we use HD textures:

Lets look closer to rupies texts:

IMO, this fix brings more troubles than fixes. Thus, I removed it: 38917373b1

Test build for Project64:
https://drive.google.com/file/d/1-Zpt7qq_UDbgsE0q-TXeQHMAeVaVh2QP/view?usp=sharing
Disable shader cache!

You may compare it with the previous build. May be you will find a case where alpha smoothing definitely makes filtered textures look better. In that case it can be added as an option.

[gizmo98]

Yeah. Alpha smoothing has side effects and is not a optimal solution. I know two cases which look better with it.

Kirby "%" sign:

Zelda MM load screen after you select a save state has no remnants between signs with it. Without it there are bars between signs:

[gonetz]

Again, I can't say that your example looks definitely better.
Comparison screen shot:

Alpha smooth - No alpha smooth - No alpha smooth native res - Angrylion's soft lle

[gonetz]

I found one example where filtering with alpha smoothing definitively looks better: Quest64

Without alpha smoothing
With alpha smoothing

[gizmo98]

@gonetz
Alphasmooth looks slightly different and it does not look like al or nativres. Sometimes it looks better. Sometimes it looks worse. It does not fix those black, green and pink outlines. I think we should focus on the premultiply alpha fix. Those remnants are not as annoying as wrong colors.

[theboy181]

@gonetz im getting purple text in Quest64 with the test build. is there something I am missing?

[oddMLan]

@theboy181 Did you disable shader cache?

[theboy181]

Yes

[gizmo98]

Does the pj64 test build work for anyone? I have no windows machine and project64.

It certainly does not work if you have a gles2 device. Bilinear filtering shader for these devices was not modified.

[gonetz]

@gonetz im getting purple text in Quest64 with the test build. is there something I am missing?

Test builds must work if you disabled the shader cache. You may also clear Shaders folder to be sure.
I just checked these builds, both work as intended.

[gonetz]

@gizmo98 I wanted to show that alpha smooth seldom has only good impact on game's textures. Usually some textures look worse and others may look better. So, if we will include this feature, it must be optional.
I agree, that we should focus on the premultiply alpha fix for now. If there are cases when it makes textures worse, it must be optional as well. I hope that there are no such cases.

[Kidboy17]

Found an issue with Stefan's fix for the black and violent outlines. Kirby 64, first level the background of the sky is cut off and jagged at the top.

This is how it is before the fix:

Edit: I found some more issues.
OoT and MM. The life meter and the number counter for Rupees.

Texts for the Days and Areas:

Mario Golf. The grass in the intro:

Before Fix:

The fix works great for the texts of Quest64!

[gonetz]

@Kidboy17 Do you have these issues with the build where alpha smooth is disabled (https://github.com/gonetz/GLideN64/issues/1933#issuecomment-434569386) ?

[oddMLan]

Indeed it looks like alpha smoothing is being applied judging by the screenshots.

[gizmo98]

I can reproduce kirby issue. I will check if uEnableAlphaTest or uCvgXAlpha causes the issue.

Is uCvgXAlpha = 1 only relevant if uAlphaCvgSel = 1? Flowchart: https://level42.ca/projects/ultra64/Documentation/man/pro-man/pro15/gif/f15-09.gif

According to the programming manual CvgSel is always 1 if CvgXAlpha is 1 for the described rendermodes.
https://level42.ca/projects/ultra64/Documentation/man/pro-man/pro15/index15.7.html

[Kidboy17]

@gonetz I just tried out the build you suggested and the only issue I have now is Mario Golf.

[gizmo98]

I need to find a better fix for Mario Kart. If i remove uCvgXAlpha == 1 check Mario Golf and Kirby 64 issues vanish.

[weinerschnitzel]

The article @gizmo98 linked mentions:

To solve: render (at 1:1) all of your tile textures into an FBO/RBO with no gaps, then send that FBO to the default framebuffer (the screen). Because the FBO itself is basically a single texture, you cannot end up with gaps on scaling. All texel boundaries that don't fall on a screen pixel boundary are going to be blended if you're using GL_LINEAR.... which is exactly what you want. This is the standard approach.

Looking at the current behavior where the texture is scaled and then added to the screen, the linear filter for that texture is using the textures rgb values for transparency when it should be using the background. Take a closer look at the borders for the text in https://github.com/gonetz/GLideN64/issues/1933#issuecomment-434740981

Native res is filtered properly with the background while the others create that hard edge.

The solution in the article sounds like it explains the problem, and since native res looks correct and both the scaling issues and the the colored text appear at higher resolutions, I think this would be worth investigating.

[gonetz]

To solve: render (at 1:1) all of your tile textures into an FBO/RBO with no gaps, then send that FBO to the default framebuffer (the screen).

This is what "Render 2D elements in N64 resolution" does. It is currently applied to texrects only, so 2D rendered with S2SEX ucode or with 3D triangles can't be fixed with it.

[N64Fan1992]

"Render 2D elements in N64 resolution" is very blurry though, likely due to bilinear filtering. Is it possible to make an option to make use no filtering or nearest neighbour just for these elements being stretched to the output size? Some games would look much better this way.

[thecoreyburton]

I made a request for this: https://github.com/gonetz/GLideN64/issues/1788

[weinerschnitzel]

Thanks for the clarification. I think disabling the filtering here would prevent that hard edge from being created on the texture. It could also be helpful to observe if banding issues occur when filtering is disabled.

If these textures are drawn 3d and work with native res, do we get expected results without filtering if hi res textures are used and double native resolution is used? I.e. textures twice as big in hi res, and native res x2 used.

Could test when I'm back home, but would there be any reason by the above would not work?

[CallistoNTG]

Thanks for the clarification. I think disabling the filtering here would prevent that hard edge from being created on the texture.

Disabling filtering doesn't solve the problem at all, though. The textures are supposed to be filtered.

[N64Fan1992]

Just to explain what I meant, I don't mean to ask to disable filtering on the textures before render, just change the scaling method for the final output. Let the textures be rendered at native resolution using whatever filtering they are supposed to, but have an option to choose how this is scaled to the output resolution, if it's filtered or nearest neighbour is my idea and request.

[weinerschnitzel]

Thinking more about the "hard edge" problem seen with bilinear filtering, I don't think this would be solvable with filtering the upscaled native textures.

Since these textures are tiled, the edges of the textures won't get a smooth edge when they are upscaled and filtered. They'll get a hard edge since that's the end of the texture.

For solutions to that problem I think we are only left to look at:

1. Custom hi res textures that look correct
2. Not filtering

Thought of this looking at olivier's last screenshot in #1971 which appears to display this problem in the 0's in the speedometer.

So I think we can just plain expect visual anomalies with higher resolutions and bilinear filtering.

[gizmo98]

Could someone test the following branch?
https://github.com/gonetz/GLideN64/compare/fix_texture_filtering...gizmo98:fix_texture_filtering4?expand=1

I modified fix_texture_filtering branch. Hamster 64 startup screen is not fixed in higher resolutions. Mario Kart tree textures have no black borders if bilinear filtering is enabled. There are no black textures in Mario Golf and Kirby 64 anymore. Puyo Puyon Party has no black texture borders.

This is the best fix i could figure out without adding any disadvantage compared to the current master.

[theboy181]

Could you post a BIN? I don't have my setup right now.

[gonetz]

Build with fix_texture_filtering4 by gizmo98:
https://www.dropbox.com/s/xg1q8rqialdhg6b/GLideN64_fix_texture_filtering4_by_gizmo98.7z?dl=0

I tested it a bit. Mario golf: PRESS START does not have black outline, which is correct. But since letters are more bold then they should, the text is barely readable, while with black outlines it reads fine. Unfortunately, the letters look correct only in native resolution.

Imo, while that fix is definitely useful for some games, it suffers from the same problem as our fix for S2DEX - there is no setup, which works well for all cases. I'm going to add new tab in plugin's GUI, which will contain all 2D-related settings for fine tuning 2D quality and performance. I think that texture filtering fix also must be optional, may be even several options for various modes.

[gizmo98]

@gonetz shall i add an option or will you add an option in the new 2D-related settings tab?

[gonetz]

You may add config option(s) for bilinear filtering fix and support them for mupen64plus. I'll support them in GUI.

[gizmo98]

I will try to add an option. Any suggestion how to name the new option?

[oddMLan]

Try to correct edge halos around textures

Tooltip: Helps for games which have hard to read text or colored outlines around the edges of textures. Can make textures bloated, so use only when necessary.

[gizmo98]

@oddMLan This sounds good! If i add a new option i will use your proposal.

@gonetz i could also add bilinear filter option 2 and 3 (texture.bilinearMode )
BILINEAR_3POINT_W_COLOR_BLEEDING = 2
BILINEAR_STANDARD_W_COLOR_BLEEDING_PREMULTIPLIED_ALPHA = 3

[gonetz]

Extending bilinear filter options may be the best solution. Please do it.

[gonetz]

I changed the way how that fix is enabled.
I added new option "Enable halos removal" instead of new bilinear filtering modes.
This way we can set in custom.ini
texture\enableHalosRemoval=1
and it will enable user-preferred filtering mode with halos removal fix.
Please use latest WIP build to test.

[Kidboy17]

Having the "Enable Halos Removal" option on causes the waters on Kirby's Stage in Super Smash Bros to produce errors.

[gizmo98]

Enable Halos Removal should not be enabled by default. In some games elements look more blocky. If a rendermode does not support transparency premultiplied alpha causes black textures. I exluded some rendermodes to fix the black texture problem in mario golf and kirby64.

3-Point filtering should not be affected because it only uses texture bleeding.

[weinerschnitzel]

Testing the SimCity64 build, I notice some strange behavior with a windowed resolution of 640x480 and switching Internal Resolution to Sames as output resolution, and 2x multiple of N64 resolution.

In this case, output resolution looks perfect when 2x has some issues. I would expect these to be the same, or that 2x would be better. I am curious why output resolution is better in this case.

Same as output resolution: https://i.imgur.com/faPjZdE.png

2x N64 resolution: https://i.imgur.com/69dFrGn.png

[thecoreyburton]

If a rendermode does not support transparency premultiplied alpha causes black textures.


This is an example in the second world of Chameleon Twist 2. It only happens once or twice throughout the whole game, but I think it demonstrates this problem. Enabling halo removal is required to fix some halos around some textures such as trees. I've only got a Mupen 64 (not to be confused with mupen64plus) savestate, unfortunately: Chameleon Twist 2 (U) [!].zip