Rpcs3: List of tasks on graphic components.

[vlj]

I gathered a list of TODO related to graphic code. This is likely not exhaustif and not ranked by priority or difficulty.

• [x] Fix Vulkan shadow in Hitman 2. For some reasons with Vulkan Hitman 2 shadows appear correctly but are surrounded by a black texture. I guess it's a sampler parameter not properly set but I can be wrong.
• [x] Fix Silent Hills 3 missing shadows in OpenGL.
• [ ] Also fix SH3 shadows on character in DX12.
• [x] Implement Queries (called "REPORT" in RSX terminology) support for all backend.
• [x] Implement MSAA for all backend. I have some code working in d3d12-branch which require some cleaning. Issue is that there is not a lot of game with MSAA (so far I only found Super Puzzle Fighter 2 turbo).
• [x] Support RGBA4/16 sampling of depth buffer. RSX sometimes interpret depth texture as RGBA4 (for z16) or RGBA8(for z24u8) and use a dot operation to read them (since depth texture is an unsigned 16 or 24 bits integer one it's mathematically correct). However API doesn't accept that and requires a conversion step. Again in d3d12-wip branch there's a working implementation of this but it needs some works.
• [x] Avoid buffer extra copy in GL/Vulkan. Both backend store vertex and index data to a std::vector structure which is then copied into gpu visible memory. Such copies could be avoided by directly writing into gpu visible memory which what happens with dx12. The difficulty here is that OpenGL glMap() function is really transparent and can create a stall in the gpu. glMapRange() may help with correct flag.
• [x] Factorize DX12/GL/Vulkan vertex/index handling code. The three backends manages vertex in the same way at a high level but they don't share code except for the upload function.
• [x] Implement texture cube/texture 3d for GL and Vulkan. GL doesn't support texture 3d atm (there is no glTex3dStorage call). Some code exists in Vulkan but a lots of function use a default VK_IMAGE_DIMENSION_2D by default. So far After Burner Climax has cubemap texture for the reflection effect and Resogun has 3d textures likely for smoke effect.
• [ ] Implement render target extraction for Vulkan to see what happens in RSX Debugger.
• [ ] Make a common pipelineProperties structure filled by RSXThread. Currently DX12 and Vulkan have their own structure but the same parameters are set. Using a common structure may allow to reduce memory footprint and make pipeline lookup faster.
• [x] Implement vertex buffer and index buffer caching. This one can be really difficult : contrary to texture the backend doesn't know in advance how big a vertex buffer can be if there is an index buffer. At RSX level buffer boundaries are also not that clear since several draw call can use the same buffers internally but we only can access subbuffers used for draw call. On the other hand implementing this correctly may bring a huge performance boost, it's common that backends spend several ms uploading vertex buffers that barely change between frame.
• [x] Unify texture cache code. DX12 has a texture cache implementation that uses hashtable for O(1) address to texture mapping and that can reuse texture objects if a texture was modified but kept the same size. GL and Vulkan implementation use an std::vector<> and parse it when looking for a texture which can be very costly if there are hundreds of them...
• [x] Optimise invalidate_texture() call. DX12/Vulkan/GL uses the same algorithm here : they parse a std::vector<> of range when an access violation is encountered. I suspect we loose a lot of time here that isn't reported by d3d12 overlay. To improve things I think having a mixed hashtable/vector implementation can fasten things a lot : for instance an array of 512 std::vector , one for every MB of rsx memory, holding a pointer to textures using the corresponding Mb block. This way invalidate_address only has to parse a std::vector of maybe 1 or 2 textures at every access violation instead of every created texture.
• [x] Try to make all backend file structure and backend member closer. This way working on the 3 backends is easier.
• [x] Support non 0 offset in viewport (see Scarygirls and maybe Disgaea 3 loading screen).
• [x] Avoid unecessary descriptor binding/update in GL/Vulkan (see m_dirty* in d3d12 backend)
• [x] Support Alpha testing in Vulkan/GL.
• [x] Support separate front/back diffuse and specular color for Vulkan and GL (see Outrun Online)
• [x] Support mirrored gl_Position (see After Burner Climax) in GL and Vulkan.

[kd-11]

The hitman 2 shadows thing on vulkan i also found to be odd. It doesn't seem to be due to border clamp color (it is set to white, 1.f). I was going to open a separate issue, but I guess this can serve as a tracker for this.

Another issue is that Afterburner climax does not work on GL for some reason. With vulkan, it just worked out of the box, and it was a direct opengl port back then, so I really dont know why it doesnt work.

[vlj]

I looked at After Burner Climax with GL. What happens is that rtt are created as gl_texture_2d but are sampled with unnormalized coords. Unfortunately in gl texture 2d and rectangle are different object from gl spec point of view, they can't even be mapped using texture view.
It works in Vulkan because sampler 2d can be transformed into a sampler rest by setting the right bit in sampler object.

Fix for gl is the same as for dx12, we need to divide input by texture size.

[kd-11]

I'll extend GL to use explicit samplers through ARB_sampler_objects and see if it helps. Should help make it similar to vulkan in that regard.

EDIT: Seems ARB_sampler_objects doesn't handle unnormalized coordinates. We'll just use the shader method as in dx12

[vlj]

Vulkan doesn't support non 0xFFFF primitive restart index, this makes SH3 crashes on nvidia (kernel exception). I think it tries to access vertex "-2" which is not contained in the vertex buffer.

[kd-11]

That'd be 0xFFFE. I think we have to fill the index buffer ourselves and replace the primitive restart index with FFFF or FFFFFFFF depending on the type

[vlj]

Actually I was wrong SH3 does not crash because of primitive restart but because of cubemap (which are not supported atm : what happens is that the extra layer becomes mipmap level atm).
Some geometry is missing though so maybe it's tied to primitive restart.

[kd-11]

Another issue i've noticed on gl: missing/corrupt geometry. Notable on afterburner climax, hitman 2 (hair), resogun. prince of persia the sands of time

[vlj]

Geometry in GL is apparently here but misplaced, at least in Hitman 2. For some reason the hair is using wrong vertex attribute apparently.

[kd-11]

Its interesting that there are attribute issues on openGL and not on vulkan. The initial vulkan commit was a near direct port of the OpenGL backend and there are no issues there.

[vlj]

I found the issue, vertex buffer size was wrong: https://github.com/vlj/rpcs3/commit/979eea638ab61d19918f176da647839cebcef83e

vertex_draw_count is the number of vertex draw, in case of non indexed draw it's exactly the count of element in the vertex buffer array however for indexed draw there is no relationship between count of element in vertex buffer and vertex_draw_count. Usually the buffer has less element (since indexed usually reuse a vertex several time) however in some case the vertex array must be bigger.

[kd-11]

Nice catch. Vulkan initially had a similar bug but i somehow completely missed this when reviewing the opengl code recently trying to fix this bug.

[vlj]

I have almost finished factorizing vertex attribute upload code between dx12 and Vulkan, will soon send a pr.

Btw it looks like there is a bug with rgba32 format and vertex buffer with some Radeon card, I've signaled it to Vulkan project and it looks like a driver bug and not a hw limit. I'm keeping the workaround for now but I will remove it if it get fixed with a driver update.

[danilaml]

@vlj btw, it's unrelated but while you have a contact with AMD driver dev could you ask if missing glsl 450 support is intentional (because otherwise amd drivers are oGL 4.5 compliant) or it's an oversight.

[kd-11]

@danilaml If I remember correctly, newer drivers report glsl 450 for GCN on windows since the 16.x crimson series. I could recheck when I get home though if I'm wrong.

[kd-11]

@vlj Its interesting that texelFetch should respect the swizzle mask under vulkan but not openGL. Under OpenGL glsl, the texelFetch instruction completely ignores the component masking state of the sampler for both amd and nvidia. I'm not sure if its a bug or just an undefined part of the spec since I havent found anything dictating expected behaviour.

[danilaml]

@kd-11 wow. Indeed it does. I'm pretty sure that's got added recently though (I think I checked on first crimson drivers with no luck so it was either 16.2 or 16.3 or something in between).

[tambry]

Slightly unrelated, but it would be nice to instead create the VertexProgram and FragmentProgram files in a separate folder. It's quite annoying to have the bin folder filled up with 100's of them.

[vlj]

I agree.

[danilaml]

@vlj btw, if you haven't lost your contact, you can probably ask him about this issue: https://community.amd.com/message/2710176#2710176

[kd-11]

https://github.com/RPCS3/rpcs3/pull/1722 should fix hitman and other games with the black shadow covering everything

[kd-11]

https://github.com/RPCS3/rpcs3/pull/1743 should fix the alpha testing in opengl and vulkan backends.

[raven02]

Having https://github.com/RPCS3/rpcs3/pull/1763 , all backends should have Logic Op implemented

[kd-11]

Extra buffer copying in opengl has been removed as of https://github.com/RPCS3/rpcs3/pull/1772

[raven02]

@kd-11 , i think the extra buffer copy still remains for Vulcan ? or should be removed as well

[kd-11]

I'm going to rewrite the vulkan upload code after I track down the nv resizing bug. It still does a double copy which is slow.

[raven02]

thanks @kd-11

[raven02]

Just added the DX12 heap size into TODO list above

[kd-11]

Buffer copying fixed for vulkan with latest commit, so that's one more off the list

[kd-11]

separate front/back diffuse and specular color for Vulkan and GL is done. DH implemented GL and I implemented dx12

[raven02]

Thanks @kd-11.Updated the list.

[raven02]

Besides , i think the oversizing of heap memory in DX12 probably becasue of https://github.com/RPCS3/rpcs3/issues/1912 and also sometime leads to "Working buffer not enough"

[raven02]

Regarding Disgaea 3 loading screen special effect , it works previously long time ago using GL with write color buffer option

[danilaml]

Btw, might be interesting: https://github.com/KhronosGroup/Vulkan-Hpp

[kd-11]

@danilaml I saw that earlier, but I'm on the fence about it. We kinda already have our vk:: namespace wrappers in place at a higher level than the hpp provides. Using it becomes using a wrapper within another wrapper although it shouldn't affect much else. I'm not against it if someone is looking to invest time into it though.

[kd-11]

Closing as the relevant bits are implemented for a long time.