Xenia: 3D and stacked texture research

[Triang3l]

As of June 21, 2018, Xenia's 3D and stacked texture implementation is highly incomplete and makes a lot of failed assumptions.

3D texture tiling

Thanks to the .map file in Call of Duty 4 alphas, I was able to find the implementation of XGRAPHICS::TileVolume and reconstruct the tiled address code:

static uint32_t TiledOffset3D(uint32_t x, uint32_t y, uint32_t z,
                              uint32_t pitch_h, uint32_t pitch_v,
                              uint32_t log2_bpp) {
  uint32_t base = ((z >> 2) * (pitch_v >> 4) + (y >> 4)) * (pitch_h >> 5);
  uint32_t micro = ((z >> 2) + (y >> 3)) & 1;
  micro += (((micro << 1) + (x >> 3)) & 3) << 1;
  uint32_t macro = ((((y & 6) << 2) + (x & 7)) << (log2_bpp + 6)) >> 6;
  macro = ((((((base + (x >> 5)) << (log2_bpp + 6)) & 0xFFFFFFF) << 1) +
            (macro & ~15)) << 1) + (macro & 15) + ((z & 3) << (log2_bpp + 6)) +
          ((y & 1) << 4);
  return ((((((((macro >> 6) & 7) + ((micro & 1) << 3)) << 3) +
             (micro & ~1)) << 2) + (macro & ~511)) << 3) + (macro & 63);
}

(Variable names base, micro and macro chosen at random and don't truly reflect their meanings — according to the .pdb, there are BankSelectZ and MacroOffsetZ local variables in the address function. pitch_h and pitch_v are 32-aligned texture width and height, block_pitch_h and block_pitch_v in TextureExtent.)

Storage of tiny mip levels of 3D textures

There is a function called GetPackedTileOffset in Xenia that packs small mipmaps in very small tiles. I don't have any information on how it would work for 3D textures yet, this requires some serious investigation.

EDIT: This has been reverse-engineered, though no testing done yet.

Using 3D and stacked textures in Vulkan

The current code uses the VK_IMAGE_TYPE_3D type and VK_IMAGE_CREATE_2D_ARRAY_COMPATIBLE_BIT for stacked textures. This is wrong since this doesn't seem to have any effect on the way mipmaps are stored, which is the main difference between 3D and stacked textures. Instead, stacked textures should have VK_IMAGE_TYPE_2D without any special flags (with one exception that the type in SPIR-V must be declared with arrayed = true).

In stacked textures, each layer is a complete 2D texture with its own mipmap pyramid (each layer is a separate set of subresources), so if you have a 512x512x16 stack, you'll have 16 512x512 images, 16 256x256 images. 16 128x128 ones, and so on. Mipmaps are contained within layers.

In 3D textures, the Z axis is treated the same way as X and Y, so you have 512x512x16, 256x256x8, 128x128x4 mipmaps. The whole parallelepiped at each mip level is an individual subresource — layers are contained within mipmaps.

It looks like the shader ISA of Xenos uses the same tfetch3D instruction for sampling 3D and stacked textures (though I can't verify). Sampling a texture of a wrong dimension is highly dangerous in Vulkan. We need to investigate if we can use some currently unused bit in tfetch3D to distinguish between 3D and stacked textures. If there's no such information, we need to use a pipeline specialization constant — a bitfield that will store whether each currently bound texture referenced by tfetch3D is a real 3D texture (BitScanForward can be used to quickly filter only tfetch3Ded samplers when checking fetch register values).

If the texture is 3D, it will be sampled from the textures3D descriptor set, otherwise it will be sampled from a separate one (or possibly even from textures2D, but in this case, Z will have to be zeroed for 1D and 2D tfetch).

Refactoring required

Handling of 3D and stacked textures needs to be more consistent in general.

For instance, functions like GetMipExtent completely ignore the fact that 3D textures are mipmapped along Z. Code to handle 3D mipmapping needs to be added near almost every std::max instance.

Games for testing

3D textures:

• 🎮 Tales of Vesperia — fonts (512x512x16 pre-tiled DXT5 without mipmaps). The first time a 3D texture is used for text is after starting a new game, during the intro with grey clouds.
• 💀 The Orange Box — color correction lookup tables (32x32x32 runtime-tiled RGB8 without mipmaps).

Unfortunately, I don't know any game using stacked textures yet.

[Triang3l]

For small mip level offset, there is void XGRAPHICS::GetMipTailLevelOffsetCoords(unsigned int Width, unsigned int Height, unsigned int Depth, unsigned int Level, unsigned long GpuFormat, BOOL Tiled, BOOL Border, XGPOINT3D * pOffset) in Call of Duty 4 build 253 SP at sub_8236D010.

[Triang3l]

GetMipTailLevelOffsetCoords reconstructed: https://gist.github.com/Triang3l/4cc2f5c2901cd4e3ad2926c79c4f6858

[Triang3l]

Wrote some code to support untiling and mipmapping of 3D textures, looks like it works: https://github.com/Triang3l/xenia/commit/e7d13bc50e692d1addc3707b5f5deebbb8480125

Also Call of Duty 4 has some kind of lightmaps (on dynamic objects, but maybe on the world too) in 3D textures. Lighting is still far from great there though.

[x1nixmzeng]

Would it be possible to identify (log/prompt) any titles which use this feature? It would be useful to have other test cases I'm sure

[Triang3l]

Parsing all the logs from game-compatibility would be great, of course, maybe some day I will (this may be especially helpful for stacked textures, because I haven't seen any games using them, though @gibbed has). But if you want to do that, looks for dim: 3D or stacked: yes.

[Parovozik]

@Triang3l I tried your commits
'tiling untiling' was fixed subtitle fonts for Tales of Vesperia and much improved graphics for Diablo 3
'cube map sampling' was fixed some graphics parts for Sonic 2006 and Dante Inferno.
I think they works is fully safe (i tried many games).

[Parovozik]

Cube map works for Sonic 2006, Dantes Inferno and OutRun Online Arcade :)

[Triang3l]

According to this presentation https://www.docme.ru/doc/374601/next-generation-graphics-programming-on-xbox-360 stacked textures are sampled with 0…1 coordinates rather than PC-like 0…max_layer, that's probably a part of the reason why Halo 3 samples them with explicit unnormalized offset.

[Triang3l]

Except for proper meanings of VolMag/MinFilter in various cases (for stacked, Burnout Revenge uses linear to filter the color grading LUT along the blue axis, but this is just one specific usage case), this has been done in the Direct3D 12 GPU backend. Keeping this open for now due to filters.