Godot: Vulkan Ray Tracing?

[ViktorEvil]

Will it be possible to implement ray tracing with the swap to the Vulkan renderer in Godot 4.0?

Nvidia Dev Blog

[Madaraz1024]

ray tracing later 4.1 or 4.2+ future (depends plan @reduz)

[ray314]

Can ray tracing be implemented using Radeon Rays?

[Calinou]

@ray314 See https://github.com/godotengine/godot/issues/21315. It'd make sense to use it for baking lightmaps, not so much for rendering everything (as very few GPUs can currently afford this at 60+ FPS).

[hackini]

Path tracing will (and is already) be happening in gaming. However it is still very new and not well clear which direction it is going.

• Currently the Vulkan extension for raytracing is Nvidia-only. I don't think we should waste developers time to implement Nvidia-only solutions (same applies for CUDA, OptiX and all closed, non-standard- proprietary Nvidia frameworks).
• Radeon Rays is definitively the way to go for light baking (because it doesn't use CUDA and works on all CPU/GPU) but not the state of the art for real-time. For real-time we really need to use the dedicated hardware.
• Currently only Nvidia offers dedicated hardware for raytracing. I hope AMD will follow and that the Vulkan extension for raytracing will get standardized to work on both cards.
• Realtime path-tracing only make sense if we denoise it. There has been the hype over Optix because it is AI based but actually it might be an overkill. The best results currently have been reached with classical denoising with gradient-based temporal accumulation (https://cg.ivd.kit.edu/atf.php).

So nothing is really hard to implement (even the de-noising papers are not that hard to understand) but the main problem is that AMD still has no dedicated hardware for ray-tracing and has not standardized a Vulkan API. So I think we should wait for this to happen, especially since maybe the API will change a lot when (if) it get standardized.

Concerning the denoising, as long as you can access the N last frames' depth-buffer & normals buffer & Albedo buffer that's all you need (everything is done in image-space)

[Megalomaniak]

* For real-time we really need to use the dedicated hardware.

Compute shader based solutions should work too. Crytek has a demo running 30 fps _at 4k_ on Radeon Vega 56.

[hackini]

* For real-time we really need to use the dedicated hardware.

Compute shader based solutions should work too. Crytek has a demo running 30 fps _at 4k_ on Radeon Vega 56.

woa that's a great achievement. Maybe we don't need dedicated hardware after all then. If AMD plans on never implementing a dedicated API then we could go that way. But it'd be a pity to implement everything ourself and 6 month later AMD standardizes an API in Vulkan similar to Nvidia's

[NathanWarden]

I'm really excited about ray tracing, but at the same time my opinion is that ray tracing in games is so new that we should wait until it's more standardized before the core team spends time on it. There are many other general improvements that could be made that are likely much more important.

Obviously if someone else wants to implement it then that's another thing :)

[UnleavenedEagle]

Why is it a problem, that AMD does not support it? The engine is for, and to be compliant with Vulkan, and not NVIDIA or AMD.

[hackini]

Why is it a problem, that AMD does not support it? The engine is for, and to be compliant with Vulkan, and not NVIDIA or AMD.

As far as I understand, Vulkan has both prefixed non-standard vendor extensions and core standard specifications. So Nvidia's extension is not part of standard Vulkan, this is a non-standard extension to Vulkan.

Vendor extensions can become core standard specifications once/if vendors eventually agree on the API (see this stackoverflow answer).

This seems very much the same as the -ms- , -webkit-, -apple-, etc. prefix on the web, if something is really useful to everyone, they might end up in the CSS spec if the vendors manage to agree on the API.

[hackini]

This might be also an idea to look into if we have path tracing : Ray-Traced Irradiance Fields . Basically instead of going full path-tracing for all the screen pixels, we could keep the classic GIProbe workflow, and only path-trace them.

[bhack]

https://github.com/KhronosGroup/Vulkan-Docs/issues/686

[Anutrix]

Something worth checking out: https://openimagedenoise.github.io/ .
Couldn't find a right issue to mention this.
https://blenderartists.org/t/intel-open-image-denoise-released-on-github/1144893

[Hopefullyidontgetbanned]

shouldn't denoising be possible using SVGF Denoising Filter, it produces amazing results with 1SPP, this blog post is a good way to get more information on it.
Blog Post

[devshgraphicsprogramming]

Can ray tracing be implemented using Radeon Rays?

I've just been in Radeon Rays land, and I can tell you its not pretty. RR 2.0 MIT-Licesnsed old version is broken (no working Vulkan back end and broken two level BVH).

RR 3.0 is what you'd want, but its closed source (unless that changes when you get a commercial license) plus it has significant drawbacks of not offering programmable Intersection and an Any Hit routine (unless you get the source, but then whatever you modify/fix you can't distribute).

The lack of Any Hit is a huge pain in the ass, because it makes Alpha-Tested geometry (light mapping for trees and foilage) as expensive as multiple mirror reflections, you basically have to stop the ray, load it up in compute, check the texture, write out the ray and start it again if alpha test fails. This is a PITA for even shadow and AO rays, as it will slow it down by unimaginable orders of magnitude, its like doing depth peeling for transparency.

Intersection routine (shader) would be nice for analytical area lights (they would converge faster), but thats only a small speed up compared to Any Hit.

But it'd be a pity to implement everything ourself and 6 month later AMD standardizes an API in Vulkan similar to Nvidia's

VK_NV_raytracing is DXR API literally to the T. AMD supports DXR via a compute fallback layer just like Intel. If a VK_EXT_raytracing or VK_KHR_raytracing comes it will have very similar shader syntax (if not verbatim same but with NV suffices dropped) and similar C API.

If you tried hard enough you could most probably cook up a compiler extension to glslang or shaderc that would compile the VK_NV_raytracing syntax Miss, Closest Hit, Any Hit, and Intersection shaders into a single compute shader that would run in a single dispatch (as opposed to 1 dispatch per ray-call-stack-level), it would need a scratch buffer for the ray queue and some special re-indexing of descriptors in its descriptor sets and either EXT_descriptor_indexing or EXT_buffer_device_address+some sort of bindless texturing.

[devshgraphicsprogramming]

Compute shader based solutions should work too. Crytek has a demo running 30 fps at 4k on Radeon Vega 56.

That's probably what they used to test the DXR fallback layer ;)

[UnleavenedEagle]

Well, Vulkan has now announced ray tracing support (without a vendor extension).

[Megalomaniak]

Right, and I think the thing in godot that might need the RT treatment the most right now would be shadows, but we'll see if someone emerges wanting to implement raytracing in some form for Godot 4.x or later. I'm sure it's just a matter of time though.

[bhack]

https://github.com/KhronosGroup/Vulkan-Docs/issues/1206

[clayjohn]

Great discussion. As pointed out in one of the earlier comments, Ray Tracing in Godot will likely be targeted for 4.1 or 4.2. Right now the Vulkan raytracing extension is not well supported. Once it gains wider support and more hardware can take advantage of it, we will reevaluate.

For now:
Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.

The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.

If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!