Gfx: [RFC] BAL: Graphics API framework

[msiglreith]

Introduction

We have multiple (upcoming) clients of low level gfx-rs with strong performance or validation constraints. Our current HAL interface strongly adapts the Vulkan API together with the constraints burdened upon the backends (e.g pipeline layout compatibility).

gpuweb will have a higher level API giving room for backend specific optimization without having to go through the Vulkan/HAL bottleneck. Additionally, it will require specific validation steps in contrast to other APIs. Other clients like Amethyst or WebRender could also benefit from optimization in case of higher level or small specialized interfaces.

Concept (BAL)

Gradually lift the bottleneck imposed by HAL by adding a smaller domain specific framework for building graphics API on top of the lower level APIs. This framework, called BAL, should be seen as floating target not stabilizing anytime soon (or at all) and provide basic backend-specific and agnostic primitives for constructing higher level APIs (portability, WR, gpuweb). These primitives might be special cache structures required for only a part of the frontend APIs.

Implementation

The transition towards the new BAL structure can happen gradually by starting to publically expose more primitives from the backends, shifting HAL slowly towards a higher level frontend API for Rust users only and allow users to hookup a basic Vulkan library (e.g Ash) to be able to start working with a more or less fixed API.

Graphics API implementation

Being able to quickly implement and optimize a graphics frontend API (e.g gpuweb) should be the main goal of BAL. A library author could start working with the Rust Vulkan library for implementation of the Vulkan path, automatically gaining support for the other backends due to portability initiative. Certain fast paths can be sped up writing backend specific paths on top of the primitives provided by BAL.

Future of HAL

HAL would be mainly interesting as safer and rustic wrapper of Vulkan or moved more towards a middleground of all APIs. It should be seen as low overhead frontend but not as the only API exposed from gfx-rs. Future higher levels abstractions like gpuweb could be more appealing for certain user groups due to slimmer and simpler interface.

[zakarumych]

So BAL will be a collection of APIs each backend supports only partially.
And HAL will be essential subset of BAL that is supported by all backends.
Am I understand this correctly?

[msiglreith]

@omni-viral At the beginning, yes. I'm feeling that we should shift HAL towards the 'frontend' side and treat it like WebRender, gpuweb and portability which basically just stack upon the primitives provided by BAL as I would like dropping the Vulkan backend completely as BAL backend and users (and ourselfes) just directly act upon Vulkan.
So essentially BAL would be some kind of utility library over D3D12, Metal, GL, ... thin wrapping components with identical semantics and also directly exposing certain features (e.g possibly RootSignatures or DescriptorsHeaps on d3d12 side) by the backend crates.

// portabilty draw command
fn gfxCmdDraw(..) {
   #[cfg(features = "dx12")]
   {
       buffer.user_data.flush();
       buffer.command_list.draw();
   }
   #[cfg(features = "vulkan")]
   {
       buffer.draw();
   }
   ...
}

// gpuweb draw command
fn gwCmdDraw(..) {
   validate();
   #[cfg(features = "dx12")]
   {
       // insert transition, UAV, aliasing barriers possibly
       buffer.validate_dx12();
       buffer.command_list.draw();
   }
   #[cfg(features = "vulkan")]
   {
       // insert pipeline barriers
       buffer.validate_vk();
       buffer.draw();
   }
   ...
}

The latest discussion about RawC, Ash, HAL, .. was often influenced by what our main user base will maybe look like and trying to somehow satisfy all their constraint but IMO that's not really possible. Therefore shifting about more of the complexity onto the clients (similar to the low level APIs heh). At the end users will choose which API/frontend fits best for them.

[zakarumych]

@msiglreith Could you also add a snipped for user who wants to write code once for all platforms.

[zakarumych]

Would it be possible for user with backend-agnostic code to add some backend-specific code without rewriting stuff all over the place?

[msiglreith]

@omni-viral For writing backend agnostic code on top one of the 'frontends' (HAL, Ash, gpuweb in the future) should be used. 'BAL' is more like an implementation detail/layer allowing us to provide better support for other APIs. Somewhere between a Rust D3D12/Metal/.. wrapper and current HAL.

The easiest way for being able to add backend specific paths would be via Vulkan(Portability)/Ash due to it's special status of being backend and frontend at the same time. So in the first iteration relaying on the portability implementations and in further iterations add backend specific parts. HAL for example could also expose some backend specifics but this would then go through the HAL API as some kind of extension.

The difficulty with backend specific paths when building on top of frontends whould be requiring us to expose internals of the frontend implementation. I don't think BAL would have some sort of 'Backend' trait with different resource traits but rather frontends building for example the CommandBuffer via some primitives:

// HAL: dx12
pub struct CommandBuffer {
   command_list: bal::dx12::CommandList,
   pipeline_cache: bal::dx12::PipelineCache,
   gr_user_data: bal::dx12::UserDataCache,
   cp_user_data: bal::dx12::UserDataCache,
   ...
}

impl hal::CommandBuffer for CommandBuffer {
  fn draw() {
     self.gr_user_data.flush();
     self.command_list.draw();
  }
 ...
}

// gpuweb dx12:
pub struct CommandBuffer {
   command_list: bal::dx12::CommandList, // Or a deferred command list ..?
   pipeline_cache: bal::dx12::PipelineCache, // for example gpuweb using 2 slot model but no pipeline layout compatibility
   gr_state_context: bal::dx12::StateValidator, // do some validation stuff on graphic action commands
   ... 
}

impl gpuweb::CommandBuffer for CommandBuffer {
   fn draw() {
     self.transition_resources(Cmd::DRAW);
     self.gr_state_context.validate();
     self.command_list.draw();
  }
   }
   ...
}

[zakarumych]

So if I use hal there wouldn't be easy way to write backend specific code or hal will expose underlying structures in non-generic environment. Like if I have instance of gfx_backend_dx12::CommandBuffer instead of opaque <B as Backend>::CommandBuffer will I be able to access dx12 specific methods?

[Elzair]

If I wanted to write cross-platform graphics code, I should use the Vulkan portability library. Internally, portability will map its commands to either the Ash Vulkan wrapper or BAL (which can use either Direct3D12 or Metal {or OpenGL 2}). Is this correct?

[msiglreith]

So if I use hal there wouldn't be easy way to write backend specific code or hal will expose underlying structures in non-generic environment.

That's rather a design decision on how HAL would look like at the end (just a rustic vulkan wrapper?). But I would rather avoid exposing internals and I don't see that much gain from it. We can also think later on how we want to define some extensions to provide better support for backend details.

If I wanted to write cross-platform graphics code, I should use the Vulkan portability library.

Any of the mentioned frontend libraries:

• HAL: API not fixed but more rustic
• Vulkan PI: Fixed API but more like raw C bindings
• (WebRenderer: Very domain specific frontend)
• gpuweb-rs: In the future, higher level than Vulkan but probably easier to use

Internally, portability will map its commands to either the Ash Vulkan wrapper or BAL

Exactly! To clarify, I don't think BAL will look like the current HAL in terms of having some shared trait implemented by all backends. Rather portability will map to Vulkan, BAL-D3D12, BAL-Metal, BAL-OpenGL. The BAL prefix means some zero abstraction wrappers above the native backend APIs + common primitives for easing the implementation of the frontend APIs.

[anderejd]

I'm a little confused by this thread, here are some questions.

Will HAL be supported or will it go away?
Is HAL considered a failure or is it still the future of graphics programming in rust?

[kvark]

@anderejd this is a non-trivial topic, it's totally ok to be confused. Let me clarify

Will HAL be supported or will it go away?

HAL will be supported, see this item in the frontend list of the previous comment:

HAL: API not fixed but more rustic

Is HAL considered a failure or is it still the future of graphics programming in rust?

We've been giving HAL an evaluation in #2206, and the conclusion (wasn't obvious at all!) we reached is to move on with HAL that is built using BAL components.

It's hard to call it "the future of graphics programming in rust" just yet. There are many different clients and requirements. Less demanding users would just use existing engines. A bit more demanding would use something like WebGPU native library. Most demanding would use HAL. And super hardcore would use BAL. As you can see, it's not clear how much space there is for HAL in this scheme. At least we can run Warden tests on it and provide examples, plus Amethyst and WebRender already have ports on it, so it is useful.

[anderejd]

That clears up my confusion, thank you! :)