Windows-rs: DX12 Support

[Vincent-P]

Hello,
I am currently experimenting with Rust and DX12 using this crate.
I am trying to port some samples from C++ to Rust (currently this one https://github.com/microsoft/DirectX-Graphics-Samples/blob/master/Samples/Desktop/D3D12HelloWorld/src/HelloWindow/D3D12HelloWindow.cpp).

I am having weird debug layer messages so maybe something is wrong with the bindings?
My code is fairly close to the C++ sample: https://gist.github.com/Vincent-P/4692d93cd3716878992e2c60e033f6fe.

Here are the error messages:

D3D12 WARNING: ID3D12CommandList::Dispatch: No threads will be dispatched, because at least one of {ThreadGroupCountX, ThreadGroupCountY, ThreadGroupCountZ} is 0. [ EXECUTION WARNING #1254: EMPTY_DISPATCH]
D3D12 ERROR: ID3D12CommandList::Dispatch: No pipeline state has been set in this command list.  The runtime will use a default no-op pipeline state. [ EXECUTION ERROR #1045: COMMAND_LIST_PIPELINE_STATE_NOT_SET]
D3D12 ERROR: ID3D12GraphicsCommandList::Reset: Reset fails because the command list was not closed. [ EXECUTION ERROR #544: COMMAND_LIST_OPEN]
D3D12 ERROR: ID3D12CommandList::ResourceBarrier: D3D12_RESOURCE_STATES has bits that mismatch support required from D3D12_RESOURCE_FLAGS. [ RESOURCE_MANIPULATION ERROR #524: RESOURCE_BARRIER_MISMATCHING_MISC_FLAGS]

I don't understand why there are messages about Dispatch() when the only commands are some ResourceBarriers and a ClearRenderTargetView...
The Reset() also is weird because I am definitely closing the command list before resetting it.
When starting the executable in Visual Studio, I am getting an exception in the first ResourceBarrier call.

let barrier = D3D12_RESOURCE_BARRIER {
    Type: D3D12_RESOURCE_BARRIER_TYPE::D3D12_RESOURCE_BARRIER_TYPE_TRANSITION,
    Flags: D3D12_RESOURCE_BARRIER_FLAGS::D3D12_RESOURCE_BARRIER_FLAG_NONE,
    Anonymous: D3D12_RESOURCE_BARRIER_0 {
        Transition: D3D12_RESOURCE_TRANSITION_BARRIER_abi {
            pResource: self.render_targets[self.frame_index as usize].as_mut().unwrap() as *mut _ as _,
            StateBefore: D3D12_RESOURCE_STATES::D3D12_RESOURCE_STATE_PRESENT,
            StateAfter: D3D12_RESOURCE_STATES::D3D12_RESOURCE_STATE_RENDER_TARGET,
            Subresource: D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES,
        },
    },
};
self.command_list.as_ref().unwrap().ResourceBarrier(1, &barrier);

Maybe the union or the call itself isn't working?

I am sorry if this isn't the right place to post this, but I am somewhat confident that the problem shouldn't be in my code as it is almost the same as the C++ sample which is running fine on my machine.

[kennykerr]

Sorry, I don't know anything about D3D12. It's possible there is a bug in the win32 metadata for those APIs but you'd have to root cause the issue first.

Perhaps @robmikh or @damyanp can spot something obvious.

[damyanp]

Superficially the code looks correct. My suspicion at this moment is that for some reason the vtables don't line up, so that, for example, your call to ResourceBarrier (or similar) actually becomes a call to Dispatch, with essentially garbage parameters.

If you're able to share a full project that I can easily compile and run it myself to see if I can reproduce the problem and confirm (or refute) my hypothesis.

[Vincent-P]

I have linked a gist to a full file but here is a full project: https://github.com/Vincent-P/rust-dx12
One thing to add is that the application actually runs (with synchronization errors) when the ResourceBarrier are commented and there are no errors about Dispatch() or Reset().

[damyanp]

Thank you - I can repro your issue on my machine and fortunately (or unfortunately depending on your perspective) it's not the problem I thought it was. I'll keep digging.

[damyanp]

The problem seems to be that the pResource field in the resource barrier is not getting set to the correct value. If I replace:

pResource: self.render_targets[self.frame_index as usize].as_mut().unwrap() as *mut _ as _,

with:

pResource: self.render_targets[self.frame_index as usize].abi(),

Then it seems to work. I'm afraid I'm very much a rust beginner (day 2) so this may be accidentally correct. I'm guessing that the first one ends up casting the wrong thing to a *mut c_void.

Another way I've found to write it, which might be nicer, is:

            let barrier = D3D12_RESOURCE_BARRIER {
                Type: D3D12_RESOURCE_BARRIER_TYPE::D3D12_RESOURCE_BARRIER_TYPE_TRANSITION,
                Flags: D3D12_RESOURCE_BARRIER_FLAGS::D3D12_RESOURCE_BARRIER_FLAG_NONE,
                Anonymous: D3D12_RESOURCE_BARRIER_0 {
                    Transition: D3D12_RESOURCE_TRANSITION_BARRIER {
                        pResource: self.render_targets[self.frame_index as usize].clone(),
                        StateBefore: D3D12_RESOURCE_STATES::D3D12_RESOURCE_STATE_PRESENT,
                        StateAfter: D3D12_RESOURCE_STATES::D3D12_RESOURCE_STATE_RENDER_TARGET,
                        Subresource: D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES,
                    }.abi(),
                },
            };

Note this uses the non-ABI version of D3D12_RESOURCE_TRANSITION_BARRIER and then calls abi() on it.

I'd be very interested to hear from someone more expert on rust and windows-rs comment on what the preferable approach would be here.

[kennykerr]

Yeah, so this is overly hard in Rust because unions in Rust are very limited. Nested types also don't exist in Rust, so I have to simulate those as well. I'd like to come up with a better way of projecting such structs into Rust but at the moment when the code generator comes across a C-style union it has to revert to using ABI types for its fields.

I'll see if I can come up with a simple example for how best to manage this sort of thing and ideally how to convince Rust to support this more elegantly.

[damyanp]

Many of the d3d types follow a pattern where there's a type-selector and then the union picks a single type. So in the case of resource barriers, maybe the right approach would be to have a D3D12_RESOURCE_BARRIER_TRANSITION and a D3D12_RESOURCE_BARRIER_ALIASING etc. which have the right byte layout / padding to match the D3D12_RESOURCE_BARRIER + union. Resource Barrier is a more simple one - look at D3D12_SHADER_RESOURCE_VIEW_DESC for a more extreme example of this pattern.

You'd want some way to make a heterogenous array of them as well.

[kennykerr]

We could add hints to the metadata to help Rust (and other languages) model this better. The metadata is generated by this project. Have a look at the metadata for the D3D types in a tool like ILSpy and let me know if you think there's potential.

[damyanp]

Unfortunately there doesn't seem to be anything in the auto generated metadata that would help here - which isn't too surprising, since there's nothing in d3d12.h that the metadata would be able to key off. I think we'd need to add something to Direct3D12.manual.cs. Maybe an attribute on each member of the anonymous union indicating how that one is chosen. Something like (warning, naive "I've forgotten how attributes in winmd work" code follows):

public struct D3D12_RESOURCE_BARRIER
{
    [StructLayout(LayoutKind.Explicit)]
    public struct _Anonymous_e__Union
    {
        [FieldOffset(0), Selector(Type, D3D12_RESOURCE_BARRIER_TYPE_TRANSITION)]
        public D3D12_RESOURCE_TRANSITION_BARRIER Transition;

        [FieldOffset(0), Selector(Type, D3D12_RESOURCE_BARRIER_TYPE_ALIASING)]
        public D3D12_RESOURCE_ALIASING_BARRIER Aliasing;

        [FieldOffset(0), Selector(Type, D3D12_RESOURCE_BARRIER_TYPE_UAV_BARRIER)]
        public D3D12_RESOURCE_UAV_BARRIER UAV;
    }

    public D3D12_RESOURCE_BARRIER_TYPE Type;

    public D3D12_RESOURCE_BARRIER_FLAGS Flags;

    public _Anonymous_e__Union Anonymous;
}

[Vincent-P]

Wow, I didn't expect such a quick answer!
The ABI parts (_abi structs, abi() and set_abi()) are definitely confusing.
I guess the preferred way is to use the non-ABI struct and then call .abi()?

[samlh]

A future possibility:

With some sort of "Selector" metadata, it might even be possible to generate the Rust type as an enum instead of a union - it would depend on whether the layouts match one of the 2 layouts listed here:
https://github.com/rust-lang/rfcs/blob/master/text/2195-really-tagged-unions.md#guide-level-explanation

(It would also depend on a currently nightly-only feature to set the discriminant: https://github.com/rust-lang/rust/issues/60553)

Something like this:

#![feature(arbitrary_enum_discriminant)]

#[repr(i32)]
enum D3D12_RESOURCE_BARRIER {
    Transition {
        Flags: D3D12_RESOURCE_BARRIER_FLAGS,
        Transition: D3D12_RESOURCE_TRANSITION_BARRIER,
    } = 0i32,
    Aliasing {
        Flags: D3D12_RESOURCE_BARRIER_FLAGS,
        Aliasing: D3D12_RESOURCE_ALIASING_BARRIER,
    } = 1i32,
    UAV {
        Flags: D3D12_RESOURCE_BARRIER_FLAGS,
        UAV: D3D12_RESOURCE_UAV_BARRIER,
    } = 2i32,
}

fn main() {
    let barrier = D3D12_RESOURCE_BARRIER::Transition {
        Flags: D3D12_RESOURCE_BARRIER_FLAGS::D3D12_RESOURCE_BARRIER_FLAG_NONE,
        Transition: D3D12_RESOURCE_TRANSITION_BARRIER {
            pResource: None,
            StateBefore: D3D12_RESOURCE_STATES::D3D12_RESOURCE_STATE_PRESENT,
            StateAfter: D3D12_RESOURCE_STATES::D3D12_RESOURCE_STATE_RENDER_TARGET,
            Subresource: D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES,
        },
    };
}

Playground: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=7a95fc3acbd0de16c26cacd74a0aebd9

---

The obvious downside is that it would only be possible if the layout is supported natively by Rust, which could cause inconsistencies between different structs.

[damyanp]

@samlh - it would be excellent to get to that point. Unfortunately, it looks like the D3D12_RESOURCE_BARRIER struct layout won't work here, since there's a "Flags" field between the Type and and the union:

typedef struct D3D12_RESOURCE_BARRIER {
  D3D12_RESOURCE_BARRIER_TYPE  Type;
  D3D12_RESOURCE_BARRIER_FLAGS Flags;
  union {
    D3D12_RESOURCE_TRANSITION_BARRIER Transition;
    D3D12_RESOURCE_ALIASING_BARRIER   Aliasing;
    D3D12_RESOURCE_UAV_BARRIER        UAV;
  };
} D3D12_RESOURCE_BARRIER;

I'm not sure the arbitrary_enum_discriminant feature can support this as it is?

[samlh]

@damyanp - That's correct, Rust doesn't natively support that layout.

In practice, the enum I wrote in my comment above matches the C struct's layout (same size and field offsets), but I agree that asking a code generator to figure that out is infeasible, and would likely just cause confusion for users in the long run.

I do think that a third-party project manually generating such bindings might be valuable - if I ever get some free time, I'll try experimenting with that.

However, I think I've sidetracked things a bit here - sorry about that!

---

I guess the preferred way is to use the non-ABI struct and then call .abi()?

My guess is that is the best approach for now - I'm not sure I really understand the purpose of the _abi structs, myself, so I can't be sure, though😄.

If things change to the point where the _abi structs are no longer exposed, then the code delta would be pretty small to remove the .abi() call.

[kennykerr]

I'm not sure I really understand the purpose of the _abi structs

The ABI struct ensures the proper memory layout _and_ calling semantics, stdcall, caller packs the stack, callee unpacks the stack, and so on. It must match the calling patterns used by a C-struct that is trivially destructible. The projected and ABI structs have the same memory layout and size but not necessarily the same calling semantics and lifetime. For example, projected types may implement Drop whereas the ABI struct won't. Rust then imposes further restrictions around unions that compound this further.

[Ciantic]

I have bunch of DX12 examples here:

https://github.com/Ciantic/dx12-learning/tree/master/examples

I just updated those to Windows-Rs 0.9.1

[MarijnS95]

@Ciantic Nitpick: That doesn't look like 0.9.1 but rather windows-rs master with massive namespace refactoring, scheduled for the next (0.10.0?) release :grin:

[Ciantic]

@MarijnS95 my repository is targeting the git version of windows-rs, which in the lock file says "0.9.1". But you can go back in history to see other versions too.

[MarijnS95]

@Ciantic Of course, because the git version hasn't been bumped yet - that obviously only happens when a new release is made. It is however significantly different from the published 0.9.1 release and as such claiming 0.9.1 compatibility isn't exactly accurate.

(@kennykerr perhaps again a request to at least tag existing commits with the version number even though you want to automate that - including creating releases - at some later point in time. I personally always go over .cargo_vcs_info.json to have an exact commit hash to tag. If you want I can push these to my fork - then all you need to do is git fetch --tags from my fork, and git push --tags to this repo)

EDIT: By the way something like cargo-release can help to push a (version+1)-alpha.0 after publishing a release, avoiding this confusion.

[Ciantic]

Yeah, okay. I think this should be closed though. DX12 support hasn't been a problem for a long time.