Projectreunion: Please provide bindings for DirectX.

[pjmlp]

Proposal: Bindings for DirectX.

Since the introduction of WinRT, later UWP, non C++ developers have suffered from lack of access to DirectX, even though they could be easily improved as Windows Runtime Components, given that they are already based on COM.

Summary

Provide the necessary metadata for Rust, C#, and JavaScript developers also to be able to integrate DirectX as Windows Runtime Components into their applications without being forced to write C++ code.

Rationale

Since XNA was put to sleep, we have been managing with community projects like Monogame, SharpDX, Vertice and many others.

They usually tend to run out of steam or fail to keep in sync why the latest DirectX improvements.

So anyone that wants to bind DirectX into their applications is eventually forced to either deal with the faults of such implementations, or having to learn C++ alongside its variants, initially C++/CX, nowadays C++/WinRT and try as much as possible to create an interoperability layer for their application.

Other DirectX related technologies like XAudio and DirectML already offer Windows Runtime Components,

There is also the Win2D project that seems to have become stale, eventually because it wasn't part of DirectX team efforts to start with.

So by having official Windows Runtime Components as part of the official APIs for DirectX, we would be assured to have up to date bindings, integrated into Visual Studio tooling, available to all Project Reunion official languages, instead of being only available to C++ developers.

Scope

| Capability | Priority |
| :---------- | :------- |
| This proposal will allow developers to integrate DirectX into their applications | Must |
| This proposal will allow developers to develop 3D games in managed languages | Could |
| This proposal will allow developers to use 3D in WPF, UWP and WinUI without writing C++ code | Could |

Important Notes

• SharpDX (now discontinued)

Open Questions

[Nirmal4G]

There was a DirectX projection for C++ by @kennykerr during the days for Windows 8. I would like a similar effort with @microsoft/xlang.

[smourier]

Just mentioning this other community project (of my own) "DirectN": https://github.com/smourier/DirectN the difference with other projects is the whole files (more than 7000) are generated from the SDK so automatically can keep up with latest versions. It allows you to use most low level API from C# pretty easily without any runtime or projection, it's just source code. For example, I use Direct2D and DirectWrite without too much effort.

However, note that many improvements in latest Windows are not available at all even from plain C++ as "old" COM interface, only to WinRT. So I can't integrate them into that project.

For example: Direct Composition was a simple COM API that has evolved as Windows.UI.Composition (like a "dcomp v2") which is a WinRT only API (which is relatively easy to use also from plain C# with external Dispatcher Queue as explained here: https://docs.microsoft.com/en-us/windows/apps/desktop/modernize/using-the-visual-layer-with-windows-forms). The only technical way to get full integration with DWM (like for the back drop brush) is to use this Windows.UI.Composition WinRT API.

Now it's becoming Microsoft.UI.Composition I believe with WinUI3.

I wonder what these 3 levels of API are going to become with Reunion? Will we be able to use all these back again using "plain" C or C++ or even "old" .NET (with interop p/invoke like I do) without adding runtime and projection layers?

Or is the general way of adding API to Windows will still be "new Windows API" => "only to WinRT"? For example if there's a DirectX 13 or 14, will it be for WinRT only?

[pjmlp]

@smourier thanks for commenting here, I wasn't aware of DirectN.

[jonwis]

On the WinUI3 and Microsoft.UI.Composition front, the interop overhead should be extremely low from all runtimes using modern projections. Do C#/WinRT or C++/WinRT meet your projection needs? We aim for the projection "cost" to be extremely low and not require you to pick a specific runtime. Issue #18 asks for a "flat C" projection of metadata-based APIs for "old" callers who cannot use C++/WinRT.

@smourier - you said:

However, note that many improvements in latest Windows are not available at all even from plain C++ as "old" COM interface, only to WinRT. So I can't integrate them into that project.

Can you help me understand more? WinRT types are available to all COM-ready clients through "ABI" headers that take HSTRING and UINT32 and emit HRESULT and IAsyncAction. WinRT's based on the COM interface model, with IInspectable being the generic base type instead of IUnknown (although unsurprisingly, IInspectable derives from IUnknown.) It's still feasible, although not terribly pleasant, to use the most modern Windows Runtime APIs from assembly the same way you'd have used COM interfaces.

(I've also reached out to the DirectX team for their commentary, stay tuned)

[smourier]

@jonwis - actually, that point "So I can't integrate them into that project" is not a blocker. As you say it's possible to use WinRT from C# today.

It was more a remark about the fact that if you want to write an application that uses for example Direct2D+DirectWrite (which is a COM interface only) and Direct Composition (for which the latest version is a WinRT interface in Windows.UI.Composition), you have to mix both worlds: 1) it's quite complicated and 2) it's not really documented although there are "interop" sample about this (ICompositionDrawingSurfaceInterop).

There shouldn't be a need to use "interop" to have these world talk. It would be ok if everything was WinRT (and accessible from desktop apps) I guess.

[jonwis]

if you want to write an application that uses for example Direct2D+DirectWrite (which is a COM interface only) and Direct Composition (for which the latest version is a WinRT interface in Windows.UI.Composition), you have to mix both worlds

It would be ok if everything was WinRT (and accessible from desktop apps) I guess.

So if we had metadata-based definitions of all those APIs, and a consistent projection of that metadata to your target languages, that would help?

One of Project Reunion's explicit goals is to define as many APIs using metadata as possible, existing "flat C" or "ABI COM." Once we have that metadata tooling like the */WinRT series can project a consistent API surface for your favorite languages. This would eliminate ugly "code seams" where you have to transition from C# to ABI COM to WinRT to p/invoke.

[pjmlp]

@jonwis Basically what I am voicing is a recurring request since the introduction of WinRT and rampdown of Managed DirectX and XNA frameworks from the respective teams.

Since the introduction of WinRT there has been a barrier for certain APIs, like DirectX, whose access is only possible via C++, instead of making it visible via WinRT projections.

So since Windows 8, accessing DirectX from .NET has been a mix of:

• Just use a wrapper library like SharpDX. Usually community driven, playing catch up with official API, most of them one person show, and eventually unmaintained (like it has happened to SharpDX);

• Make use of a game engine like Unity or Stride (former Xenko). Not really something desirable when the goal is to integrate DirectX capabilities into regular LOB applications;

• Make use of Win2D. Only usable for 2D scenarios and it is unclear how much we should rely on it going forward

• Write the bindings ourselves in C++. Usually a pain for those .NET developers that aren't confortable using C++, an alternative that has gotten worse with the dismissal of C++/CX, given that C++/WinRT tooling is still found lacking versus C++/CX (no VS syntax highlighting and completion support for IDL files, manually copying generated files into VS projects, manual boilerplate for XAML integration)

With the evolution of UWP, the _Windows.Graphics.DirectX.Direct3D11_ finally appeared, but it is only meant for interoperability with Visual Layer and little else.

Ideally DirectX would be fully exposed under _Windows.Graphics.DirectX.Direct3D11_ and _Windows.Graphics.DirectX.Direct3D12_ and so forth, so that any .NET developer could be able to directly access DirectX capabilities without having to learn C++ or rely on game engines for LOB scenarios.

Same applies to other Windows APIs that aren't currently exposed via WinRT projections.

Naturally I am not placing this issue for myself alone, I already have my solution in place, just as something that I think the Windows developer community at large would benefit from, regardless of which language projection they make use of.

Now if this is something that Project Reunion projections will enable out of the box, then it would already be a quite good solution.

[jonwis]

Thanks for all the discussion on this issue! I talked to Shawn on the DirectX team about this and they say:

This is an interesting topic but we’re mostly focusing on native clients right now. DirectX’s API has a number of constructs that don’t easily port to non-native runtimes like JS or C#. We suggest you use a layered approach where your direct calls into the DirectX API surface are from C++ and you provide an app-specific interface from that component into your application.

We’re happy to help you use Project Reunion tooling – WinMDs, C++/WinRT implementation of projections, metadata-based projections to other languages – to produce that glue layer, and if it evolves over time to meet more apps needs we’d welcome it as a Project Reunion Family member. Check out https://github.com/microsoft/RegFree_WinRT or other http://github.com/microsoft/xlang samples for how to create IDL-based object definitions and a CLR-ready projection.

Your best bet out - right now - for GPU-accelerated 2D rendering (or inspiration for your wrapper) is https://github.com/microsoft/Win2D

@smourier - could your tool produce IDLs, then an implementation with C++/WinRT, and then use C#/WinRT to access it from the CLR?

[jonwis]

Shawn also suggests checking out their Discord - https://discord.com/invite/directx for further discussion.

[smourier]

I think the question was not "how can we do that?". But more "we know how to do it today, but it's far from being straightforward, please include DirectX in your thinking".

Speaking for myself, I know how to do it today, not specifically looking for hints nor tech support, more for API unification which I though Reunion was about.

Now you're making clear DirectX (and friends) are not part of project Reunion, Duh....

Win2D is only for UWP, it's not really a product as such (it has its own doc, it's not really updated), plus it has some terrible dependencies over VCRT, so it's nothing but a general answer. In fact I though Reunion was about that: getting rid of "things" such as Win2D.

[pjmlp]

Basically it means that despite WinRT and UWP, the DirectX team has no plans to support anything beyond C++ and plain old COM, and it is up for us to do the work for free as Microsoft cannot afford it.

I wonder what is the goal of Project Reunion after all, or even how Rust/WinRT will also have their own community provided bindings for DirectX.

Well, I guess it is settled then, thanks for clarifying, at least it wasn't closed right away.