Proton: Compile without dxvk, with vkd3d.

[TaiTair]

Feature Request

See https://github.com/doitsujin/dxvk/issues/685
DXVK can be configured to use wine's dxgi.dll instead of its own dxgi.dll. This is good because it means we can run VKD3D alongside DXVK in a proton build. The main drawback is losing the ability to change DXGI-related config options provided by DXVK's dxgi.dll.
Not being able to change DXGI options is a big tradeoff.

Till we figure out a way to have both DXVK and VKD3D without compromising on config options could we have some compilation flags to indicate that we want either or?

I confirm:

• [x] that I haven't found another request for this feature.
• [x] that I have checked whether there are updates for my system available that
  contain this feature already.

Description

(This explanation of the compilation process is simplified and incomplete.)
When compiling proton:

• It compiles DXVK first.
• Then it compiles wine.
• Moves DXVK dlls into wine prefix.
• Sets dll overrides for wine.

There are at least 2 different ways of implementing my idea:

A. Replace

Using a build flag, check in the makefiles if build flag is set, if it is set then do not compile DXVK at all and do not override dlls, use wined3d with wine dxgi.dll. Compile wine with VKD3D support. Finish the rest of the build process.

B. Combine

Using a build flag, check in the makefiles if build flag is set, if it is set then compile DXVK without dxgi.dll and make DXVK use wine's dxgi.dll. Compile wine with VKD3D support. Add DXVK's d3d11.dll to the wine prefix. Override d3d11.dll. Finish building.

In both cases, the vagrant VM needs to be capable of compiling VKD3D from source which is currently not the case.

Suggested names for build flags: DXVKD3D, VKD3DONLY, etc... If no build flag is provided then build as normal with only DXVK.

Conclusion

This would help test VKD3D and get it out the door. Since it would need to be compiled manually at this stage, it would be fairly inaccessible to people who are not command-line savvy.

References [optional]

https://github.com/ValveSoftware/Proton/issues/138

[ryao]

Plagman said on the VKx discord that VKD3D would be added in the future. I am not going to try paraphrasing him, but my own understanding is that it is not ready to be merged yet. The last I looked, there were still hacks needed to get games working.

Also, this might depend on switching the OS target to Windows 10. That has its own problems because DXVK doesn't go past Direct3D 11.1 (Windows 8) while Windows 10 implies Direct3D 11.3 support. There are also things sort of working that just stop working the moment that Windows 10 is set in winecfg.

[TaiTair]

I had no doubt that this was already being worked on and that it's a complicated job. Just wish I could help test and gather reports. Right now the only way to test VKD3D is to compile wine with it and use wine instead of proton. Problem is my DirectX12 games are all on steam and I can't run steamPlay with wine instead of proton. Even if it was possible to use wine with steam instead of proton I would lose all the proton patches doing that.

Nice to know VKD3D is part of the plan though.

[ryao]

@TaiTair Andrew Eikum confirmed that Direct3D 12 support is coming in the future:

https://www.codeweavers.com/about/blogs/aeikum/2019/8/20/a-year-since-protons-launch

By the way, I would really like to see an easier way to test VKD3D too. The way that it is architectured makes developing Direct3D 12 applications for Linux possible, but unless there is a way of building it as a standalone DLL, it makes testing inside the official builds of Proton more difficult. Building it as a Windows DLL would also allow for testing from Windows, which could yield some interesting information in itself.

[TaiTair]

Hell yeah!
Building it as a Windows DLL would be very neat indeed. A d3d12.dll. It would be interesting to see if Denuvo-laden games would benefit in performance from running on Vulkan.
If it was built as a Windows DLL I could probably copy it in the wine prefix and apply dll overrides for DX12. Doesn't solve the issue of proton trying to load DXVK though. Does it matter if DXVK runs alongside VKD3D but VKD3D doesn't use dxgi.dll? Would they conflict?

[aeikum]

Yes, we have an internal branch that supports building vkd3d. The dxgi issue remains unsolved. The branch "solves" it by just always using built-in dxgi. This is likely how it will be solved officially, but we haven't thought this through 100% yet.

[TaiTair]

Would it be possible to open up that branch for the command-line savvy people? Having more insight might be helpful to figure out the dxgi issue.

[pchome]

@aeikum

The dxgi issue remains unsolved. The branch "solves" it by just always using built-in dxgi.

You could move dxgi.dll(.so) out of "wine-proton" tree, into the separate directory. Then just use WINEDLLPATH to set either Wine's or DXVK's dxgi. Should work.

_I currently use similar approach, but with WINEMODPATH patch. D{9,X}VK divided by parts (dxgi, d3d9, d3d10, d3d11) into separate directories, then just loaded on demand "on top"(instead) of default wine-proton libs._

[aeikum]

@pchome The catch is you don't know which DXGI you need for a given game until it's too late. And some games have both dx11 and dx12 renderers. Which do you use for those?

@TaiTair Sure, why not. Check out branch proton_4.11-2-vkd3d. If it breaks everything, you get to keep the pieces ;) Note the branch doesn't set DXGI to builtin; you can use the PROTON_USE_WINED3D switch in user_settings to do that.

[TaiTair]

Yeah using VKD3D for DX12 will only work on games that are 100% DX12. I have one in mind I'm going to test. @pchome that is a nifty solution! Thanks for sharing!

@aeikum Thank you so much!

[pchome]

@aeikum

The catch is you don't know which DXGI you need for a given game until it's too late. And some games have both dx11 and dx12 renderers. Which do you use for those?

I have no single dx12 game to test, but I have dxgi "switch" just in case. So I always on DXVK's dxgi and never tested how dx11 games work w/ Wine's built-in dxgi in Proton.

If you say my suggestion won't always work, then built-in dxgi by default should be ok too. I going to try built-in dxgi w/ dx11 games and "prepare" to the future changes.

[ryao]

@TaiTair It looks like VKD3D is being prepared for a new release:

https://github.com/ValveSoftware/Proton/commit/87321d3d3b02b2799a806af38f80fa3d1b87e06e

[imaami]

How challenging would it be to modify one of the existing dxgi implementations into something that dynamically switches between dxvk/vkd3d? What exactly are the obstacles?

[doitsujin]

What exactly are the obstacles?

The fact that you simply don't know in advance whether the application will create a D3D11 device or a D3D12 device (or both at the same time). Both APIs are built around DXGI.

[TaiTair]

Well actually we can compile dxvk as d3d10,d3d11 dlls which can then use Wine's DXGI dll. The real obstacle is how do we expose DXGI options while making dxvk use wine's DXGI dll? I think the solution might be to modify wine directly to expose DXGI options. Then it would be a bonus if VKD3D had a d3d12.dll but it would still work without.

We can run DXVK alongside VKD3D right now by chosing to compile the d3d10,d3d11 DLL version of DXVK and then using those DLLs. That's not the problem anymore.

@doitsujin Could you explain how to build DXVK without its dxgi.dll?

[ryao]

@TaiTair You don't. You just don't use its dxgi.dll.

[TaiTair]

Ah gotcha. It figures it out itself.

Does that mean that when I go PROTON_USE_WINED3D=1 it still uses DXVK and not the openGL implementation of D3D?

[ryao]

@TaiTair That uses WineD3D, which translates to OpenGL. It does not use DXVK. I am not sure why you would think that it would, but it is easy to check by seeing that DXVK_HUD stops working when you do that.

[TaiTair]

I was assuming that because DXVK can use wine's dxgi dll. The override forces DXVK off. So if I want to force DXVK alongside VKD3D I just have to replace dxgi.dll in the wine prefix?

[doitsujin]

I just have to replace dxgi.dll in the wine prefix?

No. I don't know how to do this in proton, but with regular wine, not copying up the DXGI DLL and setting the override will do the trick. You could try removing dxgi.dll from steamapps/common/Proton 4.11/dist/lib64/wine/dxvk and see if things still work.

I think a potential "solution" to the DXGI mess might be to ship both and use the DXVK one for the handful of games that need it, without touching wine code, while defaulting to wine's dxgi. Most of DXVK's DXGI options are actually implemented in the D3D11 module these days, not in DXGI itself. The only exception to that is vendor ID overrides.

There are some more issues that will be tricky to solve, e.g. implementing DXGI's memory budget functionality properly.

[TaiTair]

No. I don't know how to do this in proton, but with regular wine, not copying up the DXGI DLL and setting the override will do the trick. You could try removing dxgi.dll from steamapps/common/Proton 4.11/dist/lib64/wine/dxvk and see if things still work.

Tried that, insta-crash on game launch.
EDIT: Setting dll override for dxgi to built-in with winecfg seems to work. I have no way of knowing for sure yet though because the game I'm trying to run hangs. Gonna try with a different game but I have to download it first.

I think a potential "solution" to the DXGI mess might be to ship both and use the DXVK one for the handful of games that need it, without touching wine code, while defaulting to wine's dxgi. Most of DXVK's DXGI options are actually implemented in the D3D11 module these days, not in DXGI itself. The only exception to that is vendor ID overrides.

So what we need to do is expose vendor ID overrides inside wine's DXGI and stop using DXVK's DXGI. For the ones that are in the D3D11 module could we use dxvk.conf?

There are some more issues that will be tricky to solve, e.g. implementing DXGI's memory budget functionality properly.

Can that be implemented in Wine or in D3D11.dll?

[oscarbg]

can we expect a prebuilt proton from proton_4.11-2-vkd3d branch to be selectable from Steam soon?

[ryao]

@oscarbg No. If you want that, ask @imaami or @GloriousEggroll to provide one that you could install and then select. I believe that they are both doing third party proton builds.

[doitsujin]

Can that be implemented in Wine or in D3D11.dll?

No, this cannot be moved to d3d11.dll since it's IDXGIAdapter functionality which doesn't involve the actual D3D device. Implementing this in wine using Vulkan functionality might be possible, but I'm not sure how meaningful the results would be since at least AMD drivers cannot report system-wide memory allocation right now (also, wine's dxgi isn't based on Vulkan). Using OpenGL might work, but requires driver-specific code and likely won't be accurate either.

[pchome]

@TaiTair

I just have to replace dxgi.dll in the wine prefix?

No. I don't know how to do this in proton, but with regular wine, not copying up the DXGI DLL and setting the override will do the trick. You could try removing dxgi.dll from steamapps/common/Proton 4.11/dist/lib64/wine/dxvk and see if things still work.

In proton it's similar: https://github.com/ValveSoftware/Proton/commit/ab8574645a8836003218b4930d6b145cb2cd6f23 (Note: it's not an commit in Proton, it's demo patch)
This is how it's likely will be done.

So what we need to do is expose vendor ID overrides inside wine's DXGI and stop using DXVK's DXGI.

AFAIK the trick known as "NvAPI HACK" already used by Proton, which in theory should do the same as DXVK's "NvAPI HACK" (expose AMD gpu for all games). The only thing likely should be done -- sync overrides with DXVK's list. Which likely became an additions to wine.inf, in form e.g. HKCU,Software\Wine\AppDefaults\GameName.exe\Direct3D,"VideoPciVendorID",0x10001,0x000010de, but I'm not sure it will be "straight forward" conversion.

[TaiTair]

Glad to see a constructive conversation about the subject. I'm sure we will find a good way. Using wine.inf seems like it could work.

Tried launching Watch_dogs 2 which is supposedly DX12, hangs 100% on one logical cpu.
Tried launching Sniper Elite 4 which is DX12, insta-crash.
If you have ideas for other games to test let me know.

[imaami]

can we expect a prebuilt proton from proton_4.11-2-vkd3d branch to be selectable from Steam soon?

I'm building Proton-i 4.13-4-vkd3d right now. Will upload a release package when it's ready.

(Ugh, these version number/variant combos are getting ridiculous.)

[oscarbg]

@ryao thanks..
@imaami thanks.. will try&test soon..

[oscarbg]

@imaami I see build is published.. great..
if you feel adventurous for next proton vkd3d build you could add this VKD3D patch that adds Shadow Tomb Raider DX12 support:
https://gist.github.com/Plagman/ea37697137e32cf88c6275391b862ead
according to:
https://www.youtube.com/watch?v=7SFvFFhpwbI
really I tested also and it's true with this patch SOTTR DX12 works..

[ryao]

@TaiTair

I just have to replace dxgi.dll in the wine prefix?

No. I don't know how to do this in proton, but with regular wine, not copying up the DXGI DLL and setting the override will do the trick. You could try removing dxgi.dll from steamapps/common/Proton 4.11/dist/lib64/wine/dxvk and see if things still work.

In proton it's similar: ab85746 (Note: it's not an commit in Proton, it's demo patch)
This is how it's likely will be done.

So what we need to do is expose vendor ID overrides inside wine's DXGI and stop using DXVK's DXGI.

AFAIK the trick known as "NvAPI HACK" already used by Proton, which in theory should do the same as DXVK's "NvAPI HACK" (expose AMD gpu for all games). The only thing likely should be done -- sync overrides with DXVK's list. Which likely became an additions to wine.inf, in form e.g. HKCU,Software\Wine\AppDefaults\GameName.exe\Direct3D,"VideoPciVendorID",0x10001,0x000010de, but I'm not sure it will be "straight forward" conversion.

That will not work because Wine implements that in WineD3D rather than dxgi:

https://github.com/ValveSoftware/wine/blob/b83f9e037077e84bef04735276596fddaadf51db/dlls/wined3d/wined3d_main.c#L314

[pchome]

@ryao

That will not work because Wine implements that in WineD3D rather than dxgi

I can be wrong, but I guess that will work because of the same reason.

https://github.com/ValveSoftware/wine/blob/b83f9e037077e84bef04735276596fddaadf51db/dlls/dxgi/adapter.c#L375

[ryao]

@pchome Nice catch. You are right. I didn't think wined3d was loaded when direct3d 11 was being provided by DXVK. It appears that I was wrong.

[doitsujin]

@oscarbg That patch really hurts overall performance though. It shouldn't be applied unconditionally.

[MegWATTT]

An other problem is the inability to select a specific GPU if wine's dxgi is is used.