Dxvk: VK_ERROR_DEVICE_LOST

[MrChebik]

I've already met this problem VK_ERROR_DEVICE_LOST in games:

• Batman: Arkham City - After watching the initial videos, who released the game.
• Batman: Arkham Origins - During the game launch, after the game logo.
• Dark Souls III - I can start playing. I go to the first monster, and hang.
• Fallout 4 - Loading a new game.
• Far Cry 5 - Loading a new game.
• Outlast 2 - Beginning of the game, hangs after listening to a fragment of the memory of a girl.
• Prey: Mooncrash - I can start playing. A load of equipment arrives to me, I insert it for virtual reality, after I sit down on an armchair. There is a load of people to save, and hangs.
• Skyrim Special Edition - Occurs after a minute of driving on a wagon.

Software information

All settings in both games to low, resolution to 864x486.
Batman: Arkham Origins:

• Tried with window.
• Tried without Vsync.

System information

• GPU: Intel HD 5500
• Driver: Mesa 18.1.2
• Wine version: 3.10-staging
• DXVK version: 0.61

Log files

Batman: Arkham City

• debug
• d3d11.log
• dxgi.log

Batman: Arkham Origins

• apitrace x32
• d3d11.log
• dxgi.log

Fallout 4

• apitrace x64
• d3d11.log
• dxgi.log

Outlast 2

• debug
• d3d11
• dxgi

Prey: Mooncrash

• debug

[doitsujin]

Can't reproduce, please run the games with Vulkan validation layers enabled and post the full output.

[MrChebik]

This is strange, but Fallout 4 does not display as much information as Batman: Arkham Origins. Google Drive

[doitsujin]

And the errors in Batman all occur after the device lost error, making them effectively meaningless. Can't really do anything.

That said, does your Broadwell GPU have full Vulkan support? Because Haswell doesn't. It'd be great to know if this works on Skylake or not.

[MrChebik]

I can not say for sure, but I do not have any problems with Dota 2, The Talos Principle on Vulkan.

[jekstrand]

Yes, Broadwell is fully Vulkan conformant and stuff should work there. Before filing any bugs against mesa, though, please run with the validation layers and ensure that validation is happy. If it is, please file a mesa bug and we'll look into it. Broadwell is, however, somewhat more limted than Sky Lake so it could be that DXVK is hitting unexpected limitations.

[jekstrand]

I've successfully reproduced this on Broadwell. Unfortunately, as far as I can tell, it's some sort of DXVK synchronization issue. Most likely destroying a resource (pipeline, command buffer, image view, etc) while it's still busy on the GPU. Hacking up the driver to do a vkQueueWaitIdle after every vkQueueSubmit seems to make the hang go away. I have no idea why this would affect Broadwell and not Sky Lake.

I'm not sure if this information is useful but it seems to happen right around the time when the game decides to upload a bunch more data. In the Skyrim case, it often hangs right as you approach Helgen and the game suddenly needs to upload data for walls and things.

[doitsujin]

If this is true, this is devastating news. Is there any way at all to narrow down what this might be?

I double-checked the code and did end up finding a small resource tracking issue related to framebuffer attachments (fixed in 67fe452580222fc082418ee0bc6d91dfd2f825e2), but that one would only cause gpu->cpu synchronization to fail when the game tries to map a render target. And I'm not even sure if that is possible at all in Dx11.

[jekstrand]

Yeah, it would be very bad. Unfortunately, I wasn't able to narrow it down any further today. Tomorrow, I'll throw some more instrumentation in the driver and see if I can find the busy buffer. I'll also pull the latest dxvk so I know I have your framebuffer tracking patch.

[jekstrand]

I'm also somewhat suspicious of that conclusion because Skyrim is rock-solid on Sky Lake and a busy tracking problem doesn't seem likely to only affect one platform. I'll keep digging.

[jekstrand]

Quick update: Things definitely seem harder to hang today since upgrading DXVK so 67fe452 may have helped.

[MrChebik]

Tested:

• Batman: Arkham Origins
• Dark Souls III
• Fallout 4
• Outlast 2
• Prey: Mooncrash

Did not help.

[MrChebik]

Well, this error appears only in Broadwell, and few will encounter it, so forgive for the time spent.

Many thanks to @jekstrand and @doitsujin. Do not puzzle yourself, you have many other works. I will not write any more about the malfunctions of my system.

[jekstrand]

Update: I think I finally have at least the Skyrim hang narrowed down. It's a very Broadwell-specific issue and I think I've got a fix. Haven't tested any of the others yet though.

[MrChebik]

@jekstrand I tried your anv-dxvk branch. At the Skyrim SE I got to the Whiterun, everything went smoothly, thank you!

As for other games - it seems the problem remains. Fallout 4, Prey: Mooncrash - did not work out.

@jekstrand How I can build a 32-bit version of Mesa with meson?

[jekstrand]

I got Batman: Arkham City to run by which I mean that I got it to hit an assertion in my debug build of mesa. Definitely a driver bug. :smiley:

[jekstrand]

The batman hang was caused by us not properly implementing vkCmdClearColorImage on linear RGB images. Patches to fix it are in my wip/anv-dxvk branch.

[jekstrand]

@MrChebik, so far, these all seem to be anv issues. Could you please file one bug per app at bugs.freedesktop.org? We should stop chattering about anv GPU hangs on a DXVK issue. I'll file new DXVK issues if I find an actual DXVK problem with any of them.