Ppsspp: (Libretro) Crashes with vulkan when started

[orbea]

What happens?

When running the libretro core in RetroArch with vulkan it segfaults immediately when starting.

Thread 1 "retroarch" received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007fffeb5eb5b3 in VulkanContext::CreateInstance (this=0x1eaf7c0, 
    info=...)
    at /home/orbea/gittings/forks/ppsspp/Common/Vulkan/VulkanContext.cpp:245
#2  0x00007fffeb21f0f7 in create_device (context=0x7fffffffcf80, 
    instance=0x1c4ffa0, gpu=0x0, surface=0x1, 
    get_instance_proc_addr=0x7fffe836b6b0 <vkGetInstanceProcAddr>, 
    required_device_extensions=0xf04230 <device_extensions>, 
    num_required_device_extensions=1, 
    required_device_layers=0xf04238 <device_layers>, 
    num_required_device_layers=1, required_features=0x7fffffffcfb0)
    at /home/orbea/gittings/forks/ppsspp/libretro/LibretroVulkanContext.cpp:41
#3  0x0000000000749c15 in vulkan_context_init_device (vk=0x1794c00)
    at gfx/common/vulkan_common.c:1683
#4  0x000000000074b1c1 in vulkan_surface_create (vk=0x1794c00, 
    type=VULKAN_WSI_XCB, display=0x1795580, surface=0xfbffd8 <g_x11_win>, 
    width=748, height=211, swap_interval=0) at gfx/common/vulkan_common.c:2483
#5  0x00000000007185f0 in gfx_ctx_x_set_video_mode (data=0x1794bb0, 
    video_info=0x7fffffffd570, width=1680, height=1050, fullscreen=true)
    at gfx/drivers_context/x_ctx.c:906
#6  0x00000000004a932f in video_context_driver_set_video_mode (
    mode_info=0x7fffffffda64) at gfx/video_driver.c:3264
#7  0x0000000000740f0e in vulkan_init (video=0x7fffffffdac0, 
    input=0xf5bf90 <current_input>, input_data=0xf5bf98 <current_input_data>)
    at gfx/drivers/vulkan.c:1206
#8  0x00000000004a50df in video_driver_init_internal (
    video_is_threaded=0x7fffffffdb87) at gfx/video_driver.c:1065
#9  0x00000000004a64d7 in video_driver_init (video_is_threaded=0x7fffffffdb87)
    at gfx/video_driver.c:1761
#10 0x00000000004abaa9 in drivers_init (flags=1023) at driver.c:361
#11 0x000000000043e1fc in retroarch_main_init (argc=4, argv=0x7fffffffe188)
    at retroarch.c:1429
#12 0x000000000045b1ea in content_load (info=0x7fffffffe050)
    at tasks/task_content.c:291
#13 0x000000000045c49f in task_load_content (content_info=0x7fffffffe050, 
    content_ctx=0x7fffffffdf60, launched_from_menu=true, launched_from_cli=true, 
    error_string=0x7fffffffdf58) at tasks/task_content.c:982
#14 0x000000000045d9df in task_load_content_callback (
    content_info=0x7fffffffe050, loading_from_menu=true, loading_from_cli=true)
    at tasks/task_content.c:1672
#15 0x000000000045dbf0 in task_push_load_content_from_cli (core_path=0x0, 
    fullpath=0x0, content_info=0x7fffffffe050, type=CORE_TYPE_PLAIN, cb=0x0, 
    user_data=0x0) at tasks/task_content.c:1753
#16 0x0000000000439e7b in rarch_main (argc=4, argv=0x7fffffffe188, data=0x0)
    at frontend/frontend.c:140
#17 0x0000000000587822 in main (argc=4, argv=0x7fffffffe188)
    at ui/drivers/qt/ui_qt_application.cpp:188

Full GDB log - https://paste.ee/p/mQdAx

I bisected the crash and found the first bad commit.

8e1a5ef3d6f2aef0367257245da3b6aa35ce1d14 is the first bad commit
commit 8e1a5ef3d6f2aef0367257245da3b6aa35ce1d14
Author: Henrik Rydgard <[email protected]>
Date:   Tue Feb 5 10:05:22 2019 +0100

    Minor refactor of physical device property/feature detection, to allow for more extension use.

:040000 040000 376dc456235a3b1338c09d6b4a944c86cef5c27e 9b5ecbe86c0bf643682fd18e01f57fabd88142a9 M  Common
:040000 040000 36b61ef106dd0b869122d49f1a3d9dee6e2d5e5f 699dd62624b1e0915c6906aafd1e0dfccb65c46c M  GPU
:040000 040000 9caaf9119c9e2bdbb99dddf3a6fb2a84ba3ed17e f350808fa446047e743dc47df9a7a242e426ff96 M  Windows
:040000 040000 a480d68f4f12d337e304aaadd80e1f180ed67a07 2b7b57ee610b60c65bb6cf5d8d6aee705acc564d M  ext

8e1a5ef3d6f2aef0367257245da3b6aa35ce1d14

This uncovered a second related issue where before it started to crash, the rendered window displayed only in the top left corner of RetroArch as a very tiny window.

I bisected again, but it was unsuccessful due to unrelated segfaults and build errors before the last good commit and the first known bad commit.

There are only 'skip'ped commits left to test.
The first bad commit could be any of:
68f391707f47d2d3a0f76d0b91cb2b105b54dafa
6f1996413515f6afcf05d59ce1c6a5e0d796b141
0758925fe40f42f823b4ed29446bdd8b2fa11c61
7ccf23542de31aa2b2b6e1c09fc595ed44afb0fd
33803b08f513cdfb71f5b9b71355dcd428a4f638
3fd216ad3abcdc223c5f4931f083f413df188c21
ca6298e24de6e93f6a8972d1a34c5992e33af2ba
We cannot bisect more!

68f391707f47d2d3a0f76d0b91cb2b105b54dafa
6f1996413515f6afcf05d59ce1c6a5e0d796b141
0758925fe40f42f823b4ed29446bdd8b2fa11c61
7ccf23542de31aa2b2b6e1c09fc595ed44afb0fd
33803b08f513cdfb71f5b9b71355dcd428a4f638
3fd216ad3abcdc223c5f4931f083f413df188c21
ca6298e24de6e93f6a8972d1a34c5992e33af2ba

I was not able to reproduce these issues with the SDL2 standalone frontend.

What should happen?

PPSSPP should not crash.

What hardware, operating system, and PPSSPP version? On desktop, GPU matters for graphical issues.

OS: Slackware64-current
GPU: Radeon RX Vega 56
mesa: https://github.com/mesa3d/mesa/commit/0aa5a97b03159f59513bd96dbf7c25c2d22e3c1c
RetroArch: https://github.com/libretro/RetroArch/commit/3cbae9c767af8c29d3778bc32ed7a7cb8f0c3dc4
linux: 4.20.12

[orbea]

@hrydgard Maybe you can help since all of the related commits are yours?

[fr500]

No crash here on the latest build (6f2a6e2)
But the quarter screen issue does happen

[fr500]

Seems fine in D3D11 and GL

[orbea]

Thanks for testing, GL is also fine here excluding preexisting bugs reported in other issues.

[orbea]

@hrydgard I would really appreciate it if you could look at these issues. The only way I could use ppsspp without at least one annoying or major issue was with libretro + vulkan...

[orbea]

@hrydgard This issue still occurs, can you please fix it?

[G-Ray]

Are you using wayland or xorg ?

[orbea]

@G-Ray I'm using xorg.

[ferrazpedro]

I'm having the exact same crash and I'm using Windows 10.

[julianxhokaxhiu]

@hrydgard can you please let us know if this is something you can pick up or should we ask the Libretro community?

[orbea]

Still broken in commit https://github.com/hrydgard/ppsspp/commit/1618aa8f8c56e072fba56f98bec0d74de7fbb24c.

@hrydgard The fact that you not only broke this, but that you can't even reply after half a year is really unsavory.

[hrydgard]

I don't see how this is my responsibility, you guys wrote the integration, keeping it up to date is up to you. I do plan to look at it eventually but it's just lower priority than the standalone releases for me.

[orbea]

@hrydgard You are the one that wrote the code that broke this feature. I bisected it to commit https://github.com/hrydgard/ppsspp/commit/8e1a5ef3d6f2aef0367257245da3b6aa35ce1d14.

[hrydgard]

Well I don't see how that commit could possibly cause a crash. But I will look into it.

[orbea]

I just know that I bisected it to that commit and that I can still confirm its the bad commit manually. The crash occurs in https://github.com/hrydgard/ppsspp/commit/8e1a5ef3d6f2aef0367257245da3b6aa35ce1d14, but not in the parent commit https://github.com/hrydgard/ppsspp/commit/fdd0d7acb479e220d19616f8b8deaee4e27f43f0.

However please fully read the OP, there is a second issue as shown in the images which is also caused by one of your commits, but the bisect was not able to narrow it down to a single commit. It was one of the following.

68f3917
6f19964
0758925
7ccf235
33803b0
3fd216a
ca6298e

[hrydgard]

@orbea I'm trying to reproduce. I don't normally use retroarch, so I tried installing it, but I can't figure out how to get it to use my newly built ppsspp_libretro.so.

What's the most convenient way for development to run a specific core? Can I pass it on the commandline or something?

Never mind, found https://docs.libretro.com/guides/cli-intro/

Never mind again, that doesn't seem to work at all:

henrik@henrik-laptop-linux:~/dev/ppsspp$ retroarch -L ~/dev/ppsspp/ppsspp_libretro.so ~/Downloads/cube.elf
henrik@henrik-laptop-linux:~/dev/ppsspp$ 

It just exits instantly. @orbea any idea why? with --verbose:

ERROR] Frontend is built for dynamic libretro cores, but path is not set. Cannot continue.
[ERROR] Fatal error received in: "init_libretro_symbols()"

Additional nevermind: Got it to run by actually specifying the correct path. A more descriptive error message would have helped.

[hrydgard]

I have repro.

[hrydgard]

@orbea Should PPSSPP even do its own Vulkan intiialization?

Is there something like https://docs.libretro.com/development/cores/opengl-cores/ that is about writing Vulkan cores for Retroarch?

[orbea]

Should PPSSPP even do its own Vulkan intiialization?

Sorry I don't know, I didn't do anything regarding vulkan in libretro or RetroArch and its exactly because I don't know enough about vulkan and C++ that I didn't fix this myself. Most of the work I did for RetroArch is in their configure script and general debugging.

[hrydgard]

Well, who might know?

[orbea]

@Themaister Knows most about vulkan in libretro, but I don't have any way direct way to contact him.

@twinaphex and @m4xw Might also know more than me.

[hrydgard]

So there's already LibretroVulkanContext.cpp/h in libretro/ which tries to smooth things out between libretro and PPSSPP. Unfortunately it looks pretty broken, and needs some rework.

[hrydgard]

Yeah, it's pretty much completely wrong, I'm mostly surprised anything worked.

I'll try to fix it up during the next week.

[orbea]

Thank you, I really appreciate it.

[hrydgard]

Turns out it's not quite as broken as I first thought, but the approach is scary - it replaces some Vulkan functions with retroarch-specific implementations, that forward things to the libretro Vulkan API. This approach apparently doesn't work with some of the extensions we started trying to use.

I apparently just merged that code without reading it when it was added, heh.

[Themaister]

FWIW, it's possible to create your own VkDevice and VkQueues and hand that off to libretro, that way you can enable any device extensions/features you want.

One missing thing I might get around to if it's needed is to support letting the core control instance creation as well. That's necessary for allowing instance extensions which a core might need, frontend just enables VK_KHR_surface and friends. In practice, just having apiVersion of 1.1 in the application info callback should be enough to make sure physical_device_features2 can be used at least without changing the API ...

[seriouslyunderappreciated]

Is this still being looked at? I love PPSSPP but I've been transitioning everything to Retroarch and I would like to be able to use this core without this crashing.

Honestly at this point if anyone could just point me to the last working version of this libretro core that doesnt crash with vulkan, that would be great.

[hrydgard]

Sorry I'll get to this soon...

[escalade]

@hrydgard

Any update on this? It's been broken for nearly a year now. Wayland is broken as well.

[hrydgard]

This was initially gotten to work by somebody using an incredibly hacky method that hooked into Vulkan underneath us, but it didn't support some stuff we've started to use since. That makes it a rather daunting task, especially not being familiar with RetroArch development...

I will give it a shot at some point though.

[julianxhokaxhiu]

Would you be able to point at least where in the source code that is happening? This may leverage contributors ( even myself ) in helping in case. Thank you in advance.

[hrydgard]

Go have a look at the stuff in libretro/libretro_vulkan.cpp.

[orbea]

Should be fixed in https://github.com/hrydgard/ppsspp/pull/13328.

Any remaining issues should be tracked in new issues.