Ppsspp: Persona 1 crashes when loading starting screen (Linux only)

[JPTIZ]

What happens?

In Persona 1 (PSP), the game starts with a spinning card in the corner. Right after that, PPSSPP crashes with the following message:

terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid
Aborted (core dumped)

This _seems_ to be related to ingame text (which is present in the screen right after the spinning card), but I'm not sure. In Windows it works fine.

What should happen?

The game should just follows naturally without crashes, showing this screen:

What hardware, operating system, and PPSSPP version? On desktop, GPU matters for graphical issues.

- Hardware: tested in both i5-7300HQ with NVidia GeForce GTX1050 and i7-8565U with NVidia GeForce MX110.
- Operating System: Arch Linux 5.7.6
- PPSSPP version: both 1.10.2 and 1.10.3.

[unknownbrackets]

That's odd. If you can get a stack trace using gdb it'd probably help a lot.

-[Unknown]

[JPTIZ]

I'm trying to follow this troubleshooting guide first, but if it does not work I can try checking things with GDB :) Does it need running PPSSPP with any special flags? (Or compile it from scratch using a debug flag?)

[JPTIZ]

Ok, disabling things wasn't enough (even tried software rendering). Gonna check gdb.

[unknownbrackets]

Using -DCMAKE_BUILD_TYPE=Debug in the cmake line should ensure you get symbols, but may not be necessary.

-[Unknown]

[JPTIZ]

Okay, using gdb directly with pacman's ppsspp package gave me no useful debug infos:

#0  0x00007ffff63c2355 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff63ab853 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff674286a in __gnu_cxx::__verbose_terminate_handler () at /build/gcc/src/gcc/libstdc++-v3/libsupc++/vterminate.cc:95
#3  0x00007ffff674ed8a in __cxxabiv1::__terminate (handler=<optimized out>) at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:48
#4  0x00007ffff674edf7 in std::terminate () at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:58
#5  0x00007ffff674f09e in __cxxabiv1::__cxa_throw (obj=obj@entry=0x7fffc40f4650, tinfo=0x7ffff687b208 <typeinfo for std::logic_error>, dest=0x7ffff6764e10 <std::logic_error::~logic_error()>) at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:95
#6  0x00007ffff6745445 in std::__throw_logic_error (__s=0x555556046eb7 "basic_string::_M_construct null not valid") at /build/gcc/src/gcc/libstdc++-v3/src/c++11/functexcept.cc:66
#7  0x00005555559733e9 in ?? ()
#8  0x0000555555960945 in ?? ()
#9  0x000055555586b406 in ?? ()
#10 0x00005555557a999e in ?? ()
#11 0x00005555354372f4 in ?? ()
#12 0x000000000152bec0 in ?? ()
#13 0x000000000152bec0 in ?? ()
#14 0x00005555563defe0 in __bss_start ()
#15 0x000055555576fa90 in ?? ()
#16 0x0000555556374ab8 in ?? ()
#17 0x00005555558a8340 in ?? ()
#18 0x00000000454e972b in ?? ()
#19 0x000055555588af77 in ?? ()
#20 0x00005555558ad173 in ?? ()
#21 0x00005555556c9a2b in ?? ()
#22 0x0000555555aa3d37 in ?? ()
#23 0x00005555556a13d6 in ?? ()
#24 0x000055555568a253 in ?? ()
#25 0x00007ffff677bb74 in std::execute_native_thread_routine (__p=0x555556b5afc0) at /build/gcc/src/gcc/libstdc++-v3/src/c++11/thread.cc:80
#26 0x00007ffff7f8a422 in start_thread () from /usr/lib/libpthread.so.0
#27 0x00007ffff6485bf3 in clone () from /usr/lib/libc.so.6

Gonna clone and build to see the bt properly.

Also, one thing I forgot to mention: I tried with PPSSPPHeadless, *Qt and *SDL.

[hrydgard]

Super odd. Really gonna need a stack trace with symbols for this one indeed.

[gengar1603]

I doubt it is much help, but Persona 1 runs fine with the 5 year old v1.1 build in RetroPie's repository.

[JPTIZ]

Ok, I tried to get a stack trace building a debug build, but it worked (SDL version - didn't build the others). I'm trying building a Release build to see what happens.

[JPTIZ]

OOOOk, building a release build manually solves the issue. How could I give more information? Maybe it is something with the build distributed to Arch?

[hejops]

I am getting this error on Final Fantasy Tactics and Persona 3 - also Arch Linux. FFT crashes after an intro graphic, and P3 crashes when attempting to load/save data.

Settings were reverted to defaults. In both cases, sceMpegAvcDecodeFlush seems to be the cause.

Headless:
E: zip_read.cpp:312: Missing filesystem for 'compat.ini'
E: zip_read.cpp:365: Missing filesystem for flash0/font/jpn0.pgf

Qt/SDL:
I: GLRenderManager.cpp:201: Running first frame (0)
I: thin3d_gl.cpp:967: Linking shaders.
I: thin3d_gl.cpp:967: Linking shaders.
05:17:258 root         N[BOOT]: UI/EmuScreen.cpp:299 Loading Final Fantasy Tactics - The War of the Lions (USA) (PSN).iso...
05:17:422 psx_main     E[ME]: HLE/sceMpeg.cpp:1338 UNIMPL sceMpegAvcDecodeFlush(097ea4fc)
I: GLQueueRunner.cpp:496: Creating 960 x 544 FBO using DEPTH24_STENCIL8 texture
I: GLQueueRunner.cpp:496: Creating 960 x 544 FBO using DEPTH24_STENCIL8 texture
I: GLQueueRunner.cpp:426: Creating 480 x 272 FBO using no depth
I: GLQueueRunner.cpp:426: Creating 480 x 272 FBO using no depth

I: GLRenderManager.cpp:201: Running first frame (0)
I: thin3d_gl.cpp:967: Linking shaders.
I: thin3d_gl.cpp:967: Linking shaders.
08:40:980 root         N[BOOT]: UI/EmuScreen.cpp:299 Loading Shin Megami Tensei - Persona 3 Portable.iso...
08:41:981 user_main    E[SCEKERNEL]: HLE/sceKernelThread.cpp:2763 sceKernelResumeThread(311): thread not suspended
I: GLQueueRunner.cpp:496: Creating 960 x 544 FBO using DEPTH24_STENCIL8 texture
I: GLQueueRunner.cpp:426: Creating 480 x 272 FBO using no depth
I: GLQueueRunner.cpp:496: Creating 960 x 544 FBO using DEPTH24_STENCIL8 texture
I: GLQueueRunner.cpp:426: Creating 480 x 272 FBO using no depth
08:49:489 user_main    E[SCEUTIL]: HLE/sceUtility.cpp:289 80111102=sceUtilityLoadModule(00000300): already loaded
08:53:193 user_main    E[ME]: HLE/sceMpeg.cpp:1338 UNIMPL sceMpegAvcDecodeFlush(08d64254)

terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid
Aborted (core dumped)

Is there anything I can do to fix this?

[hrydgard]

It's trying to construct a string from a null value, for whatever reason. Can you get a backtrace/callstack? For example by running ppsspp in gdb:

gdb build/PPSSPPSDL     # or whatever
r
*wait for crash*
bt

The output of that would be very helpful.

[hejops]

Do I have to be in a particular working directory to run gdb build/PPSSPPSDL? I'm not familiar with gdb and I'm getting build/PPSSPPSDL: No such file or directory..

[hrydgard]

Just replace that part with wherever your PPSSPP binary is installed.

[hejops]

Thank you. Here is the output, I hope it is useful:

terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid
--Type <RET> for more, q to quit, c to continue without paging--bt

Thread 25 "SaveIO" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fff6e3bf640 (LWP 85447)]
0x00007ffff63b3615 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff63b3615 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff639c862 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff673586a in __gnu_cxx::__verbose_terminate_handler ()
    at /build/gcc/src/gcc/libstdc++-v3/libsupc++/vterminate.cc:95
#3  0x00007ffff6741d9a in __cxxabiv1::__terminate (handler=<optimized out>)
    at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:48
#4  0x00007ffff6741e07 in std::terminate ()
    at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:58
#5  0x00007ffff67420ae in __cxxabiv1::__cxa_throw (obj=obj@entry=0x7fff68000c10, 
    tinfo=0x7ffff686e208 <typeinfo for std::logic_error>, 
    dest=0x7ffff6757e20 <std::logic_error::~logic_error()>)
    at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:95
#6  0x00007ffff6738445 in std::__throw_logic_error (
    __s=0x555556046eb7 "basic_string::_M_construct null not valid")
    at /build/gcc/src/gcc/libstdc++-v3/src/c++11/functexcept.cc:66
#7  0x000055555596e160 in ?? ()
#8  0x000055555596518e in ?? ()
#9  0x0000555555964d94 in ?? ()
#10 0x00007ffff676ec24 in std::execute_native_thread_routine (__p=0x7fffd0e43390)
    at /build/gcc/src/gcc/libstdc++-v3/src/c++11/thread.cc:80
#11 0x00007ffff7f7d3e9 in start_thread () from /usr/lib/libpthread.so.0
#12 0x00007ffff6476293 in clone () from /usr/lib/libc.so.6

[hrydgard]

Huh, unfortunately less than I had hoped, only question marks on the part that would be interesting...

[unknownbrackets]

That indicates a savedata related crash - SaveIO. Maybe in SFO processing? Not sure where else could convert null to std::string...

-[Unknown]

[hrydgard]

Oh right, I missed that we had a thread name. That's indeed a clue at least...

[hrydgard]

Did a commit with a little code cleanup, but I don't think that could have fixed this...

[hejops]

Here is the backtrace for Persona 3 (crashes when attempting to load savedata, which I do not have):

Reading symbols from PPSSPPQt...
(No debugging symbols found in PPSSPPQt)
(gdb) r
Starting program: /usr/bin/PPSSPPQt 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[New Thread 0x7ffff1bfa640 (LWP 588298)]
[New Thread 0x7ffff08dc640 (LWP 588299)]
[New Thread 0x7fffeb1e8640 (LWP 588300)]
[New Thread 0x7fffea9e7640 (LWP 588301)]
[New Thread 0x7fffea1e6640 (LWP 588302)]
[New Thread 0x7fffe99e5640 (LWP 588303)]
QObject::connect: No such slot MainWindow::frameSkippinGroup_triggered(QAction *)
I: QtMain.cpp:548: Initializing GL graphics context
I: gpu_features.cpp:175: GPU Vendor : Intel Open Source Technology Center ; renderer: Mesa DRI Intel(R) HD Graphics 4000 (IVB GT2) version str: 4.2 (Core Profile) Mesa 20.1.6 ; GLSL version str: 4.20
I: QtMain.cpp:550: Using thread, starting emu thread
[New Thread 0x7fffe9128640 (LWP 588304)]
I: NativeApp.cpp:810: I: GLRenderManager.cpp:98: NativeInitGraphicsUpdating inflight frames to 2
I: Core.cpp:190: pixel_res: 200x81. Calling NativeResized()
I: NativeApp.cpp:1340: NativeResized - setting flag

I: thin3d_gl.cpp:967: Linking shaders.
I: thin3d_gl.cpp:967: Linking shaders.
[New Thread 0x7fffd3fff640 (LWP 588305)]
I: NativeApp.cpp:905: NativeInitGraphics completed
[New Thread 0x7fffd37fe640 (LWP 588306)]
I: Core.cpp:190: pixel_res: 960x544. Calling NativeResized()
I: NativeApp.cpp:1340: NativeResized - setting flag
I: NativeApp.cpp:1068: Resized flag set - recalculating bounds
I: screen.cpp:131: ScreenManager::resized(dp: 750x425)
[New Thread 0x7fffd2ffd640 (LWP 588307)]
[New Thread 0x7fffd0ffb640 (LWP 588308)]
[New Thread 0x7fffc7fff640 (LWP 588309)]
[New Thread 0x7fffc77fe640 (LWP 588310)]
loading control pad mappings from gamecontrollerdb.txt: SUCCESS!
[New Thread 0x7fffe85d3640 (LWP 588311)]
[New Thread 0x7fffc6ffd640 (LWP 588312)]
I: GLRenderManager.cpp:201: Running first frame (0)
libpng warning: iCCP: known incorrect sRGB profile
[New Thread 0x7fffc4bfc640 (LWP 588313)]
[New Thread 0x7fff87fff640 (LWP 588314)]
[New Thread 0x7fff877fe640 (LWP 588315)]
[Thread 0x7fffc4bfc640 (LWP 588313) exited]
I: thin3d_gl.cpp:967: Linking shaders.
I: thin3d_gl.cpp:967: Linking shaders.
26:27:794 root         N[G3D]: GLES/ShaderManagerGLES.cpp:899 Precompiling the shader cache from '/home/joseph/.config/ppsspp/PSP/SYSTEM/CACHE//ULUS10512.glshadercache'
26:27:795 root         N[G3D]: GLES/ShaderManagerGLES.cpp:980 Precompile: Compiled and linked 2 programs (1 vertex, 2 fragment) in 0.1 milliseconds
26:27:796 root         N[BOOT]: UI/EmuScreen.cpp:299 Loading /run/media/joseph/My Passport/files/downloads/Shin Megami Tensei - Persona 3 Portable.iso...
26:28:809 user_main    E[SCEKERNEL]: HLE/sceKernelThread.cpp:2763 sceKernelResumeThread(311): thread not suspended
I: GLQueueRunner.cpp:496: Creating 960 x 544 FBO using DEPTH24_STENCIL8 texture
I: GLQueueRunner.cpp:426: Creating 480 x 272 FBO using no depth
I: GLQueueRunner.cpp:496: Creating 960 x 544 FBO using DEPTH24_STENCIL8 texture
I: GLQueueRunner.cpp:426: Creating 480 x 272 FBO using no depth
26:36:319 user_main    E[SCEUTIL]: HLE/sceUtility.cpp:289 80111102=sceUtilityLoadModule(00000300): already loaded
[New Thread 0x7fff86bfd640 (LWP 588369)]
[New Thread 0x7fff863fc640 (LWP 588370)]
[New Thread 0x7fff85bfb640 (LWP 588371)]
[New Thread 0x7fff853fa640 (LWP 588372)]
[New Thread 0x7fff84bf9640 (LWP 588373)]
26:37:561 user_main    E[ME]: HLE/sceMpeg.cpp:1338 UNIMPL sceMpegAvcDecodeFlush(08d64254)
[Thread 0x7fff86bfd640 (LWP 588369) exited]
[Thread 0x7fff863fc640 (LWP 588370) exited]
[Thread 0x7fff853fa640 (LWP 588372) exited]
[Thread 0x7fff85bfb640 (LWP 588371) exited]
[Thread 0x7fff84bf9640 (LWP 588373) exited]
terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid
--Type <RET> for more, q to quit, c to continue without paging--

Thread 8 "Emu" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffe9128640 (LWP 588304)]
0x00007ffff63b3615 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff63b3615 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff639c862 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff673586a in __gnu_cxx::__verbose_terminate_handler ()
    at /build/gcc/src/gcc/libstdc++-v3/libsupc++/vterminate.cc:95
#3  0x00007ffff6741d9a in __cxxabiv1::__terminate (handler=<optimized out>)
    at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:48
#4  0x00007ffff6741e07 in std::terminate ()
    at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:58
#5  0x00007ffff67420ae in __cxxabiv1::__cxa_throw (obj=obj@entry=0x7fffcc95df20, 
    tinfo=0x7ffff686e208 <typeinfo for std::logic_error>, 
    dest=0x7ffff6757e20 <std::logic_error::~logic_error()>)
    at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:95
#6  0x00007ffff6738445 in std::__throw_logic_error (
    __s=0x555556046eb7 "basic_string::_M_construct null not valid")
    at /build/gcc/src/gcc/libstdc++-v3/src/c++11/functexcept.cc:66
#7  0x00005555559733e9 in ?? ()
#8  0x0000555555960945 in ?? ()
#9  0x000055555586b406 in ?? ()
#10 0x00005555557a999e in ?? ()
#11 0x000055553551a3d4 in ?? ()
#12 0x000000000152bec0 in ?? ()
#13 0x000000000152bec0 in ?? ()
#14 0x00005555563defe0 in __bss_start ()
#15 0x000055555576fa90 in ?? ()
#16 0x0000555556374ab8 in ?? ()
#17 0x00005555558a8340 in ?? ()
#18 0x00000000bfd3d905 in ?? ()
#19 0x000055555588af77 in ?? ()
#20 0x00005555558ad173 in ?? ()
#21 0x00005555556c9a2b in ?? ()
#22 0x0000555555aa3d37 in ?? ()
#23 0x00005555556a13d6 in ?? ()
#24 0x000055555568a253 in ?? ()
#25 0x00007ffff676ec24 in std::execute_native_thread_routine (__p=0x555556b559a0)
    at /build/gcc/src/gcc/libstdc++-v3/src/c++11/thread.cc:80
#26 0x00007ffff7f7d3e9 in start_thread () from /usr/lib/libpthread.so.0
#27 0x00007ffff6476293 in clone () from /usr/lib/libc.so.6

Is it a similar error?

[hrydgard]

Looks similar indeed. Can you build a debug build of PPSSPP and try with that? (./b.sh --debug)

[hejops]

I was unable to build:

Edit: Never mind, fixed it with git submodule update --init. Thanks Unknown.

[unknownbrackets]

Ah just need a git submodule update --init.

-[Unknown]

[hejops]

Got it. Everything seems to work fine with the debug build. So I assume there is something wrong with the pacman build?

[unknownbrackets]

I don't know Arch that well, but I think this is the build script?

https://github.com/archlinux/svntogit-community/blob/ae5f8ded2fdba7a6e3f446b68899cb9b6bb94e78/trunk/PKGBUILD

You might try those options specifically:
https://github.com/archlinux/svntogit-community/blob/ae5f8ded2fdba7a6e3f446b68899cb9b6bb94e78/trunk/PKGBUILD#L107-L117

It's also possible this is a bug already fixed if this is happening only in v1.10.3.

-[Unknown]

[scurest]

Hi. This is a really strange issue. I can reproduce on master only when using clang. Specifically, it depends on the build command:

|Command|Has Issue?|
|:---|:---|
|./b.sh --clang --debug|No|
|./b.sh --clang|Yes|
|CXXFLAGS="-O2" ./b.sh --clang --debug|Yes|
|CXXFLAGS="-O2" ./b.sh --clang|Yes|

My clang version is 10.0.1. It doesn't happen if I use g++, so anyone hitting this should try that.

gdb's backtrace implicates this line

https://github.com/hrydgard/ppsspp/blob/a3c17ee63b1c2f399d86e976cb1359b3ca48286a/Core/Dialog/SavedataParam.cpp#L1037

The throw occurs in basic_string.tcc here

````cpp
void
basic_string<_CharT, _Traits, _Alloc>::
_M_construct(_InIterator __beg, _InIterator __end,
std::forward_iterator_tag)
{
// NB: Not required, but considered best practice.
if (__gnu_cxx::__is_null_pointer(__beg) && __beg != __end)
std::__throw_logic_error(__N("basic_string::"
"_M_construct null not valid"));

...

````

However this is really weird, because either gdb or a printf reports that __beg is definitely _not_ null here!

Here is a fragment of disassembly from around that line.

asm 0x0000555555964df0 <+448>: lea 0xa67d11(%rip),%r13 # 0x5555563ccb08 <_ZN6Memory4baseE> 0x0000555555964df7 <+455>: mov 0x0(%r13),%rbp ; rbp now holds Memory::base 0x0000555555964dfb <+459>: add %rbp,%r14 0x0000555555964dfe <+462>: mov $0xd,%esi ; second arg to strnlen: sizeof(param->msData->gameName) 0x0000555555964e03 <+467>: mov %r14,%rdi ; first arg to srtnlen: param->msData->gameName 0x0000555555964e06 <+470>: callq 0x555555623f50 <strnlen@plt> 0x0000555555964e0b <+475>: mov %rax,%rbx ; return value of strnlen 0x0000555555964e0e <+478>: lea 0x1d0(%rsp),%r12 0x0000555555964e16 <+486>: mov %r12,0x1c0(%rsp) 0x0000555555964e1e <+494>: test %rbp,%rbp ; presumably this is __gnu_cxx::__is_null_pointer(__beg)... 0x0000555555964e21 <+497>: jne 0x555555964e2c <SavedataParam::GetSizes(SceUtilitySavedataParam*)+508> 0x0000555555964e23 <+499>: test %rbx,%rbx ; ...and this is __beg != __end 0x0000555555964e26 <+502>: jne 0x5555559657c1 <SavedataParam::GetSizes(SceUtilitySavedataParam*)+2961> ; goes to the throw 0x0000555555964e2c <+508>: mov %rbx,0x70(%rsp) ; string is allocated past here

The instruction at <+494> is testing if %rbp (Memory::base) is zero (it is). It seems like it should be testing if %r14 (param->msData->gameName) is.

The issue also disappears under trivial rephrasing (like std::string gameName; gameName.assign(...);).

I wonder if this could be a miscompilation by clang?

[unknownbrackets]

Seems like it. param->msData->gameName is actually using an operator overload, so it's not as crazy for it to check Memory::base as it might seem.

param->msData is a PSPPointer which is essentially (via a template):

    // Pointer into PSP memory.
    uint32_t ptr;

    inline SceUtilitySavedataMsDataInfo *operator->() const
    {
        return (SceUtilitySavedataMsDataInfo *)(Memory::base + ptr);
    }

Presumably add %rbp,%r14 is adding ptr as in Memory::base + ptr. param->msData.IsValid() is already doing a nullcheck of ptr though, and Memory::base should never be nullptr. It's definitely bad if rbp is zero at that point, although the low 32 bits of it may often be zero.

I'm more used to Intel syntax - add %rbp,%r14 modifies r14 right? In which case, the test at 494 incorrectly nullchecks Memory::base rather than Memory::base + ptr? Does it even try to read the string from there?

If yes, a workaround for the bug might even be:

-       const std::string gameName(param->msData->gameName, strnlen(param->msData->gameName, sizeof(param->msData->gameName)));
-       const std::string saveName(param->msData->saveName, strnlen(param->msData->saveName, sizeof(param->msData->saveName)));
+       const SceUtilitySavedataMsDataInfo *msData = param->msData;
+       const std::string gameName(msData->gameName, strnlen(msData->gameName, sizeof(msData->gameName)));
+       const std::string saveName(msData->saveName, strnlen(msData->saveName, sizeof(msData->saveName)));

Since that would force it to dereference the operator once for all of them... which I'd wish it was smart enough to do on its own.

-[Unknown]

[scurest]

Memory::base is always zero for me, even when building with g++, even in the middle of a game (that did seem odd to me but didn't seem to be a problem).

I'm more used to Intel syntax - add %rbp,%r14 modifies r14 right? In which case, the test at 494 incorrectly nullchecks Memory::base rather than Memory::base + ptr?

Yeah, that's what it looks like. If I force it past that one test, everything else works fine. There wasn't any change from that patch either.

[hrydgard]

I checked in unknown's workaround idea.

[unknownbrackets]

It should log the base on startup, is it really zero there too?

    INFO_LOG(MEMMAP, "Memory system initialized. Base at %p (RAM at @ %p, uncached @ %p)",
        base, m_pPhysicalRAM, m_pUncachedRAM);

If mmap() returns nullptr, we should force 0x2300000000ULL. See:
https://github.com/hrydgard/ppsspp/blob/89d563717e067bbede6622e8349b596db828c3fa/Common/MemArenaPosix.cpp#L112-L119

It really shouldn't be nullptr...

-[Unknown]

[hrydgard]

If base is nullptr, nothing will work so that has to be something wrong with how you're printing the value.

[scurest]

Reduced test case

Source code

````cpp
// a.cxx

include

include

include

include

include

uint8_t* base = nullptr;

template
struct Ptr {
uint64_t ptr;
inline T* operator->() const { return (T*)(base + ptr); }
};

struct Data {
char name[13];
};

void foo(Ptr p) {
const std::string name(p->name, strnlen(p->name, sizeof(p->name)));
printf("%s\n", name.c_str());
}

int main() {
uint8_t* blob = new uint8_t[4096];

// Fill with random data
srand(time(nullptr));
unsigned acc = rand();
for (unsigned i = 0; i != 4096; ++i)
    blob[i] = (acc ^= i * 69069u);

foo(Ptr<Data> { (uint64_t)blob });

}
````

Produces the same exception when compiled with clang++ -O2 a.cxx. Using fsanitize=undefined produces the error applying non-zero offset 94671164550880 to null pointer.

[scurest]

I'm printing it by stopping with gdb and running p Memory::base.

[hrydgard]

Wacky stuff. I think the compiler argues here that offsetting from a known null pointer is undefined, but in PPSSPP's case, it's definitely not null, but somehow the optimizer has gotten that idea...

If you declare base as volatile, the compiler might do better (but we don't want that, could slow things down).

[scurest]

    INFO_LOG(MEMMAP, "base: %p", base); 
    uint64_t aligned_base = ((uint64_t)base + 0xFFFFFFFF) & ~0xFFFFFFFFULL; 

Ah, I get

and therefore aligned base comes out to 0.

[unknownbrackets]

Just to explain what's happening with Memory::base:

The PSP-1000 has 32 MB of RAM (later models have more.) It sets this up in a very specific address space within 32 bit (really, within 25 bits for 32 MB...) 0x08000000 is the start of the 32 MB RAM, and 0x09FFFFFF is the end.

Since we're running game code, we have two major options:

1. Let the game code keep using these addresses, and find a way to make them still valid.
2. Do math or a lookup table on every memory address, to translate it to C++ / Windows / Linux memory space

Option 2 is slow, although many other emulators take that path. We take option 1 for speed.

To do this, we ask the OS to allocate 4 gigabytes of address space (not RAM, since we're not asking it to commit any of that.) This 4 gigabytes encompasses the entire 32 bits of address space that PSP CPU could access. Then we tell the OS to map memory to specific areas of that address space. Basically, rather than doing the math and lookup tables... we tell the OS to do it for us. The OS already has these features. Why do it twice?

So that's what Memory::base is. It's the result of the 4 GB allocation. If that fails, the whole thing is toast. That's why we just try to use 0x2300000000 in the worst case. If we get nullptr, it means somehow memory setup has failed and the PSP memory emulation won't work. No games should work. Everything should crash immediately. No PSP software is capable of doing anything remotely useful without access to RAM.

The PSPPointer template takes a PSP pointer (that is, a 32-bit offset into the 4 GB "array" of PSP memory address space) and converts it to a C++ pointer (that is, a real pointer to the specific part of that 4GB "array" we're talking about.)

It's not adding two pointers. A more direct comparison to what PPSSPP's doing would be:

int main() {
    base = new uint8_t[4096];
    memset(base, 0, 4096);

    strcpy(base + 1024, "hello");

    foo(Ptr<Data> { 1024 });
}

-[Unknown]

[unknownbrackets]

Okay, 0xffffffffffffffff is MAP_FAILED. I wonder why that's failing, maybe you have overcommit_memory off?

2b49c3e46 should check for MAP_FAILED and go the slightly dangerous 0x2300000000 path...

-[Unknown]

[unknownbrackets]

We might consider adding MAP_NORESERVE, assuming it's 0... maybe the heuristic is not allowing us.

-[Unknown]

[scurest]

I can confirm building with ./b.sh --clang now works!

errno for me was just ENOMEM.

[scurest]

We might consider adding MAP_NORESERVE, assuming it's 0... maybe the heuristic is not allowing us.

overcommit_memory is indeed 0. FWIW, adding MAP_NORESERVE makes the mmap succeed for me.

I guess you could also try a second time with NORESERVE if the first mmap without it fails.

[hrydgard]

Hm, maybe we should just always use MAP_NORESERVE ...

[scurest]

I think that's probably a good idea. IIUC at least 4G of that 8G is never going to actually be committed.

[hrydgard]

Did the MAP_NORESERVE change. I think we can consider this closed.

[unknownbrackets]

I think that's probably a good idea. IIUC at least 4G of that 8G is never going to actually be committed.

Much less - fewer than 128 MB of it will be committed. There are some mirrors that overlap, but won't separately commit.

-[Unknown]

[scurest]

Thanks for all the help guys!