Ppsspp: Nintendo switch port of ppsspp (request).

[iOS4all]

@hrydgard hi there.

Many people are wondering and excited about getting a Nintendo switch port of ppsspp (including me).
As you see Nintendo switch is already hacked specially on v3.0.0 we can run hombrews and emulators including retroarch on it. And also @plutoo is bringing JIT on LibNX (LibNX is the place to run hombrews). Also he and others developers are working to bring gpu drivers to libnx as well.

Finally @SciresM will bring CFW for Nintendo switch this summer.

It's very nice to get ppsspp for all platforms.

Thanks for your hard work.

[hrydgard]

Once there's a good homebrew environment for a publicly available CFW, and drivers for the GPU, a port should be fairly straightforward so I'm sure someone's gonna try it eventually. It's a bit too early still though.

[thedax]

This should be more feasible in the near future, as SDL2 has been ported, and a port of Mesa (and Nouveau) is being worked on for Horizon (the Switch's OS by Nintendo), so there will also be a standard OpenGL environment as well.

Keep an eye on https://github.com/devkitPro/libdrm_nouveau and https://github.com/devkitPro/mesa .

[iOS4all]

that’s fantastic news. Actually I saw someone on Twitter called Fincs he showed us an app running on switch using OpenGL.(so OpenGL is finally implemented to switch). So I hope we can get ppsspp on horizon OS eventually.

[thedax]

Yeah, fincs is working on it. It's still very early though, so I think it's best to wait and see before jumping right in. As for the CFW situation, the Switch has been pretty hackable for the last 3 months or so via a vunerability in the hardware's recovery mode, so full control is a non-issue on most Switch consoles (there are something like 15 or more million units vulnerable, so they're not hard to find currently). Debugging still leaves a bit to be desired, though.

Edit: https://twitter.com/fincsdev/status/1037457485983096832 I'm excited. Hopefully it won't be too buggy!

[iOS4all]

Here’s a new news about OpenGL on switch see here:

This is a great thing indeed.

To see his tweet see here
https://twitter.com/fincsdev/status/1037457485983096832?s=21

So I hope we can get a port of ppsspp on horizon OS eventually.

[thedax]

I'm keeping an eye on it. If the SDL2 port and this implementation of OpenGL work nicely together, that would mainly leave the recompiler (JIT). Horizon does provide JIT services/SVCs if I remember correctly, so it should just be a matter of talking to that to get the ball rolling. Hopefully.

[iOS4all]

That’s something pretty nice to read.
About JIT I just want to correct you. Yes @plutoo brings JIT on LibNX officially while ago.

See here:

Also here is an official GitHub link with JIT implemented.
https://github.com/switchbrew/libnx/blob/master/nx/include/switch/kernel/jit.h

[thedax]

What's there to correct? I said it does provide it if I remember correctly, and I did. :P

But yeah, I'll be waiting a bit longer, at least until the first public beta release.

[iOS4all]

Oh I’m sorry.
And all I can say good luck. Actually the Nintendo Switch really deserves to have ppsspp on it as a horizon OS especially after getting OpenGL. How nice.

[thedax]

It's fine, haha.

[hrydgard]

That means that all the big puzzle pieces are in place, at least. I'm personally not going to do Switch development until it gets a bit more stable and usable (is there already something like psplink? ) but there's nothing stopping anyone else from porting it. GL 4.3 should be alright.

[thedax]

You can get crash dumps via CFW and there are a couple WIP debuggers, but no gdb or stepping through code yet (to my knowledge).

[unknownbrackets]

Well, the nice thing about psplink is you can quickly run a program over USB on the PSP from the desktop. It allows automating running tests or programs and getting the result. That's how all pspautotests were built.

IIRC, Wii has something similar, but it's less reliable. PS2 too (and similarly less reliable.) Hence, ps2autotests.

-[Unknown]

[thedax]

Oh, then yeah, there's a program called nxlink you can use to send over NROs (homebrew executables) to the homebrew menu on the Switch via the network (USB isn't entirely understood on the Switch yet, since the API seemingly changes somewhat often..). I'm not sure if nxlink supports redirecting stdout though. I'll have a look eventually.

[iOS4all]

LibNX is now on version 1.4.0
With a huge improvements including OpenGL support. finally.

For more info see here.
https://github.com/switchbrew/libnx/commit/23fa46a8ee6610c226ce7b555b204ca236e48623

I hope this could help about getting ppsspp on switch.

[thedax]

There are still several issues to work out before PPSSPP will build (I looked into it a bit earlier), but yeah, their OpenGL stuff is progressing quickly.

[thedax]

Alright, so I took a brief look at it last night, and here's what I came up with so far with regards to porting PPSSPP:

• [ ] There's no support for std::mutex or std::thread, so threading would have to be reworked or somehow implemented with Switch-specific stuff, maybe a wrapper around libnx that implements mutex and thread (is that even feasible? hm.)
• [ ] SDL2 port doesn't yet support OpenGL (SDL_GetError returns no GL driver support despite initializing via EGL port)
• [ ] Aarch64 JIT should work fine once hooked up to proper Switch-specific memory protection stuff
• [x] Build ppsspp-ffmpeg for Switch

[hrydgard]

I think std::mutex and std::thread could quite easily be implemented on top of Switch OS primitives, although that comes from looking at the 3DS OS, I haven't looked much at the Switch. That would be best done as part of libnx though.

GL support in the SDL2 port is probably just a matter of time, or a small patch...

[iOS4all]

@thedax there’s already a bounty for getting ARM64 JIT to work on Nintendo Switch
@natinusala said this can be works on emulators such as n64. Pcsx and more I guess.

Here’s a link to that bounty:
https://www.bountysource.com/issues/63766562-bounty-write-an-arm64-dynarec

I just want to to know about ARM64 JIT because you just mentioned it on your plans list.

[natinusala]

libnx has functions to create "JIT" buffers, e.g. buffers where the memory is executable. I looked at the PPSSPP code, and I think that we'll only need to hook it to AllocateExecutableMemory.

[thedax]

I saw the comments on that bountysource link, but for the sake of convenience: PPSSPP already has an ARM64 JIT like natinusala said on there. Getting the JIT to work is the least of our problems here. :P

[iOS4all]

That’s nice to hear. I didn’t know about that indeed. So what is missing to get it to work?

Thanks a lot for your efforts.

[thedax]

Still waiting on SDL2 to be updated. Threading is another issue, but SDL2 is more important.

[natinusala]

SDL can easily be updated if you replace the GL includes by the glad includes and init the context properly. You can use mGBA or RetroArch as a reference for that, the switch-examples are borked.

[m4xw]

@thedax
Make sure to recompile libdrm_nouveau since it doesnt contain the fixes via pacman yet, IIRC.
It should just work if set-up correctly.
There are still some bugs, like u can't re-load glad if u used eglTerminate for example.
I did the retroarch port via libnx, if you have questions feel free to ask me.

[hrydgard]

@iOS4all Be patient, don't nag. If thedax wants to work on it, he will.

[iOS4all]

@hrydgard ok I’ll.

[dmiller423]

• figured I'd add some insight:

SDL2 has been updated and you can use GL context it creates.

The wrapper is partially feasible, here is a very partial impl. for pthreads : https://github.com/m4xw/RetroArch/blob/fd155db9cca5696495440a35cc7fb9fc9b8831ff/libretro-common/rthreads/switch_pthread.h

unforunately it's nowhere near enough for the __gthread wrapper (libgcc std::thread wraps) and gthr-single.h just returns 0 or -1 to everything.

There are also various other issues, mostly revolving around switch using a custom micro kernel that is not remotely posix compliant. IE: the only current method to map pages RW -> RX is shown in the jit examples, and though it seems to be done in a single process now, it gives back two address spaces: one RW and one RX.

I haven't looked into the details on Icache/block manager yet, it might not be difficult to make the changes necessary to ensure the recompiler calls the 'second' (RX) address space.

Likely you will run into many more, the project is rather large and has multiple deps.

Here is a cmake file I used to test a bit, since it's really a cross compile and I did not want to modify the base cmake file you need to configure like: cmake .. -DCMAKE_TOOLCHAIN_FILE=./devkitA64.cmake

Below is the devkitA64.cmake

## devkitA64.cmake - devkitpro A64 cross-compile
#
set(CMAKE_SYSTEM_NAME Linux) # TODO Add CMake/Modules/Platform/switch 
set(CMAKE_SYSTEM_PROCESSOR aarch64)

set(CMAKE_SYSTEM_VERSION 1)   # this one not so much


set(DEVKITPRO $ENV{DEVKITPRO})
set(DEVKITA64 $ENV{DEVKITA64}) 


if ("" STREQUAL "${DEVKITPRO}")
  set(DEVKITPRO "/opt/devkitpro")
endif()

if ("" STREQUAL "${DEVKITA64}")
  set(DEVKITA64 ${DEVKITPRO}/devkitA64)
endif()


## specify the cross compiler
#
set(CMAKE_C_COMPILER   ${DEVKITA64}/bin/aarch64-none-elf-gcc)
set(CMAKE_CXX_COMPILER ${DEVKITA64}/bin/aarch64-none-elf-g++)


set(CMAKE_FIND_ROOT_PATH  ${DEVKITA64}) # where is the target environment

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) # search for programs in the build host directories
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)  # for libraries and headers in the target directories
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)




set(DKP_LIBNX        ${DEVKITPRO}/libnx)
set(DKP_LIBNX_INC    ${DKP_LIBNX}/include)

set(DKP_PORTLIBS     ${DEVKITPRO}/portlibs)
set(DKP_PORTLIBS_LIB ${DKP_PORTLIBS}/switch/lib)
set(DKP_PORTLIBS_INC ${DKP_PORTLIBS}/switch/include)

include_directories(${DKP_LIBNX_INC} ${DKP_LIBNX_INC}/switch/kernel ${DKP_PORTLIBS_INC})


#set(LIBZIP_INCLUDE_DIR)
#set(LIBZIP_LIBRARY 
#
set(ZLIB_INCLUDE_DIR    ${DKP_PORTLIBS_INC})      #${DEVKITPRO}/portlibs/switch/include)
set(PNG_PNG_INCLUDE_DIR ${DKP_PORTLIBS_INC})      #${DEVKITPRO}/portlibs/switch/include)
set(SDL2_INCLUDE_DIR    ${DKP_PORTLIBS_INC}/SDL2) #${DEVKITPRO}/portlibs/switch/include/SDL2)

set(ZLIB_LIBRARY        ${DKP_PORTLIBS_LIB}/libz.a)    #${DEVKITPRO}/portlibs/switch/lib/libz.a)
set(PNG_LIBRARY         ${DKP_PORTLIBS_LIB}/libpng.a)  #${DEVKITPRO}/portlibs/switch/lib/libpng.a)
set(SDL2_LIBRARY        ${DKP_PORTLIBS_LIB}/libSDL2.a) #${DEVKITPRO}/portlibs/switch/lib/libSDL2.a)



## gthr-single.h shouldn't be a valid options at least ?
#add_definitions(-D_GLIBCXX_HAS_GTHREADS -D__GTHREADS=1) ## *FIXME* build in wrapper, trick std::thread 
add_definitions(-D_GLIBCXX_USE_C99_MATH_TR1 -D_LDBL_EQ_DBL)
add_definitions(-DNSW -D__SWITCH__)


set(NSW ON)
set(ARM_NO_VULKAN ON)
#set(USING_EGL ON)
#set(USING_GLES2 ON)

[iOS4all]

Happy new year for all 🎊🎈

[DuIslingr]

Any updated progress on this?

[hrydgard]

I don't think anyone is working on it at this time. But as the Switch homebrew platform gets more and more complete, this will get easier and easier so I think there's a good chance it'll happen eventually.

[natinusala]

The last requirement left is std::thread :)

[natinusala]

Threads need to be implemented into devkitA64 as a gthread implementation

[dmiller423]

Threads aren't all that difficult, ::Thread is a wrapper around gthreads, which wrap pthread, which in turn already has an implementation wrapping Thread on switch.

I also at least partially solved the issue with JIT read/write addresses, see: https://gist.github.com/dmiller423/1bac2ba090402417cfedda9122da25a9

However, there is no mmap() implementation unless this has changed since I last looked. IIRC ppsspp used mmap quite extensively. There are direct mapping implementations of mmap()... putting the pieces together and making a full posix compliant mmap() or at least enough to satisfy ppsspp is a considerable challenge though.

[natinusala]

The pthread implementation we use in RetroArch isn't enough to implement everything in gthread I think

Isn't mmap only used to allocate executable memory ? That can be worked around with JIT buffers

[dmiller423]

Yes the RA pthread wrapper is incomplete.
mmap() can be used for a variety of things, if you're referring to it's usage in ppsspp only I would have to look at the source again, @hrydgard or one of the other developers can probably answer easily though.
It could very well be used for file mapping and creating fixed vm areas to use with page guards / exceptions, in which case: the code would have to be changed (for the switch port) or the implementation would have to handle these cases as well.

[hrydgard]

We have a fallback solution, mmap mirroring is not required (although preferred). No page guards are currently used in PPSSPP, unlike Dolphin for example. We just need the ability to make a 256MB allocation (preferably with holes but not required), and that should not be an issue on the Switch.

[natinusala]

The Switch has a big issue though, it can't have RWX memory, only RW xor RX.

We need to transition between RW and RX each time some code is written to a JIT buffer, a transition before writing it and one before executing it. If it slows down N64, it will definitely slow PSP down.

[iOS4all]

The Switch has a big issue though, it can't have RWX memory, only RW xor RX.

We need to transition between RW and RX each time some code is written to a JIT buffer, a transition before writing it and one before executing it. If it slows down N64, it will definitely slow PSP down.

According to @m4xw he said if we patch the kernel we can get RWX. Of course without any slowdown. It’s likely to happen on atmosphere CFW who knows.

For more info here’s the link
https://github.com/libretro/parallel-n64/issues/538#issuecomment-456171446

[hrydgard]

Not an issue even without the patch, we already implemented such a mode for iOS. Slowdown is fairly minor.

[m4xw]

@hrydgard You will need to be able to handle 2 bases (RX and RW buffer have different addrs) and the overhead is compareable to a memcpy of the complete buffer.
If you want to have a single base, the overhead is more.

[hrydgard]

Oh, right, different bases... yeah that complicates things. But can run without the JIT initially, some games will be playable with the IR interpreter I'm sure

[m4xw]

I mean, we have a workaround for that, but it's quite the overhead (we just got support for different bases in, still have to change it to use it).
So interpreter + underperforming dynarec it likely will be.

[dmiller423]

@m4xw can you provide metrics on the overhead?
And is this to cover the cost of changing the vm map back and forth to code memory?
( IE: the example minimal mprotect() I provided ) or another method?

[m4xw]

@dmiller423 https://github.com/libretro/mupen64plus-libretro-nx/blob/mupen_next/switch/mman.h
Thats a hack I wrote for this issue, it works fine for mupen but otherwise is insufficient.
It's not so much different to what you do.
I will provide proper metrics after I implemented some new changes to get a accurate model for a real world application.

[dmiller423]

Sounds good, @hrydgard is there already a variable in the build system for the fallback mode?
Or should I be looking in code for a certain platform (if separate / #if* etc)

[hrydgard]

@dmiller423 Search for PlatformIsWXExclusive()

[dmiller423]

Sorry I meant for the mmap() fallback you mentioned earlier

[hrydgard]

Set the flag MASKED_PSP_MEMORY in MemMap.h. That replaces the mirroring with masking. Then you can create a dummy MemArena implementation that allocates a giant buffer and have CreateView returns pointers offset into that.

The buffer probably really only needs to be sized 0x0B000000 as that's the highest masked valid address that user mode PSP programs can used.

[dmiller423]

Good bit of info, I will check it out

[iOS4all]

Good bit of info, I will check it out

Anything new about this please?

[hrydgard]

@iOS4all If he has something to show, I'm sure he'll show it. Don't be impatient :)

[m4xw]

Patience isn't exactly the strength of this scene

[Gamemulatorer]

@iOS4all "Patience is a virtue"

[DuIslingr]

Atmosphere .8.4 added Jit kernel patches

Kernel patches for JIT support were added (Thanks, @m4xw!).
These loosen restrictions on caller process in svcControlCodeMemory.

https://github.com/Atmosphere-NX/Atmosphere/releases/tag/0.8.4

[m4xw]

@hrydgard Got it to compile after manually building a devkitA64 based toolchain with gthreads enabled newlib.
There are quite a few issues, glslang stuff references pthread, so that needs another toolchain update, my pthread wrapper isnt sufficient (or a rewrite using std::thread).
Lots of Issues like no ftello, fseeko, no ftruncate, no usleep.
No gethostbyname, no pread, overall I had to change ~70 files (tho i think it can probably be cut in half with some better define placement).
Didn't touch JIT stuff yet.
Libretro makefile will need quite a few tweaks I think.
Not sure if I gonna release the toolchain, as I know devkitpro is working on their own solution.
Anyway thats the most common issues that appear, oh and also there are no dynamic libs, so need to rip dlsym stuff out either way (saw that in a few places).
Also glew (as is) will cause troubles.

[hrydgard]

Sounds like good progress. In addition to the missing threads, sounds like a bunch of issues to file in libnx then, as especially the file I/O functions would be nice to have there :)

We don't really use dlsym for anything you'll notice for a Switch GL port.

Yes we really should rip out glew for a number of reasons...

[m4xw]

I will take a look at what's required in newlib to enable them (if supported)

[m4xw]

Actually I just got told there might be a devkitA64 revision very soon, with std::thread and pthreads support.
So hopefully that should do the trick for glslang, meanwhile I can prepare the codebase, also offset based file api.

[spotanjo3]

Sounds good. PPSSPP is a must for Nintendo Switch!

[hrydgard]

@m4xw pthreads is indeed now available in libnx :)

[spotanjo3]

@m4xw pthreads is indeed now available in libnx :)

What does that mean ? :)

[m4xw]

@hrydgard I didn't find a way around missing mman yet.
I tried to do a custom implementation, but I couldn't map user data.
Only got the kernel data mapped at the wanted memory regions...
Alignment etc should be fine for the pages, so no idea why it would complain about invalid memory state in that case (shouldn't be mapped by anything else)
So I rewrote a whole bunch of memareasposix to work from heap and fix all the getBaseAddresses inline funcs etc to return them & a have the process manager service map the process into 32bit address space so it wouldn't malloc in 64bit addr space and truncate the pointers in the blockallocator
Anyway i consider that a giant hack and I guess that would cause a whole bunch of problems while emulating, maybe you have a better idea how this can be handled?

Edit: nvm, got it working with masking (didn't see the memcpy is masked too), should be sufficient.
Weirdly enough masking breaks any filetype mounting, or maybe thats related to the toolchain upgrade /shrug

[iOS4all]

@m4xw pthreads is indeed now available in libnx :)

What does that mean ? :)

From my understanding these things are extremely important for ppsspp in order to run it. Of course besides other things.

[unknownbrackets]

Basically, if you imagine Switch as a world, it previously didn't have gravity. Now it has gravity.

PPSSPP is like a really awesome car, but it's kinda been built with the assumption that gravity will be working on the planet it's used on.

Now we're just working on the next problem, which is that the planet is -150 celcius and the car won't move because it's frozen. So we're trying to figure out if we need to move a sun into place near the planet or if there's some other way to generate heat.

Once all of basic physics works like we thought it would, actually having PPSSPP work will probably not be too hard.

-[Unknown]

[m4xw]

Also another headsup, ofc their pthreads support had to be incomplete (seems to be sufficient so far for glslang, tho I won't test linkage/functionality right now)

[m4xw]

https://twitter.com/m4xwdev/status/1113463278162862091
Got it to run ;)

[Aida-Enna]

Excellent work! inb4 "is it released yet" and "where can I download it". What remains to be worked on?

[m4xw]

Fix JIT, major cleanup, revise all changes, also hook up glsm properly (glew died in the battle).
Static linkage caused major issues for this

[spotanjo3]

Fix JIT, major cleanup, revise all changes, also hook up glsm properly (glew died in the battle).
Static linkage caused major issues for this

Keep up the good work. Get your time and release it when you think it is ready. Amazing.

[hrydgard]

Awesome work m4xw!

If you have managed to kill off glew, please submit that in a separate pull request before anything else. Been meaning to do that for ages, though I planned to simply grab and modify Dolphin's loader to fit us.

[m4xw]

Will do, first i need to do a major cleanup and some other updates for glsm.

[DuIslingr]

Any info regarding a standalone release for the switch?

[m4xw]

Standalone tease https://twitter.com/m4xwdev/status/1151131806055321603