Retroarch: [Feature request] GXM driver for hardware acceleration on PS Vita

Created on 24 Jun 2019  ·  105Comments  ·  Source: libretro/RetroArch

As of today, RetroArch / Libretro on the PS Vita is using software rendering. This is preventing cores from using the full potential of the device in terms of, of course, 3D emulation, but hardware acceleration could also do wonders for 2D display.

Examples of GXM code found here : https://github.com/xerpi/gxmfun
A little bit more context here : https://www.reddit.com/r/vitahacks/comments/5pbdgo/having_fun_with_libgxm_psvitas_3d_graphics_api/

Most helpful comment

Some status report:

  • I have made good progress and I got runtime shader compiler working. About distribution of the suprx, a tool is needed for the user to install it.
  • I have updated vita2d driver with latest one
  • I'm implementing scissor support for the menu driver
  • I'm taking a look at ozone and menu widgets (it is very uggly on vita)

I think the final driver will be based on vita2d but exposing the context to the cores. Of course I will add shader pipelines to it

All 105 comments

finally we will have shaders support in PS Vita with this feature?
👍

@Nube79 if someone is willing to spend thousands of hours of work to write a runtime shader compiler (after reversing GPU ISA since it's not a "standard" well documented GPU) and then proceed to translate every single GL call in GL enabled cores to GXM relatives just for few bucks, then yep.

Spoiler: Won't happen.

@Nube79 if someone is willing to spend thousands of hours of work to write a runtime shader compiler (after reversing GPU ISA since it's not a "standard" well documented GPU) and then proceed to translate every single GL call in GL enabled cores to GXM relatives just for few bucks, then yep.

Spoiler: Won't happen.

I think Libretro/Retroarch is a bigger project than ScummVM.
ScummVM has the shaders working for PS Vita, is GNU Open Source, it can be a reference guide to get it ready on retroarch.

@Rinnegatamante hi there, @frangarcj will be taking on this bounty. I believe he's still orientating his way around it. I know there's some problematic things along the way, such as Cg instead of GLSL, and so on. The runtime compiler is not strictly necessary, it would be possible to just compile them beforehand, it would just be less convenient but it could be done just for the Vita port. At least there is nothing in RetroArch's source code preventing shader binaries from being loadable. I admittedly know nothing about the Vita side though.

He also will be bringing GL over to RetroArch Vita in addition to this libgxm hardware context.

@twinaphex Runtime shader compiler not being strictly necessary means you would have to have precompiled binaries of every single possible shader a core requires (which would translate in having a mess in terms of code, for reference look at: https://github.com/Rinnegatamante/vitaGL/blob/master/source/vitaGL.c#L219-L581 (And here we're talking just about some very simple shader to emulate GL fixed function pipeline (eg: https://github.com/Rinnegatamante/vitaGL/blob/master/shaders/texture2d_v.cg +https://github.com/Rinnegatamante/vitaGL/blob/master/shaders/texture2d_f.cg ).

"Bringing GL over to RetroArch Vita". Vita has no native GL support.

There are no GL cores or HW accelerated cores on vita so that's completely
irrelevant.
They are talking about post-processing shaders.

Shipping pre-compiled packs for vita would be as easy as the current
non-compiled ones.

On Sun, Jun 30, 2019 at 11:31 AM Rinnegatamante notifications@github.com
wrote:

@twinaphex https://github.com/twinaphex Runtime shader compiler not
being strictly necessary means you would have to have precompiled binaries
of every single possible shader a core requires (which would translate in
having a mess in terms of code, for reference look at:
https://github.com/Rinnegatamante/vitaGL/blob/master/source/vitaGL.c#L219-L581
(And here we're talking just about some very simple shader to emulate GL
fixed function pipeline (eg:
https://github.com/Rinnegatamante/vitaGL/blob/master/shaders/texture2d_v.cg
+
https://github.com/Rinnegatamante/vitaGL/blob/master/shaders/texture2d_f.cg
).

"Bringing GL over to RetroArch Vita". Vita has no GL support.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/libretro/RetroArch/issues/9015?email_source=notifications&email_token=AANEFUDNEGW3Z5BO4NZQBGTP5DNWHA5CNFSM4H3AN3QKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY4PNRQ#issuecomment-507049670,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AANEFUBO547WBSOVIMCGZVLP5DNWHANCNFSM4H3AN3QA
.

What people are looking for are postporcessing, it is not necessary to have many shaders, with a scale2x or those that have ScummVM that are already adapted for Vita, it would be enough.

https://en.wikipedia.org/wiki/Pixel-art_scaling_algorithms#/media/File:Pixel-Art_Scaling_Comparison.png

I don't know what is the limit on vita GPU, there are many shader algorithms that are very great.

Uh no the purpose of this issue+bounty is more about hw accelerated 3d for the cores that would benefit from it (mupen64, pcsx, etc) than merely post processing shaders...

finally we will have shaders support in PS Vita with this feature?
👍

Sure @kwyxz , I know, but my question is about your comment:

"... but hardware acceleration could also do wonders for 2D display."

Let's clarify things:

It's better to split the bounty in two:

  1. GXM drivers for retroarch. This will allow for a better UI, (optional runtime) shader support and hw context available for cores. Maybe this will include a native gxm driver and a VitaGL one (at least one of them).

  2. A core with 3D support. e.g. Get mupen64 or pcsx_rearmed with 3D HW acceleration.

In order to do 2, you need first to do 1.

Also for cores lilke mupen64, there's no dynarec support atm.

Would you agree to leave current money / bounty to the first one?

@frangarcj I am fine with that proposal. Lets hear from the bounty backers if they agree so that the work for this bounty is clear.

I agree with it too. I wasn't exactly sure of how all this was going to work and how to word it (it really isn't a domain I'm familiar with) but my original idea was that if a gxm driver was available it could be an incentive to add hw accel to the cores themselves, maybe with specific bounties afterwards.

Thank you!

_1.GXM drivers for retroarch. This will allow for a better UI, (optional runtime) shader support and hw context available for cores. Maybe this will include a native gxm driver and a VitaGL one (at least one of them)._

This is actually why I backed this bounty. If this can improve overall UI/ playlist performance, then count me in.

Yeah! @Defkyle the playlist screenshot performance and the shader support for 2D cores is a great first goal.
And the hardware acceleration for the 3D cores sounds good for a second stage.

PD: When you get down on a large playlist, this comes terrible slow.

Then let's start working on that ;)

With my measly input of $25, I'm not sure I get a vote, but aye.
If this enables smoother scrolling through playlists, double aye.

Just added $30 to the bounty.

Hi guys!

I'm not sure to understand the first part of the bounty. I did take a quick look to retroarch sources and the vita port does use hardware rendering (via vita2d). It just miss some shaders, @frangarcj already worked on that in the past, and I did implement in scummvm and pfba. But people seems to report it use software rendering, am I missing something? (Well, probably :)).

vita2d is used only to draw the final framebuffer. This means RetroArch is indeed soft rendered.

@cpasjuste optimized gxm driver (not based on vita2d) with shader support and runtime shader compiler. It will be also exposed to cores.

PD: Glad to hear from you

Hi @frangarcj, same here, happy to see you :)

Else, "you're lucky", i have so much projects going on, but if not (and if you were not interested), i would do it 😄 Good luck with that, i'm 100% sure you'll succeed.

See you!

Let's clarify things:

It's better to split the bounty in two:

  1. GXM drivers for retroarch. This will allow for a better UI, (optional runtime) shader support and hw context available for cores. Maybe this will include a native gxm driver and a VitaGL one (at least one of them).
  2. A core with 3D support. e.g. Get mupen64 or pcsx_rearmed with 3D HW acceleration.

In order to do 2, you need first to do 1.

Also for cores lilke mupen64, there's no dynarec support atm.

Would you agree to leave current money / bounty to the first one?

As we have now doubled the amount, would it be acceptable to split the money between the two tasks? I'm also fine leaving more money for the second task and giving the current amount for the first task, it's just great that progress is being made for the Vita.

@frangarcj pcsx rearmed does not feature hw rendering. Are you implying you are willing to writte one? If so that would be great as many other platforms will be benefied.

@billy-acuna I'm thinking about a gxm specific one so maybe not directly useful for other platforms

Some status report:

  • I have made good progress and I got runtime shader compiler working. About distribution of the suprx, a tool is needed for the user to install it.
  • I have updated vita2d driver with latest one
  • I'm implementing scissor support for the menu driver
  • I'm taking a look at ozone and menu widgets (it is very uggly on vita)

I think the final driver will be based on vita2d but exposing the context to the cores. Of course I will add shader pipelines to it

I love you @frangarcj
When will we be able to test a partial alpha on nightly compilations?
https://buildbot.libretro.com/nightly/playstation/vita/

@frangarcj Some status report:

* I have made good progress and I got runtime shader compiler working. About distribution of the suprx, a tool is needed for the user to install it.

* I have updated vita2d driver with latest one

* I'm implementing scissor support for the menu driver

* I'm taking a look at ozone and menu widgets (it is very uggly on vita)

I think the final driver will be based on vita2d but exposing the context to the cores. Of course I will add shader pipelines to it

Hi there @frangarcj . First of all, let me thank you and praise your effort on this matter. This bounty has pretty much become one of the most exciting developments for PS Vita ATM :).

Progress is being followed at spanish forum elotrolado.net, but we users are confused about the outcome of part 1 of the bounty. May I ask you 3 simple questions on their behalf?

  1. When part 1 is done, does it automatically entail better performance for every 2D core? Some think it will, some think it won't.

  2. If yes: Does it mean 2X Scale and/or crisper graphics? (Obviously, depending on the system. NES or MegaDrive might do, but CPS2 could be too much). Is that too much for PS Vita GPU?

  3. If no: What performance improvements are to be expected for 2D emulation?

I backed you at bountysource, sir. You really deserve it. Your work is truly marvelous and really appreciated :). Keep it up 👍

Hi!!

  1. I think that maybe we can get a small performance improvement, but very small. Retroarch is already using HW textures for 2D cores. However I'm trying to reduce the number of memory operations.

  2. About shaders like 2X Scale, sharp bilinear, scanlines, etc... it will depend on the shader complexity but at least you will get the same shaders than with Adrenaline. And it's not a matter of the system because the system is emulated on cpu and the shaders will be a gpu thing.

You will get also a better menu experience with gradients, better performance and less memory footprint.

and finally, if you do a small setup you will have a runtime shader compiler that will be very useful for 3D cores.

unknown

BTW work is being done in Vita3k

Great!
We can disable the bilinear filter and apply Scale2X two or three times like other platforms?

Hi!!

1. I think that maybe we can get a small performance improvement, but very small. Retroarch is already using HW textures for 2D cores. However I'm trying to reduce the number of memory operations.

2. About shaders like 2X Scale, sharp bilinear, scanlines, etc... it will depend on the shader complexity but at least you will get the same shaders than with Adrenaline. And it's not a matter of the system because the system is emulated on cpu and the shaders will be a gpu thing.

You will get also a better menu experience with gradients, better performance and less memory footprint.

and finally, if you do a small setup you will have a runtime shader compiler that will be very useful for 3D cores.

unknown

BTW work is being done in Vita3k

Hey there, just wanted to say thank you for your kind reply :).

Can we try the scale2x?

I want to attempt this challenge, where could I go to get documentation about writing drivers for retroarch?

@frangarcj Any progress or updates on the drivers?

@1mpl0de He's only working on vita2d driver afaik

I'm working on an optimized Vita gxm driver and new menu driver. Now I'm on holidays but still working. Be patient

Any news?

I've detected a problem with threads while looking at cpu video filters.

* I have made good progress and I got runtime shader compiler working. About distribution of the suprx, a tool is needed for the user to install it.

@frangarcj By runtime shader compiling, do you mean we'll be able to compile shaders in source (CG) form on the device, and not have to depend on that closed-source Windows program called psp2cgc (or something like that)?

* I have made good progress and I got runtime shader compiler working. About distribution of the suprx, a tool is needed for the user to install it.

@frangarcj By runtime shader compiling, do you mean we'll be able to compile shaders in source (CG) form on the device, and not have to depend on that closed-source Windows program called psp2cgc (or something like that)?

YES

I've detected a problem with threads while looking at cpu video filters.

Yeah! I love you Fran.
Is the last nightly patched?

* I have made good progress and I got runtime shader compiler working. About distribution of the suprx, a tool is needed for the user to install it.

@frangarcj By runtime shader compiling, do you mean we'll be able to compile shaders in source (CG) form on the device, and not have to depend on that closed-source Windows program called psp2cgc (or something like that)?

YES

Wow. So the GPU ISA was fully reverse engineered? That's incredible. That also means the emulator (vitka3k) can execute any shader as opposed to having a database of known shaders!

Truly, the vita homebrew scene is a marvel. Kudos to frangarcj and other folks for all the great work!

Any news in GMX driver development?

^ I join the question

* I have made good progress and I got runtime shader compiler working. About distribution of the suprx, a tool is needed for the user to install it.

@frangarcj By runtime shader compiling, do you mean we'll be able to compile shaders in source (CG) form on the device, and not have to depend on that closed-source Windows program called psp2cgc (or something like that)?

YES

Wow. So the GPU ISA was fully reverse engineered? That's incredible. That also means the emulator (vitka3k) can execute any shader as opposed to having a database of known shaders!

Truly, the vita homebrew scene is a marvel. Kudos to frangarcj and other folks for all the great work!

Not really, the runtime shader compiler in question is still something made by Sony. Was part of old firmwares and then removed (starting from fw 1.80 or around that time). The executable is still part of PSM Runtime however which is a DRM free package meaning an extractor for that executable will be provided so that this specific runtime shader compiler will be usable for homebrews.

GPU ISA RE work is being done for Vita3K development instead for the inverse process (CG -> GLSL)

@rinnegatamante is totally right. That is the plan

News, updates, progress... Please!

Any news, updates, progress being done? (I'm NOT asking for ETA, I'm just interested in the progress being done)

@Electric1447 @frangarcj only answer to Mr. @Rinnegatamante

When do you think the Scale2x shader will be available?

Just to clear things up since i think stuff had been misunderstood by a lot of people:

What will this bounty achieve:

  • Menu/Menu Widgets working in a better way
  • Post processing shaders support

What this bounty won't achieve (and won't help development for):

  • Hardware acceleration in cores

Even when this bounty will be collected, progresses for hw accelearation in cores would be the same than now. Gonna explain why:

RetroArch workflow can be divided in two macrosections: frontend (RetroArch itself) and backend (cores).

Frontend can expose to cores an hardware context which basically is composed of a context to use GPU and a function to grab addresses for the functions exposed from that API (i'm simplifing stuffs since it's way more complex than this). Cores, after getting those functions, can use the related API to get advantage of the GPU and draw everything on a framebuffer. This framebuffer, after a frame is fully rendered is returned to the frontend which then performs a last rendering stage where it applies post-processing shaders as well as drawing on top of it menu/layout/OSD warnings etc.
So, why adding a new HW_CONTEXT (eg. HW_CONTEXT_GXM or HW_CONTEXT_VITA2D) won't make any differences in terms of cores being able to access GPU?
If you well understood my little explanation, you probably got the point: basically EVERY core would need to be COMPLETELY rewrote from the groundup to use the new exposed API instead of GL/Vulkan/D3D. This summarize in basically having to port the whole emulator/game engine the core is to a brand new API and this work won't be done by anyone (it would actually imho be simpler to just port an emulator to GXM in a standalone way instead of having to rely on libretro since libretro itself adds some limitations at least on the Vita build such as, for example, the unability to increase main thread stack size). I've recently proposed something that could actually indeed make Vita build (as well as 3DS and PSP builds) get hardware acceleration in (at least some) cores relatively easily.
This will need basically to add still a new hw context based on the already implemented "gl1" video driver. gl1 video driver atm has no hw context capability cause of how hw context itself works for GL API but things can be sorted out to work with GL1.x too. Once we have a HW_CONTEXT_OPENGL1, we can easily drop a new renderer in vita build that uses vitaGL and fully implements said hardware context. The advantages to have an OPENGL1 hw context instead of a GXM one are multiples:

  • There are tons of documentation and samples on how to use GL1 compared to GXM.
  • There are tons of experienced developers with GL1 compared to GXM.
    but MOST IMPORTANTLY:
  • Several cores actually using HW_CONTEXT_OPENGL can be easily adapted to use only GL1.X features making them easily portable.
  • Other platforms can benefit of having an OPENGL1 hw context (as mentioned, 3DS and PSP for example)
    That's the referenced opened issue for this implementation if anyone's interested on ETA/who's working on it/contributing via bounties/whatever else came in your minds: https://github.com/libretro/RetroArch/issues/9603

thanks @Rinnegatamante

This contribution of information is appreciated. Thank you! @Rinnegatamante

Thanks a lot for this detailed explanation @Rinnegatamante - I will support #9603 as well.

any news about post processing shader support?

Actual state:

  • GXM driver has more features and menu / menu widgets are working nicely
  • Needs some code cleaning but I will upload it this week
    Missing things:
  • Pipelines (your gpu shaders and some moving ribbon and snow

Bonus:

  • Scaled OZONE

2019-11-13-205202

FYI, these fixes were praised in the last episode of the Retro Game Club Podcast.

Awesome news @frangarcj and Ozone looks gorgeous!

I love you Fran, Genial!

Thanks a lot @frangarcj

Actual state:

  • GXM driver has more features and menu / menu widgets are working nicely
  • Needs some code cleaning but I will upload it this week
    Missing things:
  • Pipelines (your gpu shaders and some moving ribbon and snow

Are the changes implimented in the latest nighly?

Are the changes implimented in the latest nighly?

Currently the nightlys are broken.

image

@Nube79 This bug was hopefully fixed by @frangarcj in https://github.com/libretro/RetroArch/pull/9757

So new nightly builds will come hopefully tomorrow again.

I've detected a problem with threads while looking at cpu video filters.

The games continue to freeze after a while when you use filters.

@frangarcj Do the filters work, seems that Nube is getting freezes.

@frangarcj Do the filters work, seems that Nube is getting freezes.

Depending on the game and the filter, the game may take a few seconds or more than a minute to freeze.

I'm getting crashes when switching to gl1 from vita2d

Unless I misunderstood something, gl1 is not GXM so this is not the right issue to report here.

@Electric1447 @Rinnegatamante suggested some settings to avoid glitches:

https://twitter.com/Rinnegatamante/status/1197446070835519489

Please test next nightly with vita2d driver and let me know if it is better or worse

@frangarcj filters keep freezing and PSX Core Crash when using higher resolution

@frangarcj filters keep freezing and PSX Core Crash when using higher resolution

I'm waiting for an working Scale2x for years, filter or shader.

You'll get it soon

Any progress? @frangarcj

I want to try this before I die in WW3 lol

I have added a fix for video filters. I've been looking at increasing the performance of mame on vita. I'll resume work on gxm drivers.

https://github.com/libretro/RetroArch/pull/9942

@frangarcj

I have added a fix for video filters. I've been looking at increasing the performance of mame on vita. I'll resume work on gxm drivers.

https://www.youtube.com/watch?v=Xt8q4Ii8I7Q

¡Yeah! The filters work perfectly now.
Thanks Fran.

¡Yeah! The filters work perfectly now.
Thanks Fran.

What filters? What core?

All filters on all cores, best results with 16bit cores and lower, including mame, fba etc...

Scale2x and EPX works perfect and has good performance.

2020-01-08-174516

@Nube79 Thanks for testing, more screenshots welcome.

@Nube79 Thanks for testing, more screenshots welcome.

I created a video that shows the filter and performance:
system: gba
pixel size: 3x
filter: scale2x

Perfect performance and nice effect.
When Fran release the shaders, with scale3x it will look even better

scale2x.zip

PD: And rewind enabled without losing performance.

@Nube79 Great, where did you get the Castlevania border?

Can your give some tutorial for using this feature?

@Nube79 Great, where did you get the Castlevania border?

The logo is from NETFLIX, the images from PSX DraculaX.

Can your give some tutorial for using this feature?

it's easy:
go to https://buildbot.libretro.com/nightly/playstation/vita/
and download the last build. and don't forget the data.

in retroarch, go to video, select filters,
go to ux0:/data/retroarch/filters/video/
and select scale2x

bilinear ON customscanlines ON

2020-01-10-004554

bilinear ON customscanlines ON scale2x ON

2020-01-10-004525

bilinear ON customscanlines ON scale3x ON

(When Fran finishes adapting the shaders...)

bilinear ON customscanlines ON

2020-01-10-004554

bilinear ON customscanlines ON scale2x ON

2020-01-10-004525

bilinear ON customscanlines ON scale3x ON

(When Fran finishes adapting the shaders...)

Con eso te aguanta a 60 fps?

Con eso te aguanta a 60 fps?

sí, y con el rewind activo "Nivel de detalle de rebobinado 4"

2020-01-10-031449

Incluso bajando a 333 Mhz
333mhz

The playlists have improved in performance:
Video:
RetroarchPlaylistPerf.zip

Hello frangarcj. With your help retroarch Vita have a second breath for me and Thank for work on it.
Do you think it's possible to write SABR filter like the shader on Retroarch. That the most Smooth and BEATIFULL !
I dont know about 500mhz on it can work correctly with gpu...?
Thank for all your work.
Best regard.

@frangarcj Amazing! Glad you're resuming work on the gxm driver. I really want to get something like Super Mario 64 or Pilotwings working at great speeds on the vita for my brother. If you need any help with it hit me up I know a thing or two about graphics ;)

😕. its Just an filter.

😕. its Just an filter.

The filters/shaders on 2D emulators are very important to many people.

bilinear ON customscanlines ON

2020-01-10-004554

bilinear ON customscanlines ON scale2x ON

2020-01-10-004525

bilinear ON customscanlines ON scale3x ON

(When Fran finishes adapting the shaders...)

Hi! How do you add customscanlines? don't see the option. Thanks!

Hi! How do you add customscanlines? don't see the option. Thanks!

You need to use overlays.

I will close this issue but more things will come later

Poggers

Nice, its mean now we have big performance boost on RetroArch Vita?

When we will receive advantages from this bounty?

Some of them are already on Retroarch as you can see above. Others are integrated on the support for the VitaGL driver.

What's left? A example core with gxm hw context and cg shaders. I'll do but no eta

So, since we've been waiting a while, I actually had a question. The request was for there to be a driver for hardware acceleration. If somebody were to make a full GL implementation that could be easily applied to RetroArch by the maintainers, would that be enough? VitaGL is good in all, but it still lacks many of the standard GL features. It would be easier to just expand on that to find better, easier ways to make it work seamlessly.

In the long run, it would be better and more beneficial with less problems in the future. I feel like this bounty was paid out too prematurely. We're nearly 3 months out now and still haven't seen anything.

Yeah, that's exactly what i think, I don't feel any performance boost or improvement for RA Vita from this bounty yet, or someone can explain how to use it?

I think some kind of guide on what has improved would be great, I am also not aware and have donated hoping for something else.

Hi,

The performance boosts were added previously to closing the bounty and some of them are confirmed here like the playlist speed.

The menu is already at 60 fps, some other drivers were added like the ozone one, now you can use widgets, filters were fixed too, etc.

As I said before: What's left? A example core with gxm hw context and cg shaders. I'll do but no eta
The example hw context will be mgba which is slower on retroarch than on standalone.

This requires some obscure work on merging VitaGL and GXM drivers. What to expect can be seen on the DC emulator (that also uses runtime shaders).

I'm having several personal problems that restrain my time but I have promised to add these asap.

The bounty was claimed as requested by some of the people who have pledge more money. They were mainly worried about some changes in ToS from bountysource.

Sorry if someone was disappointed.

OK, thank you for your answer and good luck with your personal life!

Was this page helpful?
0 / 5 - 0 ratings

Related issues

parkerlreed picture parkerlreed  ·  3Comments

Chocobubba picture Chocobubba  ·  3Comments

wrldwzrd89 picture wrldwzrd89  ·  3Comments

RobLoach picture RobLoach  ·  3Comments

alki-apps picture alki-apps  ·  3Comments