RetroArch failing to display essential special characters

[WeedyWeedSmoker]

Description

There are several issues with the displaying of special characters in RetroArch :

Expected behavior

Showing the special character _œ_ on macOS and iOS in the glui, ozone, and xmb menu interfaces

Actual behavior

On macOS & iOS :

glui = No "_œ_" character _(Used for the translation of "Core" : "Cœur" in french — Important)_
ozone = No "_œ_" character
xmb = No "_œ_" character
rgui = No problems if Extended ASCII Support is turned ON
Integrated menu = No problems

On Windows :

glui = No problems
ozone = No problems
xmb = No problems
rgui = No problems if Extended ASCII Support is turned ON
Integrated menu = No problems

Could this hack found in "/menu/drivers/rgui.c" be implemented for xmb, glui and ozone on macOS ? Or maybe in the font driver as a replacement for this particular character ? Or in CoreText if someone is willing to dig into the broken implementation that still uses kCFStringEncodingASCII ?

Line 2168:

         /* Stupid cretinous hack: 'oe' ligatures are not
          * really standard extended ASCII, so we have to
          * waste CPU cycles performing a conversion from
          * the unicode values...
          * (Note: This is only really required for msg_hash_fr.h) */
         if (symbol == 339) /* Latin small ligature oe */
            symbol = 156;
         if (symbol == 338) /* Latin capital ligature oe */
            symbol = 140;

Version/Commit

• RetroArch: Latest commit

Environment information

• OS: macOS 10.13.6 and iOS 12.1.2
• Compiler: Xcode 9.4

[orbea]

Similar issues, maybe duplicates even?

https://github.com/libretro/RetroArch/issues/7602
https://github.com/libretro/RetroArch/issues/4437

We do have an issue template and its appreciated when its filled out...

[WeedyWeedSmoker]

@orbea Similar issues, yes, but the fact that the integrated menu displays all the special characters (vs. the glui, ozone, or xmb menu drivers) may be a push in the right direction ?

_BTW sorry for not using the template, fixed !_

And BTW :
@twinaphex Do you think the fact that the integrated menu displays all the special characters (but not the glui, ozone, or xmb menu drivers) is a push in the right direction ?

[WeedyWeedSmoker]

@twinaphex Just found this in "/menu/drivers/rgui.c", could it be implemented in xmb, glui and ozone ?

Line 2168:

         /* Stupid cretinous hack: 'oe' ligatures are not
          * really standard extended ASCII, so we have to
          * waste CPU cycles performing a conversion from
          * the unicode values...
          * (Note: This is only really required for msg_hash_fr.h) */
         if (symbol == 339) /* Latin small ligature oe */
            symbol = 156;
         if (symbol == 338) /* Latin capital ligature oe */
            symbol = 140;

[twinaphex]

@WeedyWeedSmoker Hi,

I think the issues you're running into is because the current CoreText implementation is incomplete, and we can't find a developer to finish it.

So what we can do in the meantime is possibly use STB unicode instead, and then just put CoreText being completed on the back burner eventually.

[WeedyWeedSmoker]

@twinaphex That would be great !

I see CoreText is still buggy, so whatever might be a fix, even temporary, would be very appreciated...

_I'm ready to test it if you can implement it, I actually tried to replace CoreText by STB unicode myself, but it failed to compile right away..._

[jdgleaver]

@WeedyWeedSmoker Just a small aside: RGUI will display the characters you mention correctly provided that Extended ASCII Support is turned on under User Interface > Appearance (this is OS independent)

[WeedyWeedSmoker]

@jdgleaver Ok, I just updated the issue with your comment, the fact that it still flew over my head even after seeing this behavior in the code is kind of dumb too...

_BTW: on an unrelated note, the RGUI interface doesn't scroll down until Extended ASCII Support is turned on..._

[jdgleaver]

BTW: on an unrelated note, the RGUI interface doesn't scroll down until Extended ASCII Support is turned on...

Oh, that's interesting...

Could you elaborate? (maybe with a screenshot or something?) I've never seen that behaviour before, and would like to fix it if there's a bug here...

[WeedyWeedSmoker]

@twinaphex That's what I got when simply trying to replace coretext_font_renderer with stb_unicode_font_renderer (compiling for iOS 64-bit)

ld: warning: -no_pie ignored for arm64
Undefined symbols for architecture arm64:
  "_stb_unicode_font_renderer", referenced from:
      _font_renderer_create_default in griffin.o
      _font_backends in griffin.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

@jdgleaver _I can't seem to reproduce the problem, but on my iPad, before enabling Extended ASCII Support, the cursor went just off the screen without scrolling, so I couldn't actually see the Extended ASCII Support option..._

_After manually editing the Retroarch.cfg file to enable just this option, the screen started to scroll again! I don't know what caused this though, even disabling it again doesn't exhibit this behavior anymore..._

[twinaphex]

@WeedyWeedSmoker OK, I added HAVE_STB_FONT for all Apple targets now (macOS + iOS). STB unicode is now being used for font rendering. Languages like Japanese seem to render and display fine in menu drivers like XMB and Ozone, same with languages reliant on accented characters.

[WeedyWeedSmoker]

@twinaphex that's GREAT news ! This has been a very long running bug, I don't know how many people will thank you for that, but let me be the first one...

Thank you so much !!!

[WeedyWeedSmoker]

@twinaphex Works damn fine for all languages... I kinda freaked out when it was extracting the assets as the font used was the same pixellated one from rgui, but upon extracting them, the font went back to the correct one, even for the integrated menu...

Closing the issue, and I think you can do the same for the other languages' issues too... Thanks !