Retroarch: RDB files are unreadable when compiled with GCC 10

Created on 26 May 2020  路  55Comments  路  Source: libretro/RetroArch

Description

The Import Content feature appears to be broken. Selecting scan by folder and file yield the same result. The GUI indicates it's stuck scanning the first file it comes across and remains at 0%, never progresses and must be restarted. Also happens when running as the root user

Expected behaviour

[What you expected to happen]

Actual behavior

[What is actually happening]

Steps to reproduce the bug

  1. Compile from source, git clone, ./configure, make, sudo make install
  2. Capture some output with retroarch --menu --verbose >> retroarch.log 2>&1
  3. Stripe out Vulkan and slang noise with sed.
  4. Attempt to import any content either file or folder style
  5. Remains at 0%, logs show the last it recognises a file and matches a 'disc label'

Bisect Results

Appeared to start with 1.8.7

Version/Commit

Git Version: fda235518a

  • RetroArch: 1.8.8

Environment information

  • OS: Fedora 32
  • Compiler: gcc/g++ 10.1.1 20200507 (Red Hat 10.1.1-1)
prs welcome

Most helpful comment

This is not a bug in GCC optimization. There is an odd behavior in those optimizations but i think the underlying problem is a buffer overflow in RetroArch code.

GCC10 in -O3 and -O2 mode removes the following conditional statement
if (strncmp(header.magic_number, MAGIC_NUMBER, sizeof(MAGIC_NUMBER)) != 0)
in libretro-db/libretrodb.c.
The generated assembly code do the comparison and then directly jump to the error label regardless of the value.

The suspected bufferoverflow is in struct libretrodb_header declaration.
typedef struct libretrodb_header { char magic_number[sizeof(MAGIC_NUMBER)-1]; uint64_t metadata_offset; } libretrodb_header_t;

IMHO should be
typedef struct libretrodb_header { char magic_number[sizeof(MAGIC_NUMBER)]; uint64_t metadata_offset; } libretrodb_header_t;

The fix is really small but i can do PR if you prefer.

All 55 comments

[INFO] RetroArch 1.8.8 (Git fda235518a)
[INFO] === Build =======================================
[INFO] CPU Model Name: AMD Ryzen 5 3400G with Radeon Vega Graphics
[INFO] Capabilities: MMX MMXEXT SSE SSE2 SSE3 SSSE3 SSE4 SSE4.2 AES AVX AVX2
[INFO] Built: May 26 2020
[INFO] Version: 1.8.8
[INFO] Git: fda235518a
[INFO] =================================================
[INFO] Version of libretro API: 1
[INFO] Compiled against API: 1
[INFO] SRAM will not be saved.
[INFO] Loading history file: [/home//.config/retroarch/content_history.lpl].
[INFO] Loading history file: [/home//.config/retroarch/content_music_history.lpl].
[INFO] Loading history file: [/home//.config/retroarch/content_video_history.lpl].
[INFO] Loading history file: [/home//.config/retroarch/content_image_history.lpl].
[INFO] Loading favorites file: [/home//.config/retroarch/content_favorites.lpl].
[INFO] Parsing CUE file '/home//rom/ps1/Alundra (Europe).cue'...
[INFO] Reading first data track...
[INFO] Comparing with known magic numbers...
[INFO] Found disc label 'SLES-01135'
[INFO] [config] Saved new config to "/home//.config/retroarch/retroarch.cfg".
[INFO] Content ran for a total of: 00 hours, 00 minutes, 00 seconds.
[INFO] Unloading game..
[INFO] Unloading core..
[INFO] Unloading core symbols..

do you have the databases?

I would think so, I've always downloaded them from the internal updater thing.

/home//.config/retroarch/database/rdb>ll
total 65M
-rw-rw-r--. 1 957K May 26 19:58 Amstrad - CPC.rdb
-rw-rw-r--. 1 123K May 26 19:58 Atari - 2600.rdb
-rw-rw-r--. 1 27K May 26 19:58 Atari - 5200.rdb
-rw-rw-r--. 1 49K May 26 19:58 Atari - 7800.rdb
-rw-rw-r--. 1 47K May 26 19:58 Atari - Jaguar.rdb
-rw-rw-r--. 1 36K May 26 19:58 Atari - Lynx.rdb
-rw-rw-r--. 1 80K May 26 19:58 Atari - ST.rdb
-rw-rw-r--. 1 35K May 26 19:58 Bandai - WonderSwan Color.rdb
-rw-rw-r--. 1 42K May 26 19:58 Bandai - WonderSwan.rdb
-rw-rw-r--. 1 209 May 26 19:58 Cannonball.rdb
-rw-rw-r--. 1 2.4K May 26 19:58 Casio - Loopy.rdb
-rw-rw-r--. 1 2.2K May 26 19:58 Casio - PV-1000.rdb
-rw-rw-r--. 1 581 May 26 19:58 Cave Story.rdb
-rw-rw-r--. 1 1.2K May 26 19:58 ChaiLove.rdb
-rw-rw-r--. 1 48K May 26 19:58 Coleco - ColecoVision.rdb
-rw-rw-r--. 1 62K May 26 19:58 Commodore - 64.rdb
-rw-rw-r--. 1 696K May 26 19:58 Commodore - Amiga.rdb
-rw-rw-r--. 1 2.0K May 26 19:58 Commodore - Plus-4.rdb
-rw-rw-r--. 1 52K May 26 19:58 Commodore - VIC-20.rdb
-rw-rw-r--. 1 298 May 26 19:58 Dinothawr.rdb
-rw-rw-r--. 1 19K May 26 19:58 DOOM.rdb
-rw-rw-r--. 1 2.1M May 26 19:58 DOS.rdb
-rw-rw-r--. 1 9.3K May 26 19:58 Emerson - Arcadia 2001.rdb
-rw-rw-r--. 1 847 May 26 19:58 Entex - Adventure Vision.rdb
-rw-rw-r--. 1 8.4K May 26 19:58 Epoch - Super Cassette Vision.rdb
-rw-rw-r--. 1 8.9K May 26 19:58 Fairchild - Channel F.rdb
-rw-rw-r--. 1 900K May 26 19:58 FBNeo - Arcade Games.rdb
-rw-rw-r--. 1 256 May 26 19:58 Flashback.rdb
-rw-rw-r--. 1 2.2K May 26 19:58 Funtech - Super Acan.rdb
-rw-rw-r--. 1 8.1K May 26 19:58 GamePark - GP32.rdb
-rw-rw-r--. 1 12K May 26 19:58 GCE - Vectrex.rdb
-rw-rw-r--. 1 12K May 26 19:58 Handheld Electronic Game.rdb
-rw-rw-r--. 1 3.1K May 26 19:58 Hartung - Game Master.rdb
-rw-r--r--. 1 1.4M May 26 19:58 HBMAME.rdb
-rw-rw-r--. 1 21K May 26 19:58 LeapFrog - Leapster Learning Game System.rdb
-rw-rw-r--. 1 977 May 26 19:58 Lutro.rdb
-rw-rw-r--. 1 29K May 26 19:58 Magnavox - Odyssey2.rdb
-rw-rw-r--. 1 439K May 26 19:58 MAME 2000.rdb
-rw-rw-r--. 1 606K May 26 19:58 MAME 2003-Plus.rdb
-rw-rw-r--. 1 862K May 26 19:58 MAME 2003.rdb
-rw-rw-r--. 1 1.6M May 26 19:58 MAME 2010.rdb
-rw-rw-r--. 1 2.1M May 26 19:58 MAME 2015.rdb
-rw-rw-r--. 1 2.3M May 26 19:58 MAME 2016.rdb
-rw-rw-r--. 1 6.8M May 26 19:58 MAME.rdb
-rw-rw-r--. 1 86K May 26 19:58 Mattel - Intellivision.rdb
-rw-rw-r--. 1 40K May 26 19:58 Microsoft - MSX2.rdb
-rw-rw-r--. 1 241K May 26 19:58 Microsoft - MSX.rdb
-rw-rw-r--. 1 630K May 26 19:58 Microsoft - Xbox.rdb
-rw-rw-r--. 1 150 May 26 19:58 MrBoom.rdb
-rw-rw-r--. 1 157K May 26 19:58 NEC - PC Engine CD - TurboGrafx-CD.rdb
-rw-rw-r--. 1 1.1K May 26 19:58 NEC - PC Engine SuperGrafx.rdb
-rw-rw-r--. 1 124K May 26 19:58 NEC - PC Engine - TurboGrafx 16.rdb
-rw-rw-r--. 1 35K May 26 19:58 NEC - PC-FX.rdb
-rw-rw-r--. 1 852K May 26 19:58 Nintendo - e-Reader.rdb
-rw-rw-r--. 1 165K May 26 19:58 Nintendo - Family Computer Disk System.rdb
-rw-rw-r--. 1 1.1M May 26 19:58 Nintendo - Game Boy Advance.rdb
-rw-rw-r--. 1 492K May 26 19:58 Nintendo - Game Boy Color.rdb
-rw-rw-r--. 1 489K May 26 19:58 Nintendo - Game Boy.rdb
-rw-rw-r--. 1 905K May 26 19:58 Nintendo - GameCube.rdb
-rw-rw-r--. 1 277K May 26 19:58 Nintendo - Nintendo 3DS (Digital).rdb
-rw-rw-r--. 1 566K May 26 19:58 Nintendo - Nintendo 3DS.rdb
-rw-rw-r--. 1 3.9K May 26 19:58 Nintendo - Nintendo 64DD.rdb
-rw-rw-r--. 1 573K May 26 19:58 Nintendo - Nintendo 64.rdb
-rw-rw-r--. 1 122K May 26 19:58 Nintendo - Nintendo DS (Download Play) (BETA).rdb
-rw-rw-r--. 1 188K May 26 19:58 Nintendo - Nintendo DS (Download Play).rdb
-rw-rw-r--. 1 2.2K May 26 19:58 Nintendo - Nintendo DSi Decrypted.rdb
-rw-rw-r--. 1 224K May 26 19:58 Nintendo - Nintendo DSi (Digital).rdb
-rw-rw-r--. 1 2.9K May 26 19:58 Nintendo - Nintendo DSi.rdb
-rw-rw-r--. 1 2.0M May 26 19:58 Nintendo - Nintendo DS.rdb
-rw-rw-r--. 1 1.4M May 26 19:58 Nintendo - Nintendo Entertainment System.rdb
-rw-rw-r--. 1 11K May 26 19:58 Nintendo - Pokemon Mini.rdb
-rw-rw-r--. 1 77K May 26 19:58 Nintendo - Satellaview.rdb
-rw-rw-r--. 1 4.7K May 26 19:58 Nintendo - Sufami Turbo.rdb
-rw-rw-r--. 1 1.2M May 26 19:58 Nintendo - Super Nintendo Entertainment System.rdb
-rw-rw-r--. 1 8.2K May 26 19:58 Nintendo - Virtual Boy.rdb
-rw-rw-r--. 1 797K May 26 19:58 Nintendo - Wii (Digital).rdb
-rw-rw-r--. 1 1.4M May 26 19:58 Nintendo - Wii.rdb
-rw-rw-r--. 1 6.1K May 26 19:58 Philips - Videopac+.rdb
-rw-r--r--. 1 0 May 26 20:02 poop.txt
-rw-r--r--. 1 225 May 26 19:58 Quake III.rdb
-rw-r--r--. 1 422 May 26 19:58 Quake II.rdb
-rw-rw-r--. 1 680 May 26 19:58 Quake.rdb
-rw-rw-r--. 1 3.8K May 26 19:58 RCA - Studio II.rdb
-rw-rw-r--. 1 161 May 26 19:58 Rick Dangerous.rdb
-rw-rw-r--. 1 615 May 26 19:58 RPG Maker.rdb
-rw-rw-r--. 1 1.1M May 26 19:58 ScummVM.rdb
-rw-rw-r--. 1 58K May 26 19:58 Sega - 32X.rdb
-rw-rw-r--. 1 378K May 26 19:58 Sega - Dreamcast.rdb
-rw-rw-r--. 1 224K May 26 19:58 Sega - Game Gear.rdb
-rw-rw-r--. 1 162K May 26 19:58 Sega - Master System - Mark III.rdb
-rw-rw-r--. 1 109K May 26 19:58 Sega - Mega-CD - Sega CD.rdb
-rw-rw-r--. 1 672K May 26 19:58 Sega - Mega Drive - Genesis.rdb
-rw-rw-r--. 1 129K May 26 19:58 Sega - PICO.rdb
-rw-rw-r--. 1 611K May 26 19:58 Sega - Saturn.rdb
-rw-rw-r--. 1 63K May 26 19:58 Sega - SG-1000.rdb
-rw-rw-r--. 1 780K May 26 19:58 Sharp - X68000.rdb
-rw-rw-r--. 1 67K May 26 19:58 Sinclair - ZX 81.rdb
-rw-rw-r--. 1 27K May 26 19:58 Sinclair - ZX Spectrum +3.rdb
-rw-rw-r--. 1 16M May 26 19:58 Sinclair - ZX Spectrum.rdb
-rw-rw-r--. 1 31K May 26 19:58 SNK - Neo Geo CD.rdb
-rw-rw-r--. 1 38K May 26 19:58 SNK - Neo Geo Pocket Color.rdb
-rw-rw-r--. 1 3.3K May 26 19:58 SNK - Neo Geo Pocket.rdb
-rw-rw-r--. 1 3.0M May 26 19:58 Sony - PlayStation 2.rdb
-rw-rw-r--. 1 197 May 26 19:58 Sony - PlayStation 3 (Downloadable).rdb
-rw-rw-r--. 1 75K May 26 19:58 Sony - PlayStation 3 (PSN).rdb
-rw-rw-r--. 1 1.2M May 26 19:58 Sony - PlayStation 3.rdb
-rw-rw-r--. 1 546K May 26 19:58 Sony - PlayStation Portable (PSN).rdb
-rw-rw-r--. 1 12K May 26 19:58 Sony - PlayStation Portable (PSX2PSP).rdb
-rw-rw-r--. 1 1.2M May 26 19:58 Sony - PlayStation Portable.rdb
-rw-rw-r--. 1 9.7K May 26 19:58 Sony - PlayStation Portable (UMD Music).rdb
-rw-rw-r--. 1 186K May 26 19:58 Sony - PlayStation Portable (UMD Video).rdb
-rw-rw-r--. 1 3.4M May 26 19:58 Sony - PlayStation.rdb
-rw-rw-r--. 1 65K May 26 19:58 Sony - PlayStation Vita.rdb
-rw-rw-r--. 1 227K May 26 19:58 The 3DO Company - 3DO.rdb
-rw-rw-r--. 1 217K May 26 19:58 Thomson - MOTO.rdb
-rw-rw-r--. 1 127K May 26 19:58 TIC-80.rdb
-rw-rw-r--. 1 6.0K May 26 19:58 Tiger - Game.com.rdb
-rw-rw-r--. 1 1.8K May 26 19:58 Tomb Raider.rdb
-rw-rw-r--. 1 8.1K May 26 19:58 Uzebox.rdb
-rw-rw-r--. 1 4.3K May 26 19:58 VTech - CreatiVision.rdb
-rw-rw-r--. 1 37K May 26 19:58 VTech - V.Smile.rdb
-rw-rw-r--. 1 14K May 26 19:58 Watara - Supervision.rdb
-rw-r--r--. 1 2.5K May 26 19:58 Wolfenstein 3D.rdb

This is also happening to me with RetroArch 1.8.7 on Arch Linux with all scan options except Manual Scan when selecting a specific core, that runs as expected (takes some time to start but scans all roms of that core very fast).

Same issue here with RetroArch 1.8.7 on Arch Linux on x64 system (on both computers intel core i7 and Ryzen 5). Downgrading to 1.8.6 make the import content scanner working. As soon as I upgrade to 1.8.7, the import content scanner is stuck at 0% and nothing happens.
Manual scan works on both versions.

Same problem on Manjaro. CPU at 100%.

Zero issue here on fda2355 or c2b8b3f1. Are you all using GCC 10? If so, that's likely the problem. GCC 10 has a couple massively breaking changes that have been causing headaches for a number of projects.

Mint 19.3 64-bit, GCC 9.3.0

I don't know the version of GCC used by Arch Linux to compile RetroArch. I just installed it from the official packet manager pacman.

It's unlikely to be GCC 10 in that case. Unfortunately I can't come up with anything specific, but if you manage any theories, I'm happy to check them out. My primary assumption is that it's either a permission or dependency issue, but it seems strange that it would crop up based on retroarch version in that case.

I think it is related with the access to the games databases. On an existing content library created from v 1.8.6, after 1.8.7 upgrade, when I click on the I to access the database information for a game, RetroArch crashes.
I don't think it is a permission issue because the game can be launched through the Load Content menu. For example, the User folder in the game folder is correctly created when using the Dolphin core. Also, I didn't touch any of the .rdb files between upgrade / downgrade. The owner is retroarch, group retroarch. Service is launched as retroarch, new created files owner are retroarch.
Something changed between 1.8.6 and 1.8.7.

I also completely removed retroarch and all configuration files then installed it again. Folder scanning sill doesn't work.

Scan is progressing when there is no game database. Though, as the database folder is empty, after scanning is done, no game is added in the library. As soon as I use the online updater to download the database, the folder scan does nothing and is stuck at 0%.

  • v 1.8.6 works
  • v 1.8.7 doesn't work
  • v 1.8.8 doesn't work

While I couldn't say -why- it would be important, try executing chown $USER *.rdb from the directory where your database files are being stored. That will change ownership to your user. It seems like it's halting on permission denial for the database files. I did verify that removing readability on the file will cause the exact behavior you're referring to.

EDIT: Could also just do chmod 644 *.rdb to give read perms to all. But I'd like to know what the perms on them are already.

No change here when chown to $USER, all .rdb files already have r permissions for everything.

Same for the directory I assume? Seems unlikely that'd be different though.

This problem just appeared for me as well though it appears a little different. With the ozone GUI, I get a progress bar that goes to 50%, then stops. For example, if I'm scanning a folder with 18 items in it, the progress bar stops at 18/36, and then uses 100% of a CPU core indefinitely. The program isn't frozen, and I can still move through the menu. I left it running for 15 minutes, and it never got any further.

I couldn't find anything in the logs or in stdout. Changing permissions on the database folder and files didn't help, unfortunately.

The first 18 are the archives you keep the games in. That's how it behaved when I set my Atari 2600 rdb to 000 and tried scanning. If you look, it caps out CPU usage on one core while that's happening.

chmod 755 ~/.config/retroarch/database
chmod 755 ~/.config/retroarch/database/rdb
chmod 644 ~/.config/retroarch/database/rdb/*

=> same result, scanning is stuck at 0%.

If I remove all *.rdb files, folder scanning is working (no games are added though).

In my test case, I try to scan a folder containing 3 GameCube roms. One game is .iso, the others are .gcz. If I put only the file Amstrad - CPC.rdb in ~/.config/retroarch/database/rdb, progression goes to 67% and is stuck analyzing the .iso file and never ends. No games are added in the library (it is normal, games are not in Amstrad CPC db).
Then I add the file Nintendo - GameCube.rdb in ~/.config/retroarch/database/rdb, if I launch folder scanning, it gets immediately stuck at 0% on the first .gcz it founds.

I think that Retroarch hangs as soon as it tries to match the game and the database entry. But I can't understand why.
Is it possible to try with other db formats, like .dat?

Also, I use Xorg, not wayland. I think the title of this issue is misleading, it seems to affect many Linux systems (Manjaro, Arch Linux and Fedora, at least), Xorg or Wayland.

I can confirm on Arch Linux as well, only manual scan works. The Arch Linux package is built with GCC 10, can't build with another GCC, and clang is a no go as well since compilation fails :/

I confirm the issue too.
"Scan Directory" option stucks at 0% on the first rom.
"Scan file" option has the same problem.
Only "Manual Scan" works.
Same behavior if the rom is compressed (zip or 7z) or uncompressed (tested with smc file - SNES rom)

Database has all permissions.
Using Xorg.

Arch Linux up to date (05-30-2020)
Retroarch version 1.8.8 installed with pacman from official repositories.
It was compiled with GCC 10.1.0 / 64-bit.
:/

Ah... I didn't think there was any way the AUR version would use GCC 10 so soon, when it's breaking every project in sight. I'm not sure if there's an issue open yet to handle GCC 10 fixes, but I'd bet good money this goes there if so. Frankly, building with GCC 10 at this point is a bad choice.

I think that @anderberin is saying that Arch Linux offical repos use gcc 10 to compile retroarch (and maybe all other softwares). AUR is a special repo where packages are compiled on the local machine. And I can say that my local gcc version is @10.1.0-2@. So, from the official repos or through a local compilation, the problem should be the same. I hope the retroarch project will support gcc 10.

Ah, my mistake. Seems my understanding of AUR is a bit lacking. Thanks for that info. Regardless, almost every large project right now is choking on GCC 10 in some way or other. Once my CPU's freed up I'll test a build myself, since I do have GCC 10 installed already for this very thing.

I think that @anderberin is saying that Arch Linux offical repos use gcc 10 to compile retroarch (and maybe all other softwares). AUR is a special repo where packages are compiled on the local machine. And I can say that my local gcc version is @10.1.0-2@. So, from the official repos or through a local compilation, the problem should be the same. I hope the retroarch project will support gcc 10.

  • Yes, I downloaded the binary from official repositories with pacman. And it was compiled with GCC 10.1.0.
    As Arch is a Rolling Release distro, GCC 10.1.0 is the current version to compile all new packages...

Don't know is this help.
I compiled and it is happening the same to me.
I compiled then in debug mode, ran and performed the import content by folder.
It threw me this in the terminal.

Archlinux gcc 10.1.0
Retroarch GIT version 0fedfa220f
file trace

EDIT: Now compiled again and it works. Apparently there is something strange with GCC 10
I downgraded its version to 9.3.0.1
https://archive.archlinux.org/packages/g/

Just built Retroarch with GCC 10.1.0, can confirm this is a GCC 10 issue specifically.
@osirase I recommend updating the title to accurately represent the issue at hand. Something like RDB files are unreadable when compiled with GCC 10.

Yep, can also confirm this issue when compiling retroarch-git from the AUR with GCC 10.1.0 installed. Scanning never gets past the first file, attempting to view the database entries in Information -> Database Manager -> whatever database file says "No Entries to Display". I only found out about this because Manjaro has recently started providing 1.8.8 through their repo with this issue hehe.

Is there a way to compile with an earlier GCC version in the meantime without removing GCC 10 from my system?

GCC 9 (and other versions) can coexist with GCC 10. Get that installed (not sure exactly what method is needed on Arch), then export CC=gcc-9 CXX=g++-9 and rebuilding should spit out a usable version.

I can't find a way to get GCC 9 installed so I opted for 8 instead. It compiles and the databases now work again. Thanks

Hello, Manjaro user here.
I downgraded to CCG 9.3 and scanning is still not working.

Given that you typed CCG rather than GCC, if you made the same typo in telling it which compiler to use, you likely compiled it with GCC 10 unknowingly. You can check Information > System Information > Compiler to verify.

I just made the typo here. I did GCC. I reinstalled RetroArch from Manjaros server so I don't think I rebuilt RetroArch correctly. I'm still working on it.

I just made the typo here. I did GCC. I reinstalled RetroArch from Manjaros server so I don't think I rebuilt RetroArch correctly. I'm still working on it.

I use Arch.
I downgraded GCC to 9.3 and downloaded the retroarch PKGBUILD file (https://projects.archlinux.org/svntogit/community.git/tree/trunk?h=packages/retroarch)
Then, I rebuilt it with makepkg... and installed the package with "pacman -U 'package name'... And it worked!. I don't know how is done in Manjaro...

You can check the GCC version used to compile with the command:

$ retroarch --version
RetroArch: Frontend for libretro -- v1.8.8 -- 9552f87157 --
Compiler: **GCC (9.3.0) 64-bit** Built: May 31 2020

or using the interface: Main menu > Information > System Information: "Compiler"

I do not now if this is related, but the content browser cannot detect rom or iso files anymore. It does not matter if the RetroArch is version 1.8.6, 1.8.7, 1.8.8, or built from master branch. The content browser just stopped showing iso .cue, .ccd files, only zip files are shown. Debian 10, all versions are built from source.

I start to suspect that what is broken primarily, are the database files downloaded from the Online updater. The code may have issues, but my v1.8.6 built was working before I cleaned up the ~/.config/retroarch directory contents.

Pull requests welcome. I am not on GCC 10 so I don't have anything substantive to say here.

To be completely clear, it's GCC 10 compiling retroarch that's an issue. I didn't recompile the RDBs using GCC 10, nor would that happen as part of the build process. The same RDBs that work when retroarch is compiled with GCC 9 don't work when I recompile retroarch with GCC 10. The RDBs are fine.

On Fedora 32 with GCC 10.1.1 the database browser only displays entries when retroarch is compiled in debug mode.

edit

  • -Og: works
  • -O1: works
  • -Os: works
  • -O2: broken
  • -O3: broken

Well, libretro-db can be compiled in standalone form - all the files are contained in libretro-db. Perhaps that is a better way of debugging these issues. If you run Makefile inside libretro-db, it will compile a couple of standalone executables that will allow you to read RDB files, do queries on them, and read cursors too.

On Fedora 32 with GCC 10.1.1 the database browser only displays entries when retroarch is compiled in debug mode.

edit

* `-Og`: works

* `-O1`: works

* `-Os`: works

* `-O2`: broken

* `-O3`: broken

Does compiling in debug mode negatively impact the performance of RetroArch, or is the impact only during the compilation itself?

It would negatively affect performance in RetroArch. I really would not recommend using it at all.

MeToo

As well as having the lists creation fail, it seems that the image updater is failing as well.
I hope someone can fix this issue soon, I deleted my lists earlier planning to recreate them, and now I can't. :(

I guess we have to use gdb to find where the code should be fixed...

This is not a bug in GCC optimization. There is an odd behavior in those optimizations but i think the underlying problem is a buffer overflow in RetroArch code.

GCC10 in -O3 and -O2 mode removes the following conditional statement
if (strncmp(header.magic_number, MAGIC_NUMBER, sizeof(MAGIC_NUMBER)) != 0)
in libretro-db/libretrodb.c.
The generated assembly code do the comparison and then directly jump to the error label regardless of the value.

The suspected bufferoverflow is in struct libretrodb_header declaration.
typedef struct libretrodb_header { char magic_number[sizeof(MAGIC_NUMBER)-1]; uint64_t metadata_offset; } libretrodb_header_t;

IMHO should be
typedef struct libretrodb_header { char magic_number[sizeof(MAGIC_NUMBER)]; uint64_t metadata_offset; } libretrodb_header_t;

The fix is really small but i can do PR if you prefer.

I have compiled retroarch with the suggested changes by @theowhy and both the a) Database information works and b) Scan folder/file works as expected.

gcc version 10.1.0 (GCC

[INFO] RetroArch 1.8.8 (Git 439aee616e)
[INFO] === Build =======================================
[INFO] CPU Model Name: Intel(R) Celeron(R) N4000 CPU @ 1.10GHz
[INFO] Capabilities: MMX MMXEXT SSE SSE2 SSE3 SSSE3 SSE4 SSE4.2 AES
[INFO] Built: Jun 14 2020
[INFO] Version: 1.8.8
[INFO] Git: 439aee616e

diff --git a/libretro-db/libretrodb.c b/libretro-db/libretrodb.c
index a82e2399b1..4976bdcbf2 100644
--- a/libretro-db/libretrodb.c
+++ b/libretro-db/libretrodb.c
@@ -75,7 +75,7 @@ typedef struct libretrodb_metadata

 typedef struct libretrodb_header
 {
-  char magic_number[sizeof(MAGIC_NUMBER)-1];
+  char magic_number[sizeof(MAGIC_NUMBER)];
   uint64_t metadata_offset;
 } libretrodb_header_t;

Hey that's awesome news!

We'll push the change through then.

EDIT: Pushed.

So this has been in master now since 12 hours ago, can the people in this thread that were experiencing issues re-test this issue so we can see if it is resolved? Thanks.

While not specifically on Arch, I did reproduce the issue locally and have confirmed that I can now build with GCC 10 without the issue rearing its head.

That's cool to hear. What about @BugReportering ?

Fedora 32, GCC 10.1.1, it's working now. Thank you!

It is working on Arch now.
GCC 10.1.0

The package on official Arch repository was updated.
It's working now.

alright, seems to be resolved, so I'll close this issue.

Big thanks to @theowhy for tracking down the issue and identifying the fix!

Fixed! @theowhy are you able to link to the pull request that fixed this issue?

Looks like it was this commit: 685948368e6b28b97e1b91bdfe5921519359a033

I didn't do any pull request, the fix was directly made by @twinaphex in the commit pointed by @Sanaki.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

GoronMegaZord picture GoronMegaZord  路  3Comments

charlydelta picture charlydelta  路  3Comments

blackman91 picture blackman91  路  3Comments

hyarsan picture hyarsan  路  4Comments

fr500 picture fr500  路  4Comments