Retroarch: Peeking to get playlist format at head of file creates incompatibility with other JSON writers.

Created on 28 Feb 2019 · 12Comments · Source: libretro/RetroArch

https://github.com/libretro/RetroArch/blob/2ab333e00f116a089f884630dd45a1719054e467/playlist.c#L1205
I'm trying to make a playlist generator and I noticed that this should only really be peeking the first byte to see if it's a "{" instead of looking for the full word of "version" as well when checking if the lpl file is in the old or new format. the playlist generator I've made will put "version" at the bottom of the file instead of at the top, making RetroArch load my json playlist in the old format instead.

minor duplicate

Source

ZornTaov

Most helpful comment

But that's... not how JSON works. JSON files don't care if the order of variables per object are jumbled up, it'll just load them as long as the hierarchy is correct and variables aren't missing (like if I removed "version" entirely, or forgot a "core_path" for a single entry). In fact after looking through the code RA uses it should do just that, but that line linked prevents me from testing without recompiling for myself.

ZornTaov on 2 Mar 2019

👍2

All 12 comments

Then you are making your playlists wrong and you should update your own code instead.
I also have my own scripts making correct playlists in the new format and it is not that difficult.

Their code is prepared for checking what version of json-playlist format for the future.

SwedishGojira on 1 Mar 2019

ZornTaov on 2 Mar 2019

👍2

@bparker06 Maybe you may know how to solve this?

orbea on 2 Mar 2019

While it wasn't initially designed with the interoperability of third-party applications in mind... if you change it to just scan for a { it would probably be good enough. However the spec says:

Insignificant whitespace is allowed before or after any token. Whitespace is any sequence of one or more of
the following code points: character tabulation (U+0009), line feed (U+000A), carriage return (U+000D), and
space (U+0020).

So it's not enough to just check that the very first character is a { or not, there could be any number of the above characters first (or in some cases a UTF-8 BOM as well).

bparker06 on 4 Mar 2019

By that logic it would also break the existing check, as there could be any number of whitespace characters before that first opening "{" if a user decides to modify their playlists. If there's access to regex you could do a \w*({) and that would be enough to scan for the opening "{" from, say, the first 32 bytes. If the regex matches then the file is a json file, otherwise it's in the old format. And if you're worried about UTF-8 BOM, then the regex would be [\s\xef\xbb\xbf]*({) to include that BOM in the search through the whitespace.
Alternatively, (even though this would be a LOT slower in some cases just for this check), parse the entire file anyways as if it's a json file in a try/catch, and if throws an exception (because it's not json, or malformed in some way) then load it in the old format.

ZornTaov on 4 Mar 2019

Please note that playlists that contain both the old format and the new json format are not uncommon, it happens to my history list every time I bisect...

orbea on 4 Mar 2019

Please note that playlists that contain both the old format and the new json format are not uncommon, it happens to my history list every time I bisect...

wouldn't that... completely break what RA shows for that playlist?

ZornTaov on 4 Mar 2019

wouldn't that... completely break what RA shows for that playlist?

Actually no, if I am using the newer json playlist format any entries matching the old format will appear as garbage, but new entries more or less work. The reverse is true when using older commits without the new json format.

orbea on 4 Mar 2019

Actually no, ...

Actually yes, as that's unintended behavior. you won't be able to load any of the malformed entries that way. Thus "broken".

The reverse is true when using older commits without the new json format.

that's because there was no "forwards compatability" done (how could there be) when the json format was added to 1.7.6 and people still had some sort of -1.7.5 version installed (this is definitely a problem on switch, where each core had it's own version of RA frontend packed into it and people hadn't updated their cores yet)

ZornTaov on 5 Mar 2019