Retroarch: Automatic Attract Mode

[magwe]

As RetroArch is moving into front end territory with individual game art etc I see a unique opportunity to go above and beyond what has been done before.

The most attractive front ends have not only static images but manually captured video files from individual games.

Most arcade games and many console games provide an "attract mode" that showcases the game when nobody is playing. If the core/rom was launched behind the front end as soon as a rom was in focus during game selection the output could be displayed as a non-interactive preview window. Input would still be routed to the front end at this point. Once a game is selected the front end could just zoom into the preview window and hand over input control to the core which would take over from there.

There would have to be some mechanism for the preview to quickly get to the point where the actual Attract Mode sequence starts. One way would be to speed up emulation until that point. A quicker way would be to have a special save state for when the Attract Mode sequence starts.

Games are usually sorted by system so there wouldn't be a need to reload a core as the user browses a collection. In most cases the same core would just reload a new rom.

This would provide a huge increase in visual attractiveness and usability without the need to download any media files related to individual roms. At most there would be a database of when attract mode starts for different games but just using best guesses would get us far. Any save states could be generated on the users machine.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[fr500]

And a lot of complexity!
Basically a small libretro frontend within the menu driver.

Or a ton of hacks around the current core implementation to make it render to a small texture in the menu driver.

[Alcaro]

Or maybe a big libretro frontend? We already have that... and not making another one would make "Once a game is selected the front end could just zoom into the preview window" a lot easier to implement.

Either way, running games in the background isn't realistic. Some cores take half a second to load a ROM. Some cores leak memory. Some cores don't support savestates. Some cores don't go above 70fps, so fastforwarding won't help. Some games don't have an attract mode (mostly games for older consoles). I think there's even one core that can't load two different ROMs without restarting.

It's an interesting idea, but video files are the only plausible implementation. (Though I don't think we have that either, which is a valid issue.)

[lasers]

I don't know about making videos. Making & editing videos could take a while. Some games can take more than two minutes to make a complete loop. Knowing this now, do we want to play them at normal speed or faster? 3x speed? Some games are awful slow. Each game likely will require several tweaks to make it catchy... and there are over 10,000 games.

Here are some things we would have to do with videos to keep them exciting.
Not to mention that videos easily will take up more spaces than both games and thumbnails.

• Eliminate information/copyright/other at beginning?
• Eliminate long start screen duration?
• Eliminate or speed up slowing scrolling text?
• Doublespeed things? Some games can be slow so we speed it up inadvertently making it too fast for other games.
• Remove duplicate frames and other not-listed-yet things?
• Remove blackframes?
• Record everything as-is? Perfect loop? Go for core instead.

Do we optimize things? Drop frames? Etc to make size smaller, but that could make some games look unattractive.

Also, the attention span may be too short to enjoy each video as you scroll down.. especially if they're going somewhat fast or not original because it had been customized to hell... to make it catchy... so it might be hard to pay attention to them. Also, downloading lot of little clips on a chance that it would catch our attention... make it seems like we're really not trying to use RetroArch to play games at all.

Now with cores...

If we're making start points for attract mode to load in the beginning, we might want to make one at end of the loop to prevent some things I mentioned above... or just let it loop?

What do we do with the hacked/homebrew roms? It seems like it's better to go with core because if it can be loaded with retroarch, then it can be previewed without having getting team involved for trivial tasks like this. New games means new thumbnails (and now new videos!!!). Making multiple small 256x256 videos seems like an awful chore so I made one.

Random NES Pick: 257K
Clip contains 1 loop in 256x256
Clip contains 00:03:07.433. (3 minutes!)

55M - ZN_60_ultrafast.mkv
25M - ZN_60_veryslow.mkv
6.3M - ZN_60_veryslow+blackframe.mkv
4.2M - ZN_60_veryslow+doublespeed.mkv
4.0M - ZN_60_veryslow+doublespeed+blackframe.mkv
3.3M - ZN_30_veryslow+doublespeed+blackframe.mkv
3.0M - ZN_25_veryslow+doublespeed+blackframe.mkv
2.2M - ZN_12_veryslow+doublespeed+blackframe.mkv

With doublespeed, the clip contains 00:01:33.750. (Half)

This is to give you some idea how filesizes may vary. I don't know. There are some more optimizations I could have done, but didn't... like changing blackframe percent from 98% to 90%. That might remove some in the beginning and maybe some before a long+slow text scrolling. Lot of times went into this already. I should really played that game instead. 😄

[dankcushions]

attract videos for emulated games have been extensively covered by others. eg http://emumovies.com/

playing such clips is all retroarch would ever need to do. no need to reinvent the wheel here.

[lasers]

Really cool. I went to see that game I made clips from. There is only 3 games under NES category on that site http://emumovies.com/files/category/1358-videos/ so it does not have my game / clip. It might be cool to display manuals right next to boxarts though.

[dankcushions]

Really cool. I went to see that game I made clips from. There is only 3 games under NES category on that site http://emumovies.com/files/category/1358-videos/ so it does not have my game / clip.

those are packs, not individual games.

eg http://emumovies.com/files/file/1724-nes-video-snap-pack-sq-240p/
contains 900 videos.

[lasers]

Oh sorry. Thank you for pointing it out.

[lasers]

Grid Mockup. Awesome if we could go with same banner size as Steam's... and an option to filter language. USA, Europe, Japan, German, Etc, All

[jedinjapan]

As others have stated here, running a live preview in the background when a user pauses on an item sounds nice, but would be prohibitively difficult. Even if drawing over the core's output and overlaying the menu was easy (I expect it's anything but), you would have the issue that scrolling through a list of content would be loading and dropping cores left and right as it attempted to run them in attract mode live in the background and overlay ontop of it. This would be a magnitude more complex than an other system's proposed attract mode.

If you wanted to achieve a similar effect to what you're suggesting, with far less overhead (but still quite a bit of work) you could have retroarch generate a "first 30 seconds" of save state files upon demand in a special folder, allowing it to "load state" at the point matching the attract video that matches when the user selects it. This would only work on cores that support save states however, and would take a lot of drive space.

While the effect would be cool admittedly, it would require a lot of save states per file to be attractive and "smooth", massively ballooning disk usage for the files it is enabled for, and wouldn't work on cores without save states at all, which would just load to a black screen like normal. And all of this is ontop of the fact that there is no existing attract mode at the moment.

While storing videos for attract mode might cost drive space and time, having them be pre-rendered reduces the complexity of the idea immensely. You could have multiple variants link to one attract video if you wanted to save space (the EU and US variants having the same intro screen for example).

As for the mockups, you look like you would be wanting something like emulationstation, HyperSpin, or another front end that's very media-rich in that case, not quite Retroarch. Perhaps the upcoming "zarch" menu driver will be enough for you, but I don't foresee any of the menus ever looking like some of the heavily skinned stuff seen in other frontends. Zarch is not finished yet from what I know, but looks similar to PPSSPP in terms of interface design.

[magwe]

Just a few clarifications.

The mock ups we not made by me (the OP).

Regarding "dropping cores in and out", that would just be true for a mixed sublibrary. Most people organize their games by system so the same core would just reload a rom and save state as you move up and down that sub library.

I am not sure what you suggest regarding "first 30 sec of save state files".. If you mean having one save state per game that starts at attract mode then that is the same as I proposed in my OP. But it sounds like you propose using multiple save states per game as frames in an animation? In that case I see no reason to just use regular video files.

[jedinjapan]

Of course. I didn't meant to imply you made them. I should have directed the comments more clearly. No offense meant. Regardless I expect (I am not a dev) that Zarch will be the closest Retroarch comes to such an interface for the near future. But that's just my read on the situation.

Regarding "dropping cores in and out", that would just be true for a mixed sublibrary. Most people organize their games by system so the same core would just reload a rom and save state as you move up and down that sub library.

To actually load a core and render in realtime requires loading different executables on some systems. I know that the Vita and PS3 for instance have a "soft reboot" everytime the core changes. This would cause serious problems for these cases. But even disregarding edge cases, it is very processor intensive to do this, and save state loading is not instant on many cores. Even classic cores take a noticeable half second to load often. The UI would likely be very sluggish to navigate, as you scroll over a dozen PSX titles (much bigger save states than SNES), all rapidly cycling in and out, then into N64, then SNES, and suddenly your system freezes from a memory leak somewhere amongst the 6 cores and 20 pieces of content you just loaded within 15 seconds.

I am not sure what you suggest regarding "first 30 sec of save state files".. If you mean having one save state per game that starts at attract mode then that is the same as I proposed in my OP. But it sounds like you propose using multiple save states per game as frames in an animation? In that case I see no reason to just use regular video files.

For many weak systems like a raspberry pi, constantly loading and emulating in the background, for different cores, could be very difficult as I was saying above. Even simpler systems would likely be a strain. The suggestion I made (though still hardly practical at all I think) was to create a "buffer" of pre-made savestates roughly timestamped to perhaps every 1 to 2 seconds or so of the first 30 seconds of the gameplay. Akin to the rewind function, but permanently saved to disk. Then, if Retroarch did in fact play an attract mode pre-recorded video, pressing X could cross reference the attract mode video's timestamp with the list of "start"states, and load the appropriate moment to "resume" from within the first 30 seconds or so.

Again, this is still a huge undertaking to contemplate, but this approach compared with fully loading every core and piece of content instantly would reduced the CPU load and possible memory leaks, by allowing weaker systems to not continually load content and cores in the background, and just play a attract mode video like most frontends. The UI could remain responsive, while also allowing the "zoom in" feature that was suggested. Personally I find it an impractical idea to implement, but I think this is at least something possibly attainable.

If rapid-fire core swapping and content loading weren't such a monumentally difficult and processor straining task, I'm sure that other frontends would have considered this. Not to say it cannot be done ever, but that it seems to be at odds with the project aims of trying to be usable across as many platforms and systems as possible.