Retroarch: Translations

[alfrix]

The current scheme of translatable files is very difficult to maintain, i wanted to move everything to transifex so it's easier and faster to collaborate, also when the source is updated, the languages are automatically updated with a simple command, but the current file format isn't supported,
Shouldn't a more common language be used?
Does anyone have any plans/thoughts on the matter?
@twinaphex @RobLoach @bparker06 @Alcaro @fr500

[Alcaro]

My main thought on translations is "learn english", which is probably not the project's official stance.

Yes, the current system sucks, but (under our fairly unusual requirements) I'm not convinced anything else is better. Most other systems we looked at put the translations in external files, require some heavyweight support library that doesn't run on WiiU, or both. (Though I'll admit I'm not sure why external files are bad, given that we royally screw up without the assets package. Maybe the assets didn't exist when we created this scheme.)

Transifex seems fairly expensive, too, starting at $139 per month (or a 15 day trial). Did you find something I didn't (like free service to open source projects), are you willing to pay that amount, or should we pick some other option?

I'm fine with anything the rest of the team agrees with.

[MarcoEstevez]

I think same way Alcaro does. I work on a CRM solution owned by Oracle, what they do is have 2 tables on a database, one containing the standard or default string value (usually english) then the second table containing child records, basically a string and language code, if you add some new labels on the GUI you have to create just the header (the english value), the application then finds any child that matches laguage specified by the user, if does not exist (because nobody created it) then shows the default one.

The good thing here: is that it is super easy to maintain, you just need to create the parent record, then rest of the people might want to add child rows for the desired language code or update an existing one or do not create it at all.

EDIT: Since is a SQL table join, checking what things are missing or which ones need to be updated, or even compare between languages are quite simple.

EDIT2: Once all is stored in a relational database, a simple script could later genearte dinamically all files Retroarch needs before compiling it.

[alfrix]

"learn english"

Not very friendly at all.

free service to open source projects

From their website:

What are the basic requirements to qualify as an Open Source account?

We are proud and humbled to be hosting the largest Open Source translation community in the world. We offer Transifex for free to Open Source projects that have no funding, revenue, and/or commercialization model. If you have a question whether your organization qualifies for our Open Source program, please contact us.

you just need to create the parent record

yeah, that's kind of the point, plus no need for the translator to learn git or have to do pull requests everytime someone updates something, just do an automated pull before building

[orbea]

Requiring proprietary software for RetroArch translations would be extremely lame. Please do not suggest something so wrong.

[RobLoach]

We could switch to something like the .po file format with gettext. There are a few open source tools out there to assist in managing them.

[bparker06]

We offer Transifex for free to Open Source projects that have no funding, revenue, and/or commercialization model

This means RA is disqualified

[bparker06]

@RobLoach I originally pushed for gettext when we were overhauling the translations but the current header format ultimately won, because it doesn't require any external dependencies. The downside is that it's apparently hard to get people to edit text files properly, and our UTF-8 BOM requirement for MSVC means they break left and right anyways, most likely no matter what tool is used.

[hyarsan]

I highly doubt all 7 billion+ people on this Earth know English haha

[RobLoach]

the current header format ultimately won, because it doesn't require any external dependencies

We could implement our own parser, removing the need for any external dependencies, and possibly working around UTF-8 BOM. The file format looks pretty easy to parse.

[alfrix]

Requiring proprietary software for RetroArch translations would be extremely lame. Please do not suggest something so wrong.

Please don't be so harsh, Transifex was only a suggestion it's not the only thing we can use.
What about Pootle (GPLv3) or Zanata (LGPL)
but ultimately, the format needs to be supported

[simias]

On the subject of proprietary software: as long as the translation format is open (like gettext for instance) it doesn't really matter what platform is used for the actual translation.

Gettext is very widely supported.

[Alcaro]

I highly doubt all 7 billion+ people on this Earth know English haha

Yeah, I worded that one pretty badly. My opinion is a minority around here, and I have no intention to push it. The translations exist, there's no reason to change that.

Please don't be so harsh, Transifex was only a suggestion it's not the only thing we can use. What about Pootle (GPLv3) or Zanata (LGPL)

You're asking for two things at once: Getting the translations out of those .c files and into a slightly less homebrewed format, and getting the translations out of the RetroArch repo. Neither task is trivial, and (from my point of view) neither is an obvious must-do.

For now, I vote add gettext support and cut those hashes, but put the translation files in the assets repo. This gets rid of BOM/codepage shenanigans and lets people test their translations without installing a compiler, which I believe are the current main issues. We can experiment with putting them on an external service once that's in place.

[hiddenasbestos]

It would be useful to consider localisation of core strings while you're at it (core options, input device buttons, on-screen notifications, etc.)

I guess that will amount to adding whatever localisation solution is agreed upon to libretro-common, rather than RetroArch itself?

If that's done then there doesn't need to be much in the way of changing the core interface beyond that. The FE can tell the core the user wants language X and then the core can either give text back in that language or just send back fixed English strings like it does now. The FE wouldn't need to do any further translation

Letting the core make use of a shared pool of compatible tools and techniques will make it easier for human translators to do the work - using a consistent file format / data structure / editor.

[orbea]

Please don't be so harsh, Transifex was only a suggestion it's not the only thing we can use.

Bad suggestions should be called out, if that is being harsh then so be it. As I do not use translations my only concerns are:

1. That it does not rely on proprietary software.
2. That it does not introduce additional/new dependencies.

[ramiro]

Duplicate of #4679 Or perhaps the other one could be closed in favor of this as it has seen no discussion?

[simias]

One should close the other one since it doesn't have any comments obviously.

As for translations, personally I think gettext/poe is the way to go, I'm sure the community is large enough for people to volunteer translations. I'm not sure what transifex would add. It's not like there's a mountain of text in Retroarch.

One challenge however would be localizing the strings in cores as @hiddenasbestos mentioned. One could pair each core with a translation file but it sounds clumsy. Ideally you'd want them embedded in the dynamic library but I'm not sure what's the best way to do that.

[lotharsm]

I’d love to see the retroarch translations to be handled by gettext. The current method is quite annoying if you basically have to compare files manually when a bigger update comes in. Having the possibility to do this with a simple .po file would be a great enhancement in my opinion.

I don't think retroarch needs to use Transifex, just a simpler way like gettext/po to update the translation files.

[Hedda]

I submitted request #4679 for language translations via Transifex, and my suggestion is to replicate the same GNU gettext toolset concept with external PO-file for each language same as the Kodi / XBMC project.

https://kodi.wiki/view/Translation_System

Strings are stored in on .po file format for each language with English (UK) as the primary language.

https://www.transifex.com/projects/p/kodi-main/

PO-files are more or less standard for GNU licensed code

https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html

Main benefit with is that it is a community of its own and their interface make it easy for non-developers to help with translations as it is all web-based and similar to a wiki edit from translators point-of-view.

[keithbowes]

Indeed. The current system puts far too much a burden on translators, who may not even be familiar enough with the C programming language to work with the translations. One thing to consider is that projects with their own ad hoc translation system often provide scripts to convert from and to gettext, so even if the current system is kept intact, maybe in an upcoming version (perhaps 1.8 or 2.0), such scripts could be used.

Even if Transifex isn't used, gettext is still much more user-friendly due to tools like PoEdit.

[guoyunhe]

I think every translator is screwed up by the current translation solution.

A simple example:

You have a key MENU_ENUM_SUBLABEL_AI_SERVICE_MODE. Its value was initially "Pauses gameplay during translation (Image mode), or continues to run (Speech mode)". But later it is changed to something else: "Show translation as a text overlay (Image mode), or play as Text-To-Speech (Speech mode)". But translators don't know. They have to go through thousands of lines of texts to identify every changes for these updates. Well, we have never found this kind of issues in other open source projects.

FOSS translation system & NO extra dependencies

Here are open source and free of charge platforms: Weblate you can use cloud instance or host your own instance. Both are free of charge for open source projects.

If you don't want to introduce dependencies, use JSON format, the same format RetroArch uses for playlists. It is well supported by most translation platforms and various library implementation.

[guoyunhe]

With #9513 , RetroArch will be able to integrate any translation collaboration systems, without adding any dependencies. JSON format will be converted back to *.h files after translation. Would be great if maintainers can give it a try.