Vorta: i18n: translation, localisation (overview)

[ThomasWaldmann]

recently showed borg at the local linux user group, it was well received, some people prefer gui to cli tools, so vorta is just what they waited for.

but, first question of one of the guys: "gibt's das auch auf Deutsch?" ("is this also available in German?") and i had to tell "no, not yet".

so, while i18n for borg is rather optional, because UNIX admins hacking stuff on the cli usually also speak English rather well, it's a different story for the target audience of vorta - we need i18n for them.

so, what needs to be done:

• (this ticket) code needs to prepared for i18n, qt style:

• vorta.i18n package with some i18n infrastructure code
• self.tr('english phrase') (scope comes from instance class name)
• app.translate('scope', 'english phrase')
  
  
  
  • #135 packaging
  
  
  • #136 check original texts
  
  
  • #137 infrastructure for translators, integration
  
  
  • #138 i18n policy, see also #140
  
  
  • #139 layout short vs. long texts / rtl languages
  
  
  • #141 localisation
  
  
  • UI needs to be as self-explaining as possible (because I guess we do not want to translate the docs also).
  
  
  • I'ld advise to only add 1 language at first, get everything implemented / tested for that and only after that add more languages.
  
  

[m3nu]

Feared this would come up eventually. I know the process from large-scale website translations, I've organized in the past. Was always good fun.

I'll see if there are any blockers, like the Qt UI-files. Then start marking and extracting strings for translation.

[brokenpip3]

Sign me for Italian translation :+1:

[genma]

Count me in for French translation :+1:

[m3nu]

Thanks, already noted. Hehe.

[ThomasWaldmann]

I'll try doing some work on this. No experience yet with any qt-specific stuff related to this (if any).

[ThomasWaldmann]

First slight issue I've found is that py36+ f-strings are not really compatible with using a translation function, like _(f'foo: {foo}') - the func would get the already expanded string.

https://docs.djangoproject.com/en/2.1/topics/i18n/translation/ : it looks like only the %(foo)s formatting placeholder style is supported.

So:

• get rid of all f-strings in translatable text?
• get rid of all f-strings? this would maybe allow to support py35.

[ThomasWaldmann]

https://doc.qt.io/qt-5/internationalization.html

[ThomasWaldmann]

hmm, slightly confused.

is qt doing its whole own thing concerning translations and not integrating into the usual gettext-style .po files?

how to deal with all the texts in the .ui files? is the extracted text converted to .po?
if yes, how?
if not, will there be .po files for the python strings and .ts files for the qt stuff?

[m3nu]

The QT C++ docs don't always fully apply to PyQt. For translations, the workflow seems roughly like this:

1. Extract translatable strings into .ts files using the lupdate command. Then compile them to .qm files using the lrelease command.[[1](http://doc.qt.io/qt-5/linguist-manager.html#using-lupdate)]
2. Then use the QTranslator class to load .qm files from for a specific language from a path: translator.load("qt_ru", PATH)
3. Add the QTranslator instance to the app: app.installTranslator(translator)

I didn't test this yet, but found some examples on SO.[[2](https://stackoverflow.com/questions/13032774/pyqt-english-to-finnish-translation), [3](https://stackoverflow.com/questions/19032805/pyqt-multiple-languages-qt-designer-auto-generated-ui)]

This takes care of the Qt UI files. for strings in ordinary Python files, the workflow will be slightly different. Hope we can still use a common format to have all translations in one place. Regarding f-strings, they may be complementary. I.e. first pull the translated string. Then fill the data by evaluating the translated string as f-string. There could also be a helper-function that does both – takes an English string, translates it if needed and fills in some data.

[ThomasWaldmann]

AFAIK, f-string expansion by the py36+ interpreter is only done on literal f-strings and only when they are first evaluated. While researching f-strings, i found https://pypi.org/project/fstring - not sure whether it's worth adding this as requirement or we rather should just transform the f-strings and use % expansion - you decide.

About having the translatable strings at 2 places (qt-style and gettext-style): that's a bit embarrassing, but I think this problem maybe has been solved already in another pyqt project that is translated.

[m3nu]

About having the translatable strings at 2 places (qt-style and gettext-style): that's a bit embarrassing, but I think this problem maybe has been solved already in another pyqt project that is translated.

Gettext isn't needed. The said QTranslator class can also translate ordinary strings like this:

tr("Host {0} found").format(hostName)) - from here and also described here.

An official: example. Note the self.tr('some string')

Here is also someone who mixed gettext and QTranslator. Not the greatest example and using QTranslator only is probably the way to go.

[ThomasWaldmann]

Quick update:

• Have extraction from .ui and .py working: pylupdate5 (works for .py and .ui), lrelease and I did not need to use gettext style yet, qt style stuff seems to be sufficient for everything.
• Got QTranslator working now. There's obviously a refcounting bug in pyqt, but if one keeps a reference, it works. :-(
• moved from f'foo: {foo}' to self.tr('foo: %s') % foo (for very simple cases, for more complex stuff, there must be named placeholders for better clarity and for the translator being able to change order.

PR #134

[ThomasWaldmann]

Another update:

There are cases one needs "late translation", meaning:

• being able to mark some context/string at the place where it is defined, so it can be extracted from there by pylupdate5, but not immediately translate it - i added vorta.i18n.trans_late for that.
• actually translate this stuff right before it goes to the UI, i added vorta.i18n.translate for that (as a thin wrapper around the qt app method)

Currently, this is mostly used for the messages returned by misc. actions in msg['message'] - because the returned original english message shall be written to the logger and a translated message shall be output on the UI. I used the scope messages for these translations.

[m3nu]

It may be better to use the non-country translation tag for simplicity? E.g. just 'de' instead of 'de_DE'? I doubt there will be a 'de_CH', etc.

[ThomasWaldmann]

yes, for de that is rather unlikely.

but for cn, there is a difference between taiwan and mainland china.
also, there are differences for en_GB and en_US.

so, considering the search behaviour of QT, guess I should rename to de (so it will get found starting from de_DE, de_AT, de_CH) but give cn_TW in some comment at least.

[m3nu]

Yepp. Would do it like that.

[ThomasWaldmann]

Just noticed that we need to specify whether the original strings should be en_US or en_GB.

[m3nu]

en_US is more common, I guess. But I don't see any specific existing strings. Like color, organization, etc.

[lapineige]

Count me in for French translation 👍

I'm also interested 😃

[m3nu]

Thanks for the interest, @lapineige . I added you to the official translation thread. See the details there.

[jose1711]

Hi, please add me to the thread. I'd like to work on translation to Slovak. Thanks!

[m3nu]

My pleasure, @jose1711. I've added a new discussion for Slovak here. Please see my response and next steps there: https://github.com/borgbase/vorta/discussions/978