Group-income-simple: Finish i18n translation/localization toolchain — fetch JSON and update UI

[taoeffect]

Problem

Our localization toolchain is almost finished, and the final missing piece is for the frontend to fetch the proper .json file from the server for the user's local language and update the UI.

This will require updates to both the backend and frontend code.

Solution

Backend

Be able to serve the translated .json file that the client requests.

Frontend

For now as an alpha version we will do this incorrectly and inefficiently, and in a future update we will do it properly.

So a PR for this issue will simply fetch the json file by using either navigator.language or navigator.languages[0] or navigator.languages[1] (do thorough research on this and find out what the most correct choice is!). It will then save this translation into memory and do the mapping from original english => new language inside of translations.js.

A future issue will be opened to make this more efficient by versioning the translations file and caching it locally in IndexedDB, but for now this simple method should get us up and running.

[mmbotelho]

Question - how will we decide which language to display to each user by default?

Note: There should be a setting somewhere where users can change the language.

[taoeffect]

Question - how will we decide which language to display to each user by default?

This is decided by the user's system, which affects the values of those variables I mentioned in the issue.

[mmbotelho]

Can you give me a bit more detail on that @taoeffect ? Is it the system language, the browser language, the location language...?

[taoeffect]

The system language, which might or might not be overridable by the browser, but ultimately we're going to go off of the values of those variables, however they're set.

[snowteamer]

The value of navigator.language can be split into a country code and a culture code, but currently we don't have culture-specific strings file so I guess that only the language part will be used

[taoeffect]

The value of navigator.language can be split into a country code and a culture code, but currently we don't have culture-specific strings file so I guess that only the language part will be used

That's right, whatever translation is available is what will be used. If it turns out that culture-codes are necessary then we will implement it in a way that looks up first the specific translation, and then the more general translation. I think to keep things simple we'll start out by only using the first part (e.g. en, not en-US).

[snowteamer]

Also, please note that currently, the files french.strings and english.strings have a few different keys.

This is because I thought it was a good idea to fix a few copy errors while translating, but now I realize that it's ineffective without also patching the corresponding strings in the .vue files before re-running the strings tool.

Otherwise if not patched, the translation of these English strings which no longer have an exact match in the French file will likely fail at runtime.

Namely, the following strings:

• 'The modal base as very basic functionality. Compare to normal modal it has very basic style and let us add any kind of content.',
• 'Welcome to group income',
• 'Record 1 payment',
• 'Failed to propose remove {member}: {reportError}',
• 'For now, the only way to join a group is to get an invitation. Don’t know anyone using Group Income?'

which I changed to:

• 'Welcome to Group Income',
• 'The modal base has very basic functionality. Compared to a normal modal it has very basic style and let us add any kind of content.',
• 'Record a single payment',
• 'Failed to propose to remove {member}: {reportError}',
• 'For now, the only way to join a group is to get an invitation. Don’t know anyone using GroupIncome?',
  `

[taoeffect]

@snowteamer Feel free to create an issue to update any strings needed in the .strings files... although there's two things to consider:

1. The strings found in the .vue files should never be "patched" as they are the canonical defacto correct strings to-be-translated and supercede whatever is found in the .strings and .json files.
2. The strings executable/tool/utility will keep those in sync, so simply running it before submitting a PR should remove any old translations, and insert any new needed key/value pairs in to the .strings files, as well as adding comments to indicate that they're "MISSING TRANSLATION"

[snowteamer]

@taoeffect That's precisely why those French strings are going to be out of sync after we run the string extractor.

So we can either:

• Revert the English keys in french.json to match the canonical strings,despite the small copy errors;
• Or fix the copy errors in the canonical strings so that the changes will propagate to the json files upon running the strings tool.

[snowteamer]

Regarding the fetching of the strings files:

I think these files are very much like assets. Why not put them in the frontend/assets folder?
This would likely be safer than fetching files from the backend root on behalf of the client

[taoeffect]

Revert the English keys in french.json to match the canonical strings,despite the small copy errors;

My understanding is the .json files must never be hand-modified, as doing so has no effect. Those files are derived from .strings files.

Or fix the copy errors in the canonical strings so that the changes will propagate to the json files upon running the strings tool.

Ah I see what you're saying. I admit I read your comment too quickly and didn't see that you had fixed typos in the original strings. You're right in that case, those are things that must be fixed in the UI itself and in the .vue files. That's an issue that should be opened that is separate from fixing the French strings (an issue I encourage you to open!). I've gone ahead and opened the issue for fixing the copy: #962

I think these files are very much like assets. Why not put them in the frontend/assets folder?

You are correct that these are assets, and the reason we're not putting them there is because the entire assets folder is copied into the dist/ folder on build, and we want only the .json to be copied, so that should be done as a special copy task in the .Gruntfile.babel.js file.

[snowteamer]

Regarding what @mmbotelho suggested,

I think it's possible to implement something like a state/setLanguage or state/setLocale selector:

while for now this selector would only be called once, with a value extracted from navigator.language, this would leave the door open for future addition of a language switching feature.

[taoeffect]

Sure, that's a possible way of going about it. 👍

For simplicity's sake, in the initial implementation the translations.js file can just directly access navigator.language and that should cover the vast majority of use-cases.

[snowteamer]

@taoeffect Altering the Grunt file sounds a bit complicated (at least for me), so I won't argue any longer over this point for now 😄

[taoeffect]

It's pretty simple, just involves adding a new section here:

    copy: {
      node_modules: {
        cwd: 'node_modules',
        src: ['systemjs/dist/*.{js,map}'],
        dest: distJS,
        expand: true,
        flatten: true
      },
      html_files: {
        src: 'frontend/index.html',
        dest: `${distDir}/index.html`
      },
      assets: {
        cwd: 'frontend/assets',
        src: ['**/*', '!style/**', '!svgs/**'],
        dest: distAssets,
        expand: true
      }
    },

[snowteamer]

Thank you for the hint!

After a bit more research, it appears that some i18n strings are currently computed by the L() function before the async startApp() function in main.js is called.

This is an issue since any required .json language file has to be async fetched before the L() function can use it,

These cases include page titles in router.js, and a few strings in some of the Vue components the router imports, like the DesignSystem page.

Since imports in main.js have to stay at the top of the file, and top-level await is not yet supported, I don't see any way to insert an await expression before the first L() call.

Therefore currently these strings can apparently only be rendered in English, unless we do some refactoring in main.js and/or router.js first.
Which could also possibly help decrease the perceived load time, but that's probably off-topic for this issue.

So I would suggest keeping the changes to a minimum, by allowing the aforementioned strings to stay displayed in English for now.

[taoeffect]

Yeah you can avoid the refactoring in your initial PR and that should be OK, we can open a followup issue to fix that.

Also, note that the current version of strings.mac and strings.linux doesn't translate L-function strings in .js files, but that should be an easy fix and Simon said he plans to release a new update that fixes that. So there are going to be missing translations because of that too for a little bit.

[taoeffect]

I think this has been closed now in #963 by @snowteamer! 🎉

Future iterations of this issue will focus on:

• The issue @snowteamer noted above with respect to L functions that get run prior to the call to 'translations/init'
• Any caching we would like to add so that we don't have to download the strings each time the app is run.
• Any memory-related issues, e.g. monitoring how much room all these strings take up in RAM and if it's too much storing them in something like IndexedDB instead