Site-kit-wp: Plugin translations not rendering fully translated strings

[jamesozzie]

Bug Description

As mentioned in a WordPress support topic some translations may not be rendering within the plugin, despite these translation strings existing and marked as complete.

An example of a string which should be fully translated in various languages is below (Site Kit > Settings > Connect More Services):
"_Connect More Services to Gain More Insights_"

Example below:

• Spanish (Spain) - marked as 100% translated.
  "Connect More Services to Gain More Insights" string translated but not rendering

• Bahasa Indonesian - marked as 87% translated.
  "Connect More Services to Gain More Insights" string translated but not rendering

Note also there are inconsistent results when checking from a clean WordPress install, with even less translations appearing:

Steps to reproduce

1. Switch language to Spanish in "General" settings
2. Visit Site Kit dashboard > Settings > Connect More Services (or any other pages with considerable content / strings)
3. Translations not always appearing

Additional Context

• Link to current translations & status
• Multiple languages tested with the same result, including languages with 100% completion
• When checking from other websites I encounter inconsistent translations. (One site has some Bahasa Indonesia translations which another doesn't)

---

_Do not alter or remove anything below. The following sections will be managed by moderators only._

Acceptance criteria

• Plugin translations for JavaScript files should be loaded and used as expected for WordPress versions >= 5.0.
• The existing broken approach where PHP translations are passed to JavaScript should be removed.
• It should be kept in mind that the plugin supports older WordPress versions up until 4.7, so newer core functions that are needed for translating JS file strings should only be called if they exist.

A note on testing

• The translation files that wordpress.org generates and that WordPress loads are based on md5 hashes based on the JS file names.
• Since our JS file names change between almost every build, testing this won't work outside of an actual release without the following workaround:
  
  
  
  • We put together a mini plugin (outside of Site Kit) that uses the WordPress core filter load_script_textdomain_relative_path. Download mini plugin here: https://gist.github.com/adamsilverstein/3eee29ea1370d0e119bb929a331e9f50
  
  
  • This filter allows to modify for which file path to look for a corresponding translation file.
  
  
  • We know the current file paths via the generated Manifest class - so we need to use the filter to replace all of those with the corresponding file paths from the latest release (for which we have generated translation files downloaded) - i.e. we need a map of currentBuildFilePath => latestReleaseFilePath.
  
  
  • We can either use some approach where we download the Manifest class from the latest release (e.g. https://plugins.trac.wordpress.org/browser/google-site-kit/trunk/includes/Core/Assets/Manifest.php) and parse out the contents (would work as a long-term workaround), or we can do something simpler for once-off testing where we just manually copy the file paths from the latest release into that mini plugin to make it work.
  
  

Implementation Brief

• Create our own entrypoint that exposes a global googlesitekit.i18n which is our own internalized version of @wordpress/i18n (since we want to avoid version conflicts and therefore bundle our own).
• Register that asset in PHP as googlesitekit-i18n and make it a key dependency of pretty much everything (add it to $dependencies array, but also to other key dependencies that use translation functions).
• Add an external for @wordpress/i18n to Webpack that will make it use googlesitekit.i18n in production.
• Add a private method set_locale_data to the Script class, which should be our own variant of WordPress core's WP_Scripts::print_translations method.
  
  
  
  • It should use wp_add_inline_script with a script that calls googlesitekit.i18n.setLocaleData (instead of wp.i18n.setLocaleData). The $position argument should be set to before so that the translation data inline script is output before the actual script.
  
  
  • It should rely on core's load_script_textdomain - for BC though, add this as a protected static method on our BC_Functions utility class and call it there. If the function doesn't exist, it should be implemented as a no-op that returns false (the same as if core didn't find any translations).
  
  
  • If the call to BC_Functions::load_script_textdomain returns a false-y value, simply bail and don't add the inline script.
  
  
• In the constructor of the Script class, "reassign" $this->args['before_print'] to a callback (using a closure) that first calls the new set_locale_data method and then calls the originally passed before_print callback (if any). See the Script_Data constructor for a somewhat related example.
• Remove usage of _googlesitekitLegacyData.locale.
• In our JS codebase, remove the loadTranslations utility function and all calls to it. setLocaleData is now called via the inline scripts in PHP.

Test Coverage

• N/A

Visual Regression Changes

• N/A

QA Brief

• Go to the Settings > General page and select Espanol as your site language.
• Go to the Dashboard > Updates page (/wp-admin/update-core.php) and click on the Update Translations button. After doing it, WordPress will download spanish translations for the plugin.
  
• Install and activate the following plugin: google-site-kit-language-loader.zip
• Go to the plugin pages and make sure the interface is translated into Espanol.

Changelog entry

• Fix JavaScript translations that were not appearing to work correctly, given the site uses WordPress >= 5.0, which is required for support of JavaScript translations.

[mushlih-almubarak]

Hi
Sorry, I found this bug and I am constantly monitoring its fix
But why haven't I received any updates so far? has this issue been forgotten, or what?
Looks like the last notification happened 5 days ago
I keep waiting
Thank you

[jamesozzie]

@mushlih-almubarak Rest assured we are currently investigating this. Keep watch of this issue whereby we hope to have further updates this week.

[mushlih-almubarak]

@jamesozzie Fine, thanks a lot

[adamsilverstein]

I investigated this a bit and found the following:

• JS translation JSON files are not correctly loaded, possibly due to this shortcoming.
• We can load translations for each script we enqueue in Scripts.php by adding a call to wp_set_script_translations, so in includes/Core/Assets/Script.php::enqueue() add
  wp_set_script_translations( $this->handle, 'google-site-kit' );

Once I changed this, the translations were loaded correctly, which I can verify looking at the source code:

I see the localized dashboard strings in the data, however, I'm still not seeing localization work correctly:

Hoping @swissspidy can help me track down whats going on here.

[swissspidy]

Loading the translations works by using the wp.i18n global. Sounds like the i18n package isn‘t added as an external in webpack so that it also uses the wp.i18n global.

[adamsilverstein]

Loading the translations works by using the wp.i18n global. Sounds like the i18n package isn‘t added as an external in webpack so that it also uses the wp.i18n global.

Ah! Thanks for the tip.

[adamsilverstein]

One other issue I'm working thru when debugging this: our JS file names include a hash that changes with each release. The translation file names are based on the last release version as far as I can tell, so my dev environment doesn't load the translation files correctly. Any tips to avoid that issue @swissspidy?

For the "external" mapping, I assume you mean in Webpack externals, so something like this?

[swissspidy]

Any tips to avoid that issue

Don't use hashes in file names? :-)

For the "external" mapping, I assume you mean in Webpack externals, so something like this?

Yeah, although just [ 'wp', 'i18n' ] works too.

[adamsilverstein]

Yeah, although just [ 'wp', 'i18n' ] works too.

Right, and that matches the other exports. I'll give this a try. One other idea if we want to keep our i18n bundled/in its own namespace would be to copy the translations over from the global.

[adamsilverstein]

The changes in this PR should provide a fix for JS translations - https://github.com/google/site-kit-wp/pull/2225. I haven't been able to test because my local development file hashes don't line up with the released translation versions (so the translation files never load). I do see the translations loading (in page source) by making the script change to the release plugin.

This approach "externalizes" i18n so we are using the same global object that WordPress core loads the translations onto. I looked into extracting data from wp.i18n directly and don't see a way to do that without altering the package. Given the LOE, I'm not sure this is worth trying, externalizing wp.i18n seems preferable.

[adamsilverstein]

Don't use hashes in file names? :-)

we added the hashes to avoid caching issues when we upgrade the plugin version. Otherwise some users get cached versions of some files and things break. The normal approach of versioning in wp_enqueue_scripts doesn't work because of how our build and load process works.

[mushlih-almubarak]

Hi
Has the problem been resolved?
Because I saw the plugin was updated

[adamsilverstein]

@mushlih-almubarak We are still working on a fix for this issue, thanks for your patience!

[adamsilverstein]

After further discussion, we have developed the following plan to resolve the translation loading issue:

Loading Translations

• Build a new shim file i18n-shim.js similar to hooks-shim.js.
• Add as a webpack alias for this shim (which will expose Site Kit's i18n globally).
• Create our own version of the core print_translations function - the Site Kit version will pass the translation data to our global i18n. Can be added before each script load with the wp_add_inline_script hook.

Local Testing

To test localization locally, we need a way to load translation files from the latest release of the plugin. Because translation filenames are based on source filenames (which change during our build process), we need to use the last release names when loading. We can get these files from the trunk manifest and use the load_script_textdomain_relative_path filter to set these correctly for local development.

[swissspidy]

Sounds complicated 😅 Was thinking about writing our own print_translations too, therefore curious to see the results!

[felixarntz]

To clarify: The reason that right now _some_ strings are still translated in JS even though JS translations don't work is that we currently pass PHP translation data to JS (via _googlesitekitLegacyData.locale global). This was implemented a long time ago, probably without awareness that that data only includes the PHP translation data, but _not_ the JS translation data. For example, we have a __( 'Settings', 'google-site-kit' ) in both PHP and JS, so that string will be "randomly" translated in JS.

There's another complexity here which is that Site Kit has a minimum requirement of WordPress 4.7 where all those PHP functions didn't exist at all. We can certainly rewrite our own version of WP_Scripts::print_translations, but I think rewriting/backporting functions like load_script_textdomain and load_script_translations would get a bit too crazy.

After further discussion with @aaemnnosttv and @tofumatt, I think we have two viable options here:

If we decide it's okay to not give JS translation support to pre-5.0 versions

• Create our own entrypoint that exposes a global googlesitekit.i18n which is our own internalized version of @wordpress/i18n (since we want to avoid version conflicts and therefore bundle our own).
• Register that asset in PHP as googlesitekit-i18n and make it a key dependency of pretty much everything.
• Add an external for @wordpress/i18n to Webpack that will make it use googlesitekit.i18n in production.
• Write our own variant of WP_Scripts::print_translations which uses wp_add_inline_script with a script that calls googlesitekit.i18n.setLocaleData. It will rely on core's load_script_textdomain - if that function doesn't exist, it simply won't provide JS locale data.
• Remove usage of _googlesitekitLegacyData.locale.
• No longer call setLocaleData in our JS code because it's now handled via the above approach using wp_add_inline_script.

If we decide we need to fully support pre-5.0 versions despite technical drawbacks

• Use the approach that Gutenberg had before wordpress.org supported JS translations, relying on PHP translation strings.
• Either create a POT file from our JS files and then turn that into a PHP file with the translatable strings, or alternatively use Webpack to grep translation functions (including translator comments) and parse those into a PHP file.
• Either way, wordpress.org would then just interpret those PHP translations and make them available in the PO file. We already pass that data to JS, so we could maintain that behavior.

[swissspidy]

If we decide it's okay to not give JS translation support to pre-5.0 versions

Sounds like very bad user experience for all non-English users

If we decide we need to fully support pre-5.0 versions despite technical drawbacks

Sounds like not-ideal translator experience

Between these two, I'd choose the latter.

(Ideally I'd bump the WP version requirement of course)

[aaemnnosttv]

@felixarntz IB looks solid, just a few details to clarify:

• Where would Script::set_locale_data be called from?
• We should specify that the $position for wp_add_inline_script providing translations is before rather than the default after

[aaemnnosttv]

IB ✅

[adamsilverstein]

@felixarntz I created a mini plugin that loads the correct language files - https://gist.github.com/adamsilverstein/3eee29ea1370d0e119bb929a331e9f50

I included the manifest mapping manually for now to keep things simple, this would need to get updated with each release. Testing in my local, once I added the wp_set_script_translations( $this->handle, 'google-site-kit' ); line after enqueueing our scripts, WordPress properly found the translation files installed from the last release.

[felixarntz]

@adamsilverstein Awesome, thanks! This will be great for testing.

[mushlih-almubarak]

Hi
I haven't heard of any updates in a while, has the problem been resolved?
Thank you

[jamesozzie]

@mushlih-almubarak Providing you're using WordPress 5.0 or above this should have a resolution for this soon.

[mushlih-almubarak]

Hi
I see the string "Audience overview for the last ... days" on the analytic tab has been translated in WordPress, but why not translate at my site?

[swissspidy]

@mushlih-almubarak As mentioned, this issue will be resolved soon.

[mushlih-almubarak]

So, it's not over yet?

[wpdarren]

QA Update: QA ⚠️

@eugene-manuilov a few observations on the perid dropdown on all modules, this is not translating.

Also, on Search Console, Google Analytics and Adsense modules, the period titles, i.e. Overview for the last X days and Top content over the last X days are not translated.

Should these be translated as part of this ticket? I wasn't 100% sure, so would like to check.

Verified:

• Installed and activated SK. The connection module on the Plugin page was translated - Screenshot
• When connection button was clicked the Site Kit information page appeared and was translated - Screenshot
• Google account connected to Site Kit. Dashboard loaded and translated - Screenshot 1 Screenshot 2 Screenshot 3
• Text translated on Search Console, Google Analytics and Adsense related modules (* observations above) Screenshot 1 Screenshot 2 Screenshot 3
• All settings for each modules connected were translated including dialog boxes - Screenshot 1 Screenshot 2
• Unconnected modules were translated - Screenshot
• Administrator Settings page was translated - Screenshot
• Reconnected Google Analytics and Adsense + PageSpeed Insights and all pages were translated - Screenshot

[eugene-manuilov]

@wpdarren I have checked translations and seems like it is the translation files issue itself. Here is the list of static copies that are untranslated on the admin pages and their translations in the PO file:

Overview for the last %s day

#: dist/assets/js/googlesitekit-user-input.a1b8fc80e7c0ca5a669f.js:64
#: dist/assets/js/googlesitekit-module.1de3128126ff4eb39c26.js:64
#: dist/assets/js/googlesitekit-settings.08e988e1f939c10ec910.js:64
#: dist/assets/js/googlesitekit-dashboard.8374a71cc8f439ca1dad.js:64
#: dist/assets/js/googlesitekit-dashboard-details.550792e3101137d4f96b.js:64
#: dist/assets/js/googlesitekit-wp-dashboard.c5ade2a5b7e6d9157023.js:45
#: dist/assets/js/googlesitekit-adminbar.45d91a863eee1a9cf73b.js:45
msgid "Overview for the last %s"
msgstr "Resumen de los últimos %s"

Audience overview for the last %s day

#: dist/assets/js/googlesitekit-user-input.a1b8fc80e7c0ca5a669f.js:53
#: dist/assets/js/googlesitekit-module.1de3128126ff4eb39c26.js:53
#: dist/assets/js/googlesitekit-settings.08e988e1f939c10ec910.js:53
#: dist/assets/js/googlesitekit-dashboard.8374a71cc8f439ca1dad.js:53
#: dist/assets/js/googlesitekit-dashboard-details.550792e3101137d4f96b.js:53
#: dist/assets/js/googlesitekit-wp-dashboard.c5ade2a5b7e6d9157023.js:30
#: dist/assets/js/googlesitekit-adminbar.45d91a863eee1a9cf73b.js:30
msgid "Audience overview for the last %s"
msgstr "Visión general de audiencia de los últimos %s"

Top search queries over the last %s day

#: dist/assets/js/googlesitekit-user-input.a1b8fc80e7c0ca5a669f.js:67
#: dist/assets/js/googlesitekit-module.1de3128126ff4eb39c26.js:67
#: dist/assets/js/googlesitekit-settings.08e988e1f939c10ec910.js:67
#: dist/assets/js/googlesitekit-dashboard.8374a71cc8f439ca1dad.js:67
#: dist/assets/js/googlesitekit-dashboard-details.550792e3101137d4f96b.js:67
#: dist/assets/js/googlesitekit-wp-dashboard.c5ade2a5b7e6d9157023.js:48
#: dist/assets/js/googlesitekit-adminbar.45d91a863eee1a9cf73b.js:48
msgid "Top search queries over the last %s"
msgstr "Las principales búsquedas en los últimos %s"

Last %s day / Last %s days

#. translators: %s: Number of days matched.
#: dist/assets/js/googlesitekit-modules-analytics.dddf6a327d2b0e2eb2c9.js:28
#: dist/assets/js/googlesitekit-dashboard-splash.471731318d5ab3dd6fa3.js:33
#: dist/assets/js/googlesitekit-user-input.a1b8fc80e7c0ca5a669f.js:10
#: dist/assets/js/googlesitekit-activation.173d01c116942791be20.js:23
#: dist/assets/js/googlesitekit-module.1de3128126ff4eb39c26.js:10
#: dist/assets/js/googlesitekit-settings.08e988e1f939c10ec910.js:10
#: dist/assets/js/googlesitekit-dashboard.8374a71cc8f439ca1dad.js:10
#: dist/assets/js/googlesitekit-dashboard-details.550792e3101137d4f96b.js:10
#: dist/assets/js/googlesitekit-modules-search-console.28e27a057191b7d632a3.js:10
#: dist/assets/js/googlesitekit-wp-dashboard.c5ade2a5b7e6d9157023.js:57
#: dist/assets/js/googlesitekit-adminbar.45d91a863eee1a9cf73b.js:57
#: dist/assets/js/googlesitekit-modules-pagespeed-insights.a3dbf11ed1277355b334.js:14
#: dist/assets/js/googlesitekit-modules-adsense.7855e733e9dfc511a980.js:33
msgid "%s day"
msgid_plural "%s days"
msgstr[0] "%s día"
msgstr[1] "%s días"

So, it's not a bug.

[wpdarren]

QA Update: Pass ✅

Verified:

• Installed and activated SK. The connection module on the Plugin page was translated - Screenshot
• When connection button was clicked the Site Kit information page appeared and was translated - Screenshot
• Google account connected to Site Kit. Dashboard loaded and translated - Screenshot 1 Screenshot 2 Screenshot 3
• Text translated on Search Console, Google Analytics and Adsense related modules (* observations above) Screenshot 1 Screenshot 2 Screenshot 3
• All settings for each modules connected were translated including dialog boxes - Screenshot 1 Screenshot 2
• Unconnected modules were translated - Screenshot
• Administrator Settings page was translated - Screenshot
• Reconnected Google Analytics and Adsense + PageSpeed Insights and all pages were translated - Screenshot

[felixarntz]

⚠️
@eugene-manuilov There is a follow-up bug caused by the PR for this: It removed _googlesitekitLegacyData.locale, however that global is still used in the getLocale function in JavaScript.

Can you open a follow-up PR which brings back _googlesitekitLegacyData.locale? However, we shouldn't bring back the whole JED locale data thing, we can simply assign locale as is_admin() ? get_user_locale() : get_locale() in PHP.

[eugene-manuilov]

@felixarntz PR has been created: https://github.com/google/site-kit-wp/pull/2380

[jamesozzie]

One user reported language translation issues in the support forums today. From testing on support side translations strings are not working rendering as expected, with only the menu items translated. None of the strings within the dashboard are translated in the two additional languages checked. These are languages with almost full translations (German and Portuguese Brazil)

The user who reported this is a Danish translation contributor, with more information in the supporting WordPress topic.

Additional context:
I've been able to replicate this on 2 test sites.

I've also created a video showing this behavior on one of the sites. You'll see the below:

• Language switched between German, Portuguese and English in both my WordPress general settings & user profile
• Updating from 1.22.0 to 1.23.0 (the issue persists with both plugin versions)

@felixarntz @eugene-manuilov Do you want me to create a new issue in relation to this or is this related?

[eugene-manuilov]

@jamesozzie can you go to your site, switch to either German or Portuguese language, and see if new translations are available on the /wp-admin/update-core.php page (at the very bottom of the page)? If you see the button there, could you please click on it to update translations and go back to the Site Kit dashboard to see if it helps to fix the translation issue? Let me know if it helps.

[jamesozzie]

@eugene-manuilov The same issue remains in my case. See below gif. No additional plugins. Tested on two sites.

[eugene-manuilov]

Hm... ok, @jamesozzie, could you please create a new issue for this? Also, can I get access to your sites or can you create a publicly available site with this bug?

[jamesozzie]

@eugene-manuilov Sure, #2566 now created. It seems to occur with some languages, mode details in the issue.

I've also provided logins via DM