Jetpack: Search: missing translations in the overlay

[gibrown]

We seem to be missing some translations in the search overlay. I don't see any problems with the code, but a word like "Relevance" appears to be translated for Spanish, but I don't see it in the UI when I set my locale to "Espanol".

I'm wondering if something is broken in our translations or the way we are displaying them. Strangely some of the headings for the filters are correct.

Steps to reproduce the issue

1. wp-admin -> Settings -> General
2. Set your site language to something else
3. Run a search on the front end.

[kraftbj]

I started looking into this and there's a couple of things going on, I think.

1. Need to call wp_set_script_translations for the jetpack-instant-search script.

2. Strings are not being passed into the translate.wordpress.org GlotPress. My understanding is it should find the strings when using the wp-i18n functions in any js file magically. I know it works for the blocks' editor.js, etc files, so need to dig in more to see what is different for Instant Search.

[gibrown]

Another case:

[jeherve]

I'll reopen this as it seems the last fix didn't work :(

Reported here:
https://wordpress.org/support/topic/jetpack-search-translation/

cc @jsnmoon

[jsnmoon]

It looks like wp-content/languages/plugins/jetpack-[your-language-of-choice].po is omitting translations for _inc/build/instant-search/jp-search.bundle.js:1 despite being translated for several languages already in translate.wordpress.org. I've confirmed that _inc/build/instant-search/jp-search.bundle.js was indeed committed to SVN.

I'll continue to investigate and report my findings here.

Paging @akirk who's better versed in our translations architecture than I am.

[jeherve]

Looking at the translation files available from language packs on a French site, here is what I get (I added the matching JavaScript files next to it):

jetpack-fr_FR-1c7420c5c99c6ed7a03f7b6f7fe38302.json // _inc/blocks/podcast-player/view.js
jetpack-fr_FR-4ecdb48266dceea37c977503447f349c.json // _inc/blocks/editor-no-post-editor.js
jetpack-fr_FR-554ecab996622b1334be87a17ae022de.json // _inc/blocks/editor-experimental.js
jetpack-fr_FR-6030814ecc6554f776e6c2f913459ba8.json // _inc/blocks/map/view.js
jetpack-fr_FR-e48297939dadcf2a16dec6bb472204f5.json // _inc/blocks/editor-beta.js
jetpack-fr_FR-f2b234a1eb36d7500074cd58bfbd9b1c.json // _inc/blocks/editor.js

The Instant Search file doesn't seem to be there, even though the translations seem to be available on translate.w.org:
https://translate.wordpress.org/projects/wp-plugins/jetpack/dev/fr/default/?filters%5Bterm%5D=_inc%2Fbuild%2Finstant-search%2Fjp-search.bundle.js&filters%5Bterm_scope%5D=scope_any&filters%5Bstatus%5D=current_or_waiting_or_fuzzy_or_untranslated&filters%5Buser_login%5D=&filter=Apply+Filters&sort%5Bby%5D=priority&sort%5Bhow%5D=desc

[akirk]

Looks like new language packs were generated yesterday https://translate.wordpress.org/projects/wp-plugins/jetpack/language-packs/, it now contains the file:

jetpack-de_DE-c6819ff8285929b79db5ef56423c3b36.json // echo -n _inc/build/instant-search/jp-search.bundle.js | md5sum

Likely the file was not generated yet because it uses the stable version to do so?

[jeherve]

You're right, it seems it fixed itself! Everything is now translated into French on my test site:

Closing.

Likely the file was not generated yet because it uses the stable version to do so?

I believe this is caused by this bug with the language pack generation process:
https://meta.trac.wordpress.org/ticket/5317

[julsyd]

Looks like this may still be happening, as reported in 3730908-zen.

[jeherve]

I'm able to reproduce the issue on a test site of mine. It's odd. I can see the translations on W.org, e.g.:

• https://translate.wordpress.org/projects/wp-plugins/jetpack/stable/nl/default/?filters%5Bterm%5D=Newest&filters%5Bterm_scope%5D=scope_any&filters%5Bstatus%5D=current_or_waiting_or_fuzzy_or_untranslated&filters%5Buser_login%5D=&filter=Apply+Filters&sort%5Bby%5D=priority&sort%5Bhow%5D=desc
  But I think those may be from old versions of Jetpack.

I don't see any json file generated for Instant Search if I download a fresh Dutch language pack from https://translate.wordpress.org/projects/wp-plugins/jetpack/language-packs/

❯ ls jetpack-9.4-nl_NL
jetpack-nl_NL-1bac79e646a8bf4081a5011ab72d5807.json
jetpack-nl_NL-1c7420c5c99c6ed7a03f7b6f7fe38302.json
jetpack-nl_NL-4ecdb48266dceea37c977503447f349c.json
jetpack-nl_NL-554ecab996622b1334be87a17ae022de.json
jetpack-nl_NL-6030814ecc6554f776e6c2f913459ba8.json
jetpack-nl_NL-62e43f7d6792ea2be029a71950464bf3.json
jetpack-nl_NL-c6819ff8285929b79db5ef56423c3b36.json
jetpack-nl_NL-e48297939dadcf2a16dec6bb472204f5.json
jetpack-nl_NL-f2b234a1eb36d7500074cd58bfbd9b1c.json
jetpack-nl_NL.mo
jetpack-nl_NL.po

I believe the Instant search file should be jetpack-nl_NL-7d84eb6e7bd327f115497bb12c255d7f.json:

php > echo md5( '_inc/build/instant-search/jp-search-main.bundle.js' );
7d84eb6e7bd327f115497bb12c255d7f

I wonder if some of the recent changes we've made to projects/plugins/jetpack/tools/webpack.config.search.js (maybe the IE11 changes?) may have broken the way the translated strings get minified during the build. @jsnmoon Do you think you could take a look?

Thank you!

[jsnmoon]

@jeherve: Upon investigating jetpack-9.4-ko_KR, I've verified the following:

• echo -n _inc/build/instant-search/jp-search.bundle.js | md5 returned c6819ff8285929b79db5ef56423c3b36.
• jetpack-9.4-ko_KR/jetpack-ko_KR-c6819ff8285929b79db5ef56423c3b36.json contained translations for _inc/build/instant-search/jp-search.bundle.js.
• Instant Search remained untranslated despite the site's configured Site Language.

~I think it's worth spinning up a new issue to track this translation regression; the original issue was caused by incorrect Webpack configuration preventing translation JSON generation.~

~More importantly:~ I discovered that Jetpack's wp-admin sections like "At-a-Glance" and "Settings" also remained untranslated, despite having the requisite translations within jetpack-ko_KR-1bac79e646a8bf4081a5011ab72d5807.json. Can you reproduce this in Dutch? ~If so, I wonder if there's a bigger systematic issue with our translation workflow?~

[jeherve]

discovered that Jetpack's wp-admin sections like "At-a-Glance" and "Settings" also remained untranslated, despite having the requisite translations within jetpack-ko_KR-1bac79e646a8bf4081a5011ab72d5807.json. Can you reproduce this in Dutch?

This looks ok to me:

jetpack-9.4-ko_KR/jetpack-ko_KR-c6819ff8285929b79db5ef56423c3b36.json contained translations for _inc/build/instant-search/jp-search.bundle.js

This seems to be true for all languages, but I don't understand why since we do not have that file in the Jetpack repo anymore; the bundle now lives at jp-search-main.bundle.js, and that's how we set the translations:
https://github.com/Automattic/jetpack/blob/6162b89a4ca6c2aee50685f3eb4c0e7aa2a149a1/projects/plugins/jetpack/modules/search/class-jetpack-instant-search.php#L88-L90

[jsnmoon]

This looks ok to me (screenshot of At-a-Glance in Dutch attached) (@jeherve)

My mistake; it turns out I needed to install additional translations via /wp-admin/update-core.php after changing the site language. After doing so, the wp-admin Jetpack sections are properly translated.

jetpack-9.4-ko_KR/jetpack-ko_KR-c6819ff8285929b79db5ef56423c3b36.json contained translations for _inc/build/instant-search/jp-search.bundle.js

This seems to be true for all languages, but I don't understand why since we do not have that file in the Jetpack repo anymore; the bundle now lives at jp-search-main.bundle.js, and that's how we set the translations:
https://github.com/Automattic/jetpack/blob/6162b89a4ca6c2aee50685f3eb4c0e7aa2a149a1/projects/plugins/jetpack/modules/search/class-jetpack-instant-search.php#L88-L90

Ah, good point -- I'd overlooked this part. I'll dig deeper into this today.

[jsnmoon]

Here are my findings thus far:

1. Other language packs also incorrectly point the translations to jp-search.bundle.js (c6819ff8285929b79db5ef56423c3b36) instead of jp-search-main.bundle.js (7d84eb6e7bd327f115497bb12c255d7f).

2. jp-search-main.bundle.js was properly committed to SVN during the Jetpack 9.4 release.

3. Manually generated jetpack.pot on the latest master via wp i18n make-pot . correctly references jp-search-main.bundle.js and has no references to jp-search.bundle.js.

4. The i18n CLI scanner correctly picked up strings from jp-search-main.bundle.js. This can be verified via these translations which all cite jp-search-main.bundle.js as the citation source.

5. The current stable translations incorrectly cite jp-search.bundle.js as the source. The development translations correctly cite jp-search-main.bundle.js as the source. Perhaps this is related to the issue @jeherve opened here?

I suspect that the language pack generation is going awry somewhere.

[jeherve]

Perhaps this is related to the issue @jeherve opened here?

It could be, although the problem seems a bit different. The language pack page mentions that the Dutch language pack was last updated on 2021-02-03 21:32:40 for version 9.4; this matches the 9.4 release. So it would seem that the language pack generation did happen, but the new file did not get picked up for some reason?
I wonder if we should try to rename the search bundle handle from jetpack-instant-search to something else, to see if it would get picked up?

[jsnmoon]

I wonder if we should try to rename the search bundle handle from jetpack-instant-search to something else, to see if it would get picked up?

Would it make sense for us to wait and see how the 9.5 release affects the language pack generation? I imagine it'd overwrite the current stable translations with the development translations, right?

[jeherve]

We can wait, but I'm not sure it will be any different from the 9.4, where the bundle name was already different.

I imagine it'd overwrite the current stable translations with the development translations, right?

I believe that should have happened in 9.4 already, but it would seem that the file the translations come from was not updated?

[jsnmoon]

Here's @dlind1's take over Slack (p1614295120015100-slack-C02AED43D):

On WPORG the process should be [...] that the POT files are generated for each branch automatically and then the PO/MO/JSON files are generated from t.wp.org, so it looks like something went wrong with the POT generation and import and the file comments contained jp-search.bundle.js even though the actual filename was different.

It could be that the strings extraction process has failed and the current stable translations are using the old POT file which has the old JS filename.

I'll keep an eye on 9.5 language packs following release and follow-up; we might need to loop in WPORG help to fully resolve this issue.

[jsnmoon]

It looks like the 9.5 language packs suffer from the same issue. I've reached out to the #polyglots community at WPORG Slack for additional guidance.

[danjjohnson]

Also reported in 3805831-zen

[mzakariya]

Reported in #3793606-zen and specifically cited as a reason for disabling the JP Search overlay.

[dlind1]

It looks like the issue is resolved now - the Jetpack stable project shows the correct filename and the correct language packs were generated.

[jsnmoon]

I've confirmed that the issue has been resolved via my test site, thanks for the heads up @dlind1!

@danjjohnson and @mzakariya, could you please advise the customer to update their language packs? I believe there should be a notice within /wp-admin/update-core.php.

[danjjohnson]

Thanks, I've let both users know.

[danjjohnson]

There still seem to be some missing translations in Hungarian:

Confirmed on a test site.

Reported in 3858828-zen

[jeherve]

@danjjohnson Hungarian isn't fully translated, so they do not get language packs from WordPress.org:
https://translate.wordpress.org/locale/hu/default/wp-plugins/jetpack/

As mentioned on WordPress.org, the initial language pack for the plugin will be generated when 90% of the Stable (latest release) sub-project strings have been translated (currently 33%).