Openrefine: Plurals don't work for i18n package

[tfmorris]

Describe the bug

The i18n package is missing the CLD rule parser, so plurals don't work for many languages.

To Reproduce
Per https://github.com/OpenRefine/OpenRefine/issues/1858#issuecomment-637274292

$.i18n().locale = 'en';
$.i18n("$1 de $2 {{plural:$2|page|pages}}", 5, 10);
$.i18n().locale = 'fr';
$.i18n("$1 de $2 {{plural:$2|page|pages}}", 5, 10);

The last line gives this:

project-bundle.js:28751 Uncaught ReferenceError: pluralRuleParser is not defined
    at Object.getPluralForm (project-bundle.js:28751)
    at Object.convertPlural (project-bundle.js:28751)
    at Object.plural (project-bundle.js:28744)
    at Object.emit (project-bundle.js:28744)
    at project-bundle.js:28744
    at Function.map (project-bundle.js:504)
    at Object.emit (project-bundle.js:28744)
    at Object.parse (project-bundle.js:28739)
    at a.parse (project-bundle.js:28723)
    at Function.e.i18n (project-bundle.js:28723)

Proposed solution
Include https://github.com/santhoshtr/CLDRPluralRuleParser/blob/8baf9aedc428924fe6ee508b3d952cb5564efb3a/src/CLDRPluralRuleParser.js
(and perhaps https://github.com/santhoshtr/CLDRPluralRuleParser/blob/8baf9aedc428924fe6ee508b3d952cb5564efb3a/data/plurals.json)

(check for the appropriate version to match the version of Wikimedia i18n that we're using)

OpenRefine:

• Version 3.4 beta - current master

[antoine2711]

@tfmorris: it tested the french with this example, and it still doesn't work.

so I created a simple web page with jQuery
https://raw.githubusercontent.com/antoine2711/jQuery_i18n/master/index.html, with jQuery 3.2.1, and jQuery.i18n v1.0.7. And this works fine. But, on our codebase, jQuery v1.13.1 jQuery.fn.jquery "1.11.1", French is still not working.

But, if in file jquery.i18n.language.js, If remove the "fr" definition, then, the english fallback works fine.

So, is it possible to ask dependabot to update jQuery?

Regards,
Antoine

[thadguidry]

@antoine2711 There's a long standing issue to migrate jQuery for us... but it is now a bit more involved. Not impossible, but a bit involved. In fact, I think probably now involves a double-migration, but I could be wrong?

1st thing would be updating the jQuery migrate plugin.

• here: https://github.com/OpenRefine/OpenRefine/blob/master/main/webapp/modules/core/externals/jquery-migrate-1.2.1.js
• and here: https://github.com/OpenRefine/OpenRefine/blob/master/main/webapp/modules/core/MOD-INF/controller.js

If you have approximately 40 hours to spare :-) ...
You can get started reading about migrating here: https://jquery.com/upgrade-guide/3.0/

[antoine2711]

@antoine2711 There's a long standing issue to migrate jQuery for us... but it is now a bit more involved. Not impossible, but a bit involved. In fact, I think probably now involves a double-migration, but I could be wrong?

If you have approximately 40 hours to spare :-) ...
You can get started reading about migrating here: https://jquery.com/upgrade-guide/3.0/

Hi @thadguidry,
Then, since I can get the same result with deleting 3 lines in a file (_If in file jquery.i18n.language.js, I remove the "fr" definition, then, the english fallback works fine_), this might be a far easier path! ;-)

I wonder, before asking, I search for a jQuery issue (one who would have what you just wrote) and I didn't find one. Do you think I should create one with the data you provided here?

Regards,
Antoine

[thadguidry]

@antoine2711 I think around 2013/2014? You'd have to poke around. Might not even be an issue created, but was discussed, maybe mailing list? Anyways, @ultraklon did some of that for us before. And jQuery was captured as part of that EPIC for sure in the Project UI Improvements 2019/2020 but no tasks or issues made by me because we don't have a dedicated frontend UI person AND we haven't decided on our Roadmap completely for 4.0 what it needs to include AND if Vue.js is baby-stepped AND then what remains of jQuery in places and how to handle its migration if we need to!

In essence, someone can volunteer lots of time to research and do that if they have the frontend skills while keeping in lock step with changes happening in our master branch. :-) Piece of cake, right?

I think you personally would find the jQuery migration very interesting. I even played with doing it once and found it kinda fun, a bit of detective work and step by step. I did in fact migrate some functions in my local branch (never commited), but then gave up when I saw it would take hours, but was fun to see the process and how their migrate plugin worked and testing with it.

[antoine2711]

I think you personally would find the jQuery migration very interesting.

@thadguidry: yes, I would love it. I have a busy schedule and no free time (I'm steeling time now), but I like to code OR and contribute, so yes. In order not to generate chatter here, I'll cut short.

I will propose a PR for this issue to fix the problem of plurals in "FR", till we figure for jQuery.

Regards,
Antoine

[antoine2711]

@thadguidry: PR #2762 will fix french plurals, until jQuery is updated.

Regards,
Antoine

[tfmorris]

Reopening. @antoine2711 @thadguidry feel free to reopen issues yourselves if testing demonstrates that they're not really fixed.

[tfmorris]

There's a long standing issue to migrate jQuery for us... but it is now a bit more involved.

@thadguidry What's the issue number? The only thing close that I can find is my 2013 issue #817.

[thadguidry]

@tfmorris That issue and the Project writeup for the UI improvements: https://github.com/OpenRefine/OpenRefine/projects and https://github.com/OpenRefine/OpenRefine/projects/2 and the Google Docs project writeups for Phase 1 and 2 if you haven't seen them before.

[antoine2711]

Reopening. @antoine2711 @thadguidry feel free to reopen issues yourselves if testing demonstrates that they're not really fixed.

@tfmorris: can I get your support to merge this issue, and we can talk more about jQuery in another issue.

Regards,
Antoine

[tfmorris]

can I get your support to merge this issue, and we can talk more about jQuery in another issue.

@antoine2711 Just attempting to close out the side conversation that @thadguidry and you were having. It appears there is no existing issue, so I consider the case closed. From my point of view, things that aren't documented in an issue don't exist on the backlog.

PRs get merged. Issues get closed. The associated PR, #2774, will close this issue when it gets merged (after review of course).

[antoine2711]

PRs get merged. Issues get closed. The associated PR, #2774, will close this issue when it gets merged (after review of course).

@tfmorris: perfect. Thanks.

I've read a lot of things about i18n and such, but nothing I can document here (neither text nor links). But plurals in French and English are the same. I would consider updating jQuery OT in an PR talking about French plurals unless a link, direct ou indirect, can be provided, which nobody here did. I did a simple page with jQuery 3.2.1 and i18n v1.0.7 on my rep, and french plurals with the i18n.languages.json properly filed and everything worked. Could that translate in updating jQuery in OR to v3.x would also fix this problem? Maybe, but the cost to investigate is way beyond what I could dedicate/master.

But here, I think a small step that fixes a defined issue, document possible path to go beyond, is worth being consider a step forward.

Looking forward for the next. Regards,
Antoine

[thadguidry]

@tfmorris When translators need to deal with plurals, where will the documentation for that be located? So can you also add to the scope of this issue to also:

• [x] Add information for users on how to use plurals into the translating docs wiki page https://github.com/OpenRefine/OpenRefine/wiki/Translate-OpenRefine

[antoine2711]

@tfmorris When translators need to deal with plurals, where will the documentation for that be located? So can you also add to the scope of this issue to also:

• [ ] Add information for users on how to use plurals into the translating docs wiki page https://github.com/OpenRefine/OpenRefine/wiki/Translate-OpenRefine

@thadguidry, @tfmorris: I've added a few paragraphs in the Wiki to talk about plurals, substitution, for translation. But I feel it is less than optimal, though a good start. Care to give it a look?

Regards,
Antoine

[thadguidry]

@tfmorris @antoine2711 Thank you both for collaborating on those doc improvements! Gracias! Merci!

[tfmorris]

That issue and the Project writeup for the UI improvements: https://github.com/OpenRefine/OpenRefine/projects and https://github.com/OpenRefine/OpenRefine/projects/2 and the Google Docs project writeups for Phase 1 and 2 if you haven't seen them before.

@thadguidry None of those links included any links to an issue for updating jQuery, so I created #2932. It would be super helpful if folks could create actual Github issues for identified needs (and bugs) so that tasks can be properly tracked.

[thadguidry]

@tfmorris Thanks Tom. Yes, @wetneb @magdmartin and I talked about just that the other day on advisory committee meeting along with creating more design documents for community review on Google Docs or Hackmd.io