Ultimate-geography: FR Translation Updates

Created on 25 Apr 2020 · 6Comments · Source: anki-geo/ultimate-geography

Here's the first batch of issues with the FR deck translation found by the Wikipedia checking script I wrote. Obviously my script still needs cleanup, ought to return results in a nicer format, and so on, but I wanted to share this to both improve the deck, and get some feedback about what's helpful/unhelpful about these results. The tool found a ton of potential issues, so any helpful feedback will hopefully make it more likely for people to sort through the results. Script is in https://github.com/ukanuk/scratchpad-public-viewing/tree/master/ug_to_wp

First things I'm noticing

Disambiguation pages cause difficulty. This says "Georgie" should be changed to "Georgia", but that's just because it's looking at a disambiguation page. I have an easy fix in mind for this in the works...
Caspian Sea apparently doesn't match, but it sure looks like it does.... I'll look into this more.
A lot of failures are from non-matching accents. #167 somewhat discussed this for the English version of the deck (specifically just for apostrophes), but I'm unsure how it should apply to translations.

@GrimPixel, you make want to take a look at this as well. You mentioned that with some extra features, it could be useful for language learning? I'm not sure what features you mean. For now I'm leaving it in my personal scratchpad repository, and depending on feedback/development I can create a new repository for it to have a life of its own, or merge it into axelbock/anki-ultimate-geography/src

Some stats for context:

     Total non-blank fields: 2591
            Values matching: 2284
  Values w/o Wikipedia data: 108
 Values with fuzzy matching: 131
  Total Values not matching: 68
   Values not matching (en): 1
   Values not matching (de): 12
   Values not matching (es): 12
   Values not matching (fr): 15
   Values not matching (nb): 28

Mismatches between UG:French and Wikipedia:

Fields which probably/mostly match...

... But should get human verification for missing accents, incorrect capitalization, inconsistent abbreviations (St. vs Saint), extra or missing capitals for countries with more than one, etc.:

Fields for which UG data exists, but Wikipedia data could not be found

South Africa:Capital:fr (UG=Pretoria, Le Cap, Bloemfontein), Canary Islands:Capital:fr (UG=Santa Cruz, Las Palmas), Yemen:Capital:fr (UG=Sanaa), Brazil:Capital:fr (UG=Brasilia), Guam:Capital:fr (UG=Hagåtña), United States of America:Country:fr (UG=États-Unis), Antigua and Barbuda:Capital:fr (UG=Saint John's), Zanzibar:Country:fr (UG=Zanzibar), Grenada:Capital:fr (UG=Saint-Georges), Cayman Islands:Capital:fr (UG=George Town), Guernsey:Capital:fr (UG=Saint-Pierre-Port), Nauru:Capital:fr (UG=District de Yaren), United States Virgin Islands:Capital:fr (UG=Charlotte Amalie), Pitcairn Islands:Capital:fr (UG=Adamstown), Artsakh:Country:fr (UG=Haut-Karabagh), Tonga:Capital:fr (UG=Nuku'alofa), Balkan Peninsula:Country:fr (UG=Balkans), Réunion:Capital:fr (UG=Saint-Denis), Australia (Oceania):Country:fr (UG=Océanie)

content

Source

ukanuk

😄1 👍1

Most helpful comment

This is great! Thanks!

Another disambiguation-related problem is "West Island". The English article named "West Island" is apparently about some part of Montréal (Québec); the article about the capital of the Cocos Islands is named West Island, Cocos (Keeling) Islands. In French, the name of the West Island in Québec is translated, but not of the Cocos Islands one. I'm not sure how to fix this, in a general way, though.

(The same issue occurs for Karyès — Karyes links to a village on the Peloponnese, while the capital of Mount Athos is Karyes, Mount Athos.)

Kiribati:Capital:fr UG=Tarawa-Sud WP=Tarawa

As an aside, it's nice that the edit we made to the article in November, to change the capital to Tarawa, hasn't been reverted. :) We can probably now also update the capital we use here!

aplaice on 25 Apr 2020

👍2

All 6 comments

That's a lot of errors to fix! Thanks for making this really useful script. 🥇

To trim down the list, you may want to ignore parenthesised disambiguation text in Wikipedia page titles when UG doesn't include any. This will remove false positives such as _Vienne (Autriche)_, _Montserrat (Antilles)_, etc.

However, when UG includes parenthesised text, you should definitely use it to fetch the Wikipedia page - e.g. for _Saint-Martin (Antilles françaises)_, your script returns the page of the island of Saint Martin, instead of that of the overseas territory.

Some of the unmatched results are surprising, for instance _Haut-Karabagh_, which definitely has a matching article.

axelboc on 25 Apr 2020

👍2

I mean if users have a word list and want to learn their counterparts in another language, they can make use of your script.
An example is https://polyglotclub.com/wiki/Language/Multiple-languages/Vocabulary/Animals-in-many-languages. It was made manually and cost lots of effort.

Maybe we can go even further: database of word list. Language learners can directly learn them with Anki.

GrimPixel on 25 Apr 2020

This is great! Thanks!

(The same issue occurs for Karyès — Karyes links to a village on the Peloponnese, while the capital of Mount Athos is Karyes, Mount Athos.)

Kiribati:Capital:fr UG=Tarawa-Sud WP=Tarawa

As an aside, it's nice that the edit we made to the article in November, to change the capital to Tarawa, hasn't been reverted. :) We can probably now also update the capital we use here!

aplaice on 25 Apr 2020

👍2

Some of the unmatched results are surprising, for instance Haut-Karabagh, which definitely has a matching article.

I've got to go to bed, but wanted to respond to this particular one since it involves interpretation of CONTRIBUTING.md.

The English UG country name is "Artsakh", but the title of the Wikipedia article is "Republic of Artsakh" https://en.wikipedia.org/wiki/Republic_of_Artsakh. According to CONTRIBUTING.md:

...We take the title of the Wikipedia article for the country, in the language of the given deck

Does this mean the English deck should be updated to use Republic of Artsakh for the name? In the Wikipedia article summary text, it says:

Artsakh, officially the Republic of Artsakh or the Nagorno-Karabakh Republic, is a...

Similarly for China, the article summary text says:

China, officially the People's Republic of China, is a...

I've seen this on at least a few cards, where UG actually uses the first description in the Wikipedia article rather than the article's title. I think one can reasonably argue that when the summary text says "Name, officially Xyz Name", that the former non-official name is probably the more common one. Obviously for my script, it would be easiest if UG used the Wikipedia title, but I'm feeling confident I can work with it either way.

To detail why my script failed this particular lookup, it's because it blindly searched Wikipedia for "Artsakh", which returned the disambiguation page https://en.wikipedia.org/wiki/Artsakh. You'll note the page has some language interwiki links, but none for French, thus why the script failed in finding any French page. Skipping to https://en.wikipedia.org/wiki/Republic_of_Artsakh, we see that the French interwiki link is https://fr.wikipedia.org/wiki/R%C3%A9publique_d%27Artsakh, which redirects to Haut-Karabagh as you expected. So once I figure out handling disambiguation pages and/or the UG country name is changed to "Republic of Artsakh", the script will find the correct translation.

ukanuk on 25 Apr 2020

👍1

Does this mean the English deck should be updated to use Republic of Artsakh for the name?

Yep, see #294. The title gives the most commonly used name. It seems short names are often placed before official names in intros, and sometimes the short name is not the most commonly used name.

So you use interwiki links to find articles? Why not try to use the French name as provided in the deck directly and see if it leads somewhere (and without a redirect)? Seems like a more straightforward solution if the goal is to check the correctness of existing translations.

axelboc on 25 Apr 2020

All, thank you for the great feedback!

So you use interwiki links to find articles? Why not try to use the French name as provided in the deck directly and see if it leads somewhere (and without a redirect)? Seems like a more straightforward solution if the goal is to check the correctness of existing translations.

Agreed that this is reasonable for checking correctness of existing translations. However, I partly started this in response to #239 which was for new language generation. I also felt that since English Wikipedia has the most viewers and editors by a significant margin (compare statistics in https://stats.wikimedia.org/#/en.wikipedia.org/contributing/editors/normal|line|2-year|~total|monthly), it was thus better suited for monitoring for updates to Wikipedia than individual languages separately. And for less common languages, I was concerned whether country infoboxes would even contain capital information.

However, in writing this all out I find my reasons don't hold much weight. A feature-complete script can certainly check the interwiki links, but it also ought to check the native-language pages themselves. After all, in the event that the capital of French Polynesia changed, it's more likely that the French Wikipedia article would be updated before the English one, and that's a problem if this script only watches the English article.

Lastly, when I was starting this script I naively hoped that with interwiki links, I wouldn't have to follow any redirects. I had came to the conclusion that the only way on Wikipedia to follow non-HTTP redirects was by installing selenium, which has significant extra installation requirements that would make it less convenient for others to test the script themselves. This hope to not need redirects has been proven false by Republic of Artsakh, which links to fr:République d'Artsakh and does not follow the redirect to Haut-Karabagh. With my knowledge now though, I think I may be able to use the Wikimedia API directly or wptools instead of selenium.

ukanuk on 26 Apr 2020

👍1

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Country info often excessively informative in Country -> Map cards

aplaice · 5Comments

Dealing with duplicate Country Names

aplaice · 3Comments

Flag similarities between Sudan and Palestine

josealberto4444 · 3Comments

Vectorized ocean maps

NathanaelRea · 3Comments

Add short location info to islands, microstates and small dependent territories

axelboc · 7Comments