Habitica: Articles of food (and potentially pets) are not translatable

[ghost]

I'm creating a new issue in addition to mentioning this bug in https://github.com/HabitRPG/habitrpg/issues/4522#issuecomment-110843828 because it seems not so trivial at first glance:

articles ("a") of food and pets' names are not translatable in content.coffee thus pets.json feedPet string (Feed <%= article %><%= text %> to your <%= name %>?) rendered in inventoryCtrl.js returns a mix of translated text and untranslatable article "a" in the middle

Initially reported by @ルナ in the guild chat

Slightly connected https://github.com/HabitRPG/habitrpg/issues/3571

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[crookedneighbor]

For my script to detect issues with translations strings, I had to blacklist a few strings from being checked because some translations do not use the dropArticle variable.

There has got to be a better way to do this.

[ghost]

@crookedneighbor I'm not sure I understand your message right.

The ticket is about a js line inserting English article a for some kinds of food in all the languages.

Feed <%= article %><%= text %> to your <%= name %>?

results in e.g

`スケルトン狐に_a_魚をやりますか？

Do you mean translators can just throw <%= article %> away from the string and it won't lead to Error in UI - or is this skipping only for the test purpose?

[SabreCat]

I think that "article" thingy predated the site having translations at all. Feel free to burn it to the ground.

EDIT: Burn the dropArticle implementation, not the site. xD

[paglias]

One idea could be to create a string for each food made of article+food text. And then each language can manage it differently

[SabreCat]

That's probably best!

[rbreu]

I'd be interested to look into this (and #3571 ).

[Alys]

@rbreu Thanks!

[rbreu]

The big drop item translatable string roundup

We have the following translatable keys that get used in various messages and on theri own:

dropEggXXXAdjective
dropEggXXXMountText
dropEggXXXText
dropFoodXXX
hatchingPotionXXX

Full list of messages concerning drop items, with example translations to German, in this gist: https://gist.github.com/rbreu/1b56c12a302c100ec0cef1f96b9b7660

Problem 1: Grammatical gender & articles

Languages with grammatical gender use different articles for different genders. Example: The wolf => der Wolf, the sheep => das Schaf, the cow => die Kuh

Proposed solution: Make different translatable keys for the noun, the noun + indefinite article, and noun + definite article.

Additionally, I noticed one message that's using the possessive ("feed this to *youro pet?"), again, the possessive has to adhere to the gender of the noun (e.eg. dein Wolf, dein Schaf, deine Kuh). We could either reword the message, or add a fourth key, noun + possessive.

Splitting things up like that would solve a lot of problems, but I still have a couple of questions:

• Are uncountable nouns going to cause trouble? I don't think so, you could always go with "a piece of" or similar constructions
• do all languages agree on when to use definite/indefinite/no article? This is the key question. I know there are languages that use articles a lot less. (Russian??? Japanese??? Latin??? I have no idea, unfortunately.) I they consistently drop articles where English would use them, they could just set their "article + noun" to the article-less translantion. Were're going to have troubles, though, if languages do this "randomly" different from English.

Problem 2: Grammatical gender & adjectives

Not orignally part of this ticket, but I noticed that we have separated "type of animal" and "animal name" into two keys that get patched together, e.g. "base wolf" and "white wolf" are made up of two keys. Unfortunately, the adjective also has to adhere to the grammatical gender of the noun in a lot of languages, and in German those are also different in the definive and undefinite case. E.g.:

der weiße Wolf, ein weißer Wolf
das weiße Schaf, ein weißes Schaf
die weiße Kuh, eine weiße Kuh

The only solution to this I can think of is having all combinations of potions and pets/mounts as their own key, again for each of the article versions as per solution to 1). That's going to introduce a lot of new keys, though. Food items already work that way, btw.

Problem 3: Grammatical cases

In German at least, articles and adjectives not only have different forms according to gender, but also according to grammatical case. E.g.:

a fish = ein Fisch
You have found a fish = Du hast _einen_ Fisch gefunden
the fish = der Fisch
The wolf really likes the fish = Der Wolf mag _den_ Fisch sehr gerne!

I'm not sure though how many languages do this? I can't really think of a solution to this that isn't make individual messages for each item combination. In German at least, you can often reword the translations so that they work (You have found something! It's a fish!), so I wouldn't suggest solving this problem if it's only German that's affected. I'm mainly curious whether other languages have other problems that wouldn't be solved by the cases 1) and 2).

In conlusion, I can just say that patching sentences together across different languages is hard. Any thoughts?

[cmajames]

I can comment at a basic level on Japanese. I'm guessing that a lot of what I mention is already covered by some aspect of the code base, but I've mentioned it in case it is an issue.

Articles identifying count

In this case, articles aren't a thing: other constructions are used. The most common of those is [Noun][Particle][Number][Counter][Verb]. For single objects, [Number] and [Counter] are dropped: they're said to be implicit in the "Topic-Context" of the sentence.

Snafoo: Adjectival sentences.
"Hon wo sam-pon naku-nari-chatta." "Unfortunately, I lost three books."
Now, if I want to talk about the three books that were lost, there are two primary options:
"naku-natta hon" "the lost book(s)" (Plurality implicit in context; assumed to be one if ambiguous)
"mi-tsu no naku-natta hon" "the three lost books" (Explicitly counts the books in question)
I would not, however, use "san-pon naku-natta hon," as that would refer to "the book that lost three book-like-objects."

Articles identifying people and groups

Nouns relating to people and groups conjugate in interesting ways... perhaps most notable is the "conjugation" of names.

When referring to someone, there are a number of options available in the Japanese language for referring to someone. The choice regarding which one to use is dependent on the relationship between the speaker, the listener, and the person in question. Generally, these can be divided into two groups, the in-group and the out-group. Some examples of in-group articles include "-kun" and "-chan" (anyone who watches anime will likely have heard these). Out-group articles include "-san" and "-sama." Of these, "-san" is typically the default when referring to someone else, while the use of "-sama" is used to placing the named person above one's self.

When referring to a group of people, the suffix "-tachi" is common, but not omnipresent. (Less common is the suffix "-ra"). For example, "issha-tachia" refers to a number of physicians. Depending on usage, that can be a concrete group of physicians or the practitioners of the medical profession as a class. Note that "-tachi" does not a pluralize, but a collectivize: it refers not explicitly to the group, but rather to a group of which the base noun in question is a part of. Ambiguity abounds. At the other end of the spectrum, "ano" (literally that) can be used to specify a single person or thing. Some examples:
"Ano issha wa tottemo shinnsetsu desu-ne." "That doctor was very kind, weren't they?"
"Issha wa tottemo shinnsetsu desu-ne." "The doctor(s) (is/are) very kind, aren't they?"
"Issha-tachi wa tottemo shinsetsu desu-ne." "The doctors are very kind, aren't they?"
Note: () represent an ambiguity in translation that is dependent on context.

Snafoo: Pronunciation. The particle "-tachi" will take on the form "-dachi" when preceded by another voiced consonant (g, z, j, d, dz, b, p).

Snafoo: Glossing: things stack, and not always in the ways you might expect them to. Examples:
"Tomu-tachi" "Tom's group"
"Tomu-tachi-sama" "Tom's group" (Honorific form.)
"Tomu-kun-tachi" " Tom's group" (Indicates familiarity with Tom.)
"Tomu-kun-tachi-sama" "Tom's group" (Indicates familiarity with Tom, but not with his group, and indicates that the speaker either wasn't thinking or is indifferent to a perceived status difference.)
"Tomu-ra" "Tom's group" (Indicates familiarity with Tom and his group.)

Snafoo: Context. The meaning of "-tachi" changes. "Tomu-tachi" can one moment mean "Tom's friends," the next "Tom and his siblings," and at another "Tom's extended family."

Snafoo: Friends. "Tomodachi" is the word for friend(s), again context specific. Yet collectivizing cannot occur with "-tachi" as it's already present in the base word! Here's the one instance that "-ra" must be used: ie "Tomodachi-ra" to refer to friends plural. "Tomodachi-tachi" is not techincally incorrect, but very awkward and highly frowned upon.

---

I could probably go on, but I'll spare you for now.

[zerom1]

Hi everyone, I'm French, I joined the French Linguists Guild a month ago, and I found that "potato issue" too... I'm glad I found this thread !

The potato issue is the same for us. It's written "vous avez trouvé a potato" while it should be "vous avez trouvé une* patate". What I don't get is that it seemed to me that in every other strings, <%=article%> was properly translated in French... so the issue seems to come to this particular string.

When <%=article%> is properly translated, we have the same problems as @rbreu too ! Patate (potato) is feminine, Poisson (fish) is masculine, Chocolat is masculine-uncountable, Viande (meat) is feminine-uncountable... so when translating <%=article%>, as it won't adjust to the food, we have to write "le/la" which is :

1. very unpleasant to the eye
2. still not quite right when the food is uncountable (you would write "some meat", not "the meat").

We're currently trying to replace all the nouns we can with nouns that have no gender... but it's 95% of the time impossible... these words are rare.
we are replacing "the" by "your" in "you give your pet the <%food>" too (but when you find the food, it's not yet your food, so it does not work).
and we're trying to replace all the adjectives too... for example the colors of the pets. Golden wolf is"Loup doré" but Golden spider is "Araignée dorée", so everywhere you'll see written "doré/e". We're thinking about "safran" (saffron) as it's invariable...

Those tricks are quite challenging and we love challenges, but it's frustrating when no tricks seem to do the job.

The main problem is when talking to, or talking about the user itself. We don't like to write everything in masculine while half the users are women and girls. So Habitica is full of "Vous êtes plus fort/e" (you're stronger), "Vous êtes soigné/e" (you've been healed), "Vous êtes abonné/e"(you're a subscriber)... and it's very hard to find other ways to say these things ("vous avez un abonnement" (you have a subscription) is pretty ugly, for example). But for this, only an option "I'm a boy / I'm a girl" under the language option and sometimes two translations in Transifex would do the job.

[ghost]

@Alys, All
for the sake of housekeeping and managing the discussion here it seems to me that there are two options:

• keeping this very issue focused on the problem of adding untranslatable English article "a" to all the language strings, while we have #3571 for more generalized problem. It feels like fixing this very issue can be done with far less effort as a part of the problem - taking the overall issue step by step.
• focusing on the general problem - and then this very issue will be fixed alongside with the main one. In this case I'd suggest closing this ticket in favor of #3571

[ghost]

@crookedneighbor @paglias could you please help me understand malformedStringExceptions in gulp-transifex-test.js
https://github.com/HabitRPG/habitica/commit/9d9e99045db801c041aad9fd257565a41dfcb0e6#diff-c8dd712740555cab0ddd553fb4e95638R18

Are these exceptions for build testing only or does it mean that for example translators can omit article and dropArticle variables in the strings mentioned (messageDropFood, armoireFood, feedPet) - and it won't result in the usual Error processing the string?

[Alys]

@GitHubSphinx

_"keeping this very issue focused on the problem of adding untranslatable English article "a" to all the language strings"_
I think you're right that this issue should be kept open to work on that. I feels like a fairly easy fix to me at the moment, although I haven't looked into it in depth.

That code you linked to looks like it's just used during the testing phases and isn't relevant to how the website runs. I.e., Translators can't omit the article and dropArticle variables. I'm not sure why three food-related string types are omitted from testing - possibly because the article variables haven't yet been implemented fully, although that's a wild guess.

[ghost]

@Alys thanks!

cross posting here and in the i18n guild
At the same time I can confirm, that omitting dropArticle in messageDropFood doesn't seem to give any errors and notifications work like charm. And I don't see reasonable difference with the other 2 strings - which I must have tested, but don't remember the outcome.

P.S. issue closed by mistake, I'd suggest keeping it open until an experienced developer confirms that articles can be omitted.

[ghost]

cross posting here and in the i18n guild

Another update: I see no difference in the way articles are introduced in all the three strings: messageDropFood, feedPet, and armoireFood and all of them don't produce errors if article variable is omitted - and translators of a couple of language teams have done it months or even years ago... My guess would be that JS doesn't give errors like Jade or the test exception may be mirrored somewhere in the code responsible for rendering translated strings for production site. OR it's a bug that should be fixed - because I don't remember official comments about any possibility of omitting variables - and in this context the situation doesn't feel right.

Result: as of now, the strings messageDropFood, feedPet, and armoireFood are rendered even if they have article variables omitted, BUT I'd suggest waiting for a comment from a developer before applying any changes to translations of these strings

And let me quote @Alys:

Interesting that omitting <%= dropArticle %> in messageDropFood works; I wouldn't have guessed that. I recommend though that translators don't start taking it out because we really should just fix whatever this bug is. Although I notice in website/common/script/content/index.js, only some foods have an article string, presumably based on whether we'd use one in English, so it might be more complex than it initially appears. :( I'm going to hold off on more comments until one of the other admins says anything, since they'll be more knowledgeable about it than I am. Please ping us again in a week if there's been no progress.

[lynxlynxlynx]

I guess it comes down to how lodash does templating, since that's what t() uses. I don't see anything in the examples, but this would be much simpler to test than whole Habitica:
https://lodash.com/docs/4.17.2#template
(iow, do non-matching placeholders get silently dropped)

Ah, it throws a ReferenceError in this case, so it's odd, as Habitica should catch the exception.

[lynxlynxlynx]

That's it, it throws the exception only if a param is missing, but not the other way round. So

var compiled = _.template('a <%= b %> c !');
console.log(compiled({ 'b': 'BBB', d:234 }));

works just fine without any complaints. This is why removing the token is not noticed by the system.

[ghost]

Bringing the issue up. @lynxlynxlynx I'm sorry I don't quite get it, do you mean that it's OK to remove tokens from strings processed with JS?

Could we please get an official confirmation that article variables can be omitted in these three strings: messageDropFood, feedPet, and armoireFood - as it causes no errors.

@Blade has stated in the second message of this thread that:

...some translations do not use the dropArticle variable

Thus it seems having no errors is the way it's meant to be. If so, I believe the ticket can be closed and I'll pass the instructions to Linguists.

[lynxlynxlynx]

It's ok from lodash' point of view, but Habitica devs need to make a choice — allow that or not. Leaving it be as-is is a bit dangerous as stated elsewhere, since translators could erroneously remove other tokens elsewhere without anyone noticing.

[ghost]

OK, knowing that it's OK from the lodash point of view and having the exception in the test script, I believe translators can share the practice of not including article tokens in these three strings: messageDropFood, feedPet, and armoireFood.

I'll close the issue now. Should developers want the translators to revisit these strings later in the future then either wording can be changed in English or someone can just ping me and I'll delete the translations.

[lynxlynxlynx]

It's a wider problem, but I agree a separate report would be better. Nobody commented yet though.

[ghost]

For the record:

Transifex supports custom variable delimiters now. I've added the <%= name %> pattern to the list of placeholder delimiters for JSON variables - which is used by Transifex for automatic checks. Transifex will return error and won't let a translator save a string if a variable from the original string is missing in the translation.

In other words, translators (including reviewers) won't be able to omit a variable from now on.

I believe it will be better to have this check and corresponding errors. In case a translator needs to omit a variable, a Transifex admin can turn the rule off, make the changes needed and then turn it back on.

As of now, Transifex doesn't provide a way to perform this check on existing translations, so the strings with errors will remain as they are, until they are manually found or until Transifex introduces this feature.

More info about placeholder delimiters in Transifex docs: Translating HTML Content

A bit more info about the new features of Transifex UI can be found in the blog post

About what I've done from the technical point of view: Checking for custom variables