Smf2.1: grammar issue... 1 days

[illori]

when you have the max number of days to keep a draft set at 1 days and manually save a draft you get a nice grammar issue.

The contents have been saved as a draft, and will be accessible from the Show Drafts area of your profile. Please note that all drafts will be automatically deleted after 1 days.

also in the admin panel it says days :P

[Arantor]

This sort of thing requires a more comprehensive solution than just day vs days since other languages don't work the same way.

[Yoshi2889]

Create a function named "amountString" or something which solves this?

[Arantor]

I proposed a basis for this some time ago in the language board, until someone got all shouty about how I was 'stealing their code'.

[colinschoen]

Can you link me to this, @Arantor ?

[XinYenFon]

Don't want to sound like final voice here but maybe its best to fix this issue only for English (day/days) and be done with it for this version.

Not the mentioning SMF has 79 language in its base (some of them not active), IMO this is very big & time consuming job for this version.

Applies for: #2020 #2324 #2326

[wintstar]

It is some time passed, even versions. Here I see a similar problem https://github.com/SimpleMachines/SMF2.1/pull/4830.

Don't want to sound like final voice here but maybe its best to fix this issue only for English (day/days) and be done with it for this version.

And I don't want to be a nagging voice. Therefore I would like to ask if there are now thoughts / solutions to fix this plural problem for other languages? I think SMF is not only used in english.

[jdarwood007]

The problem here is very complex. I just found Russian for example has different endings for 1, 2 and 5.

• How can we do this in the simple fashion that translators can understand?
• How can we handle it for languages like english that only need the difference between singular and plural and other languages that need 3 or more different translations depending on the number?
• Are we only trying to tackle numerical issues here (Days, posts, topics, members, etc)?
• Do some languages have different handlings if a different word (using a unreal example but lets say a language had 5 different suffixes when using the word "post" but only 3 when using "member")
• By handling just plural usage, how many languages are we getting right versus how many would still be wrong?

[wintstar]

I've been thinking about it for quite some time. I also have some ideas that are not difficult to implement. I'll prepare something and post her tomorrow.

[Arantor]

Just going to leave this here: https://www.simplemachines.org/community/index.php?topic=511780.0

(Ignore page two of the thread, it contains nothing of use or relevance.)

[jdarwood007]

That actually could work. Although some languages may have multiple repeated entries.

Also not sure how it handles if a language has repetitive endings for numbers. Don't really have a good example for this. But like say all numbers for 21 to 29, they are "twenty" + number (i.e. twentyone). I don't know if any languages have issues like that which would be present with this issue. Although I guess a way around that would be do to a 'n#' construct as well.

[Arantor]

There wasn't a good construct for that - but it's a perfectly workable starting point.

[wintstar]

Can one of the options below be realised?

$txt['you_have_new_msg'] = array(
  0 => 'You have no new messages',
  1 => 'You have 1 new message',
  'n' => 'You have %1$s new messages',
);

$txt['you_have_msg_0'] = 'you have no messages';
$txt['you_have_msg_1'] = 'you have <a href="<URL>?action=pm">1</a> message {new}';
$txt['you_have_msg_n'] = 'you have <a href="<URL>?action=pm">%1$s</a> messages {new}';
// The {new} construct is used to add the (x new) area in a language-dependent manner, using unread_pms, as below.
$txt['unread_pms_0'] = '(none new)';
$txt['unread_pms_1'] = '(1 new)';
$txt['unread_pms_n'] = '(%1$s new)';

These are examples from the linked topic. For example for holidays it could be like this:

$txt['calendar_prompt'] = array(
    0 => 'Holliday',
    1 => 'Holidays',
    'n' => '%1$s Holidays',
);

[Arantor]

Sure it can, it was even built and tested elsewhere. It wasn’t a hypothetical example I was proposing, which is why it drew the ire of certain people by page 2.

[wintstar]

I can't understand the excitement on page 2, because other forum systems have had decades this construct.
phpBB:
https://github.com/phpbb/phpbb/blob/master/phpBB/language/en/common.php#L419-L421
Example number of posts

    'MESSAGES_COUNT'        => array(
        1   => '%d message',
        2   => '%d messages',
    ),

There is no patent or license on it. Inserts something similar? It will be too late for RC1, but for the release version it would still be time to add something similar.

[Arantor]

The excitement of page 2 was because I had the idea (without seeing it in other systems), applied it in one of the forks and then was branded a traitor by the other author of said fork for giving it back to SMF after 3 years of not being really an SMF user.

Said author has done his level best to erase me from said project’s Git repo too.

You’ll also note that my example is dated before your example by some months ;)

[wintstar]

You’ll also note that my example is dated before your example by some months ;)

These are examples from the linked topic.

They are your examples. It would be great if it could be implemented.

[Arantor]

I was comparing my examples from 2013 to your links to phpBB from 2014 ;)

You could ask a certain person if you can have my code from when I wrote this in 2013, but honestly it’s not a huge function to build, or to convert the language strings (though not sure if the language editor will cope or not)

[jdarwood007]

Well looking into your phpBB example they have this nice note: https://developer.mozilla.org/en-US/docs/Mozilla/Localization/Localization_and_Plurals

Which is exactly what I needed to know. The 3rd form one shows just how complex we would have to be, but this could be done.

$txt['days_0'] = '%1$d days';
$txt['days_1'] = '%1$d day';
$txt['days_11'] = '%1$d days';
$txt['days_n'] = '%1$d days';
$txt['days_n1'] = '%1$d days';

Code logic is simple, it would look for a text string with a base of "days" and a suffix of the exact number (i.e, "days_3234"). Then if that fails, it looks for a base of "days" and a suffix of the last number (i.e. for 21 would look for 1) with a n prefix (i.e: "days_n3"). If that fails it falls back to looking for "days_n". If that completely fails, it would look for just days.

We would only need to use the language construct function when calling something that will be passing a number into it. Just like we do when we need to use sprintf.

This gets complicated for languages like the example for the 4th form. Lots of duplicated strings. But I don't see a better way. We could try to introduce a range function (i.e. "days_r1-9"), but that starts getting complicated and slow.

Thoughts?

[jdarwood007]

We could also implant those Plural rule definitions directly into the code as well. Only 18 different rules. A simple process to add that string into the index language file and then know based on that rule, how we should handle adding the suffix to the base.

[Yoshi2889]

Why don't we push this back as an issue for 3.0 when we're likely redoing all of the language code anyway?

On July 17, 2018 4:02:40 PM GMT+02:00, Jeremy D [e-mail] wrote:

We could also implant those Plural rule definitions directly into the
code as well. Only 18 different rules. A simple process to add that
string into the index language file and then know based on that rule,
how we should handle adding the suffix to the base.

--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/SimpleMachines/SMF2.1/issues/2020#issuecomment-405592932

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[illori]

might be better for a different version of SMF, we dont want too major of changes at this time.

[wintstar]

Many have been waiting for 2.1 for a long time, some are no longer waiting and have already moved away. This is an important feature for multilingual use. Many European users will then no longer wait for it.

[Yoshi2889]

We have more important things to focus on. 2.1 should be out the door ASAP, we simply shouldn't focus on these things right now, especially if it involves changing language strings and changing a core part of SMF.

On July 18, 2018 12:27:37 PM GMT+02:00, wintstar [e-mail] wrote:

Many have been waiting for 2.1 for a long time, some are no longer
waiting and have already moved away. This is an important feature for
multilingual use. Many European users will then no longer wait for it.

--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/SimpleMachines/SMF2.1/issues/2020#issuecomment-405885779

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[illori]

we have had this issue for years, it has been brought up a few times. i dont see it keeping people from using SMF.

[Arantor]

It’s not a major change but unless the plan is to have maybe 1 RC and go final, it could be implemented as a small improvement after RC1. It’s not like it would break a ton of things.

[Yoshi2889]

In my opinion we should focus on bugs instead of improvements by now. It's been long enough.

On July 18, 2018 1:40:13 PM GMT+02:00, Arantor [e-mail] wrote:

It’s not a major change but unless the plan is to have maybe 1 RC and
go final, it could be implemented as a small improvement after RC1.
It’s not like it would break a ton of things.

--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/SimpleMachines/SMF2.1/issues/2020#issuecomment-405901792

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Arantor]

I’d argue this is a bug that it doesn’t properly handle languages other than English. And almost every feature and bug can be respun in the other direction.

In any case SMF has precedents for adding new functionality in RCs: Curve (RC2), Load balancing UI and revamped buddy list UI (RC3), massively improved hooks and protected groups functionality (RC4)

[Yoshi2889]

Yeah but how long has 2.1 been in development while it isn't even classified as a major release (just a minor as in major.minor.patch)? 2.0 was a major release so many RCs and late features are expected there.

On July 18, 2018 1:47:30 PM GMT+02:00, Arantor [e-mail] wrote:

I’d argue this is a bug that it doesn’t properly handle languages other
than English. And almost every feature and bug can be respun in the
other direction.

In any case SMF has precedents for adding new functionality in RCs:
Curve (RC2), Load balancing UI and revamped buddy list UI (RC3),
massively improved hooks and protected groups functionality (RC4)

--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/SimpleMachines/SMF2.1/issues/2020#issuecomment-405903356

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Arantor]

I swear if half the energy that went into arguing against doing things were instead put into doing them, 2.1 would be done by now.

You’re seriously talking about blocking the implementation of something like a 10 line function with maybe a couple of dozen calls to it throughout the code base.

For a thing that has always been reported as a bug every time it has come up.

Had it occurred that this mentality of continuously arguing about things might actually be why 2.1 is so late?

FFS I should just give you guys a PR already, at least that would actually be productive rather than arguing about whether a bug is a bug or not when it clearly is.

[Yoshi2889]

Except it also requires work from every translator for every language SMF supports and depending on the way we implement this the LE might not even be compatible.

On July 18, 2018 2:04:09 PM GMT+02:00, Arantor [e-mail] wrote:

I swear if half the energy that went into arguing against doing things
were instead put into doing them, 2.1 would be done by now.

You’re seriously talking about blocking the implementation of something
like a 10 line function with maybe a couple of dozen calls to it
throughout the code base.

For a thing that has always been reported as a bug every time it has
come up.

Had it occurred that this mentality of continuously arguing about
things might actually be why 2.1 is so late?

FFS I should just give you guys a PR already, at least that would
actually be productive rather than arguing about whether a bug is a bug
or not when it clearly is.

--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/SimpleMachines/SMF2.1/issues/2020#issuecomment-405907230

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[wintstar]

Except it also requires work from every translator for every language SMF supports and depending on the way we implement this the LE might not even be compatible.

That's up to the translator. It is important that the requirements for correct translation into the national language are met.

[Arantor]

Ok, so because it requires a little work (but should be compatible since the LE already has arrays to deal with) we’ll just put it off again. It’s only been 4 years since this was reported, what’s another few years until SMF 2.2 has the same problem?

Now I remember why I forked multiple times... to get away from this eternal pessimism where everything is too hard and too much effort. So what if users have had to put up with this for years, that somehow legitimises not fixing bugs?

Vastly more effort has been spent on less visible issues in the last year that fewer users will actually notice, than it would have taken to fix this bug.

[frandominguez03]

Let's wait for a developer to leave their input. It's their decision in the end.

[Arantor]

Does providing a PR count? Because that’s input I can leave in the next couple of days. (And it’s not like I’ve never done this before for 2.1...)

At the time of the original proposal, I was on the dev team...

[wintstar]

It would be better to implement it for version 5. Until then, other translations are thrown overboard. Nobody will notice. Only a few believe in 2.1 that it will be released sometime. As Arantor wrote, there is simply too much discussion instead of implementation. But that's not the issue here. I think that it can be done with RC2, if you want it.

[Arantor]

OK so I have an implementation that I've been working on. My test case is the 1st, 2nd, 3rd, 4th shortenings you see in English - this requires 'st', 'nd', 'rd' forms for x1, x2, x3 numbers except for 11, 12 and 13 (because the 113th item is the correct form for example)

Total language strings needed:

$txt['ordinals'][1] = 'first';
$txt['ordinals'][2] = 'second';
$txt['ordinals'][3] = 'third';
$txt['ordinals']['x11'] = '%1$sth';
$txt['ordinals']['x12'] = '%1$sth';
$txt['ordinals']['x13'] = '%1$sth';
$txt['ordinals']['x1'] = '%1$sst';
$txt['ordinals']['x2'] = '%1$snd';
$txt['ordinals']['x3'] = '%1$srd';
$txt['ordinals']['x'] = '%1$sth';
$txt['ordinals']['x000'] = '%1$s (thousand)';

This isn't a case actually needed in SMF itself but it illustrates the problem - you need a form for x2, x3, x4, x5 in some languages and while this is duplicative, it's not tragically so.

The idea is that it tries to use the most specific version (e.g. exact match on 1 comes before x1, and if 13 were used, it would try 13, then x13 before the x3 before the x)

I've also thrown the random 'x000' entry in to test that the code is arbitrary length specificity rather than last two digits :)

Only downside is that the built-in language editor seems to choke on the array syntax but I'm not sure if that's the regex on it being finicky or not. But if this syntax seems sufficiently worth integrating I'll do a PR for both the core support and the language editor plus some of the affected entries. (I don't know what sm.org's LE will make of it unfortunately)

(And bonus points, it contains no code from the original source so no arguments over origin.)

[frandominguez03]

@MissAllSunday Any thoughts on this one? Pretty old issue.

[MissAllSunday]

Yeah, I'm in favor of including @Arantor 's idea. This is more a bug than a feature and as long as we keep track of all the changes translators need to do and offer them a concise and filtered list of things to do and how to do them we can make this work for Final.

[Sesquipedalian]

I've already made the necessary underlying changes to support this approach within SMF itself, including in SMF's built-in language editor. I've even updated creaky old LE to support it, too.

An early conceptual example of how this can be used is in the sentence_list() function I added to Subs.php a while ago. Basically, the language files describe the rules of the given language, and then that function formats the array of values according to those rules. I haven't written a function to format ordinal numbers yet, but the fundamental logic would be conceptually very similar. The same approach should also enable us to write a proper pluralizing function, once we get around to it.

In terms of the development roadmap, however, those more advanced implementations of this concept should be left until after 2.1. The functions will be easy enough to write, but the necessary changes to the language files will be wide ranging, to say the least.

[Arantor]

Honestly, the ordinal problem (as well as inappropriately compound strings) is more jarring than grammatically correct lists (even putting aside the question of the Oxford comma!)

I should point out that at some point I did even provide an implementation (and you can always get it from StoryBB’s code, it’s all BSD)

[Sesquipedalian]

True, but lists didn't require major revisions to the language files. I'm all in favour of the idea, as you know. My concern is only that I don't want to overburden our translators with more time consuming work when we are close to finishing 2.1 at last.

But I suppose I should let the lead translator weigh in on a matter that so directly affects his team, rather than deciding for him. So what do you think, @frandominguez03? Would you rather see this implemented in 2.1, even if it means a heavier workload for your team, or would you prefer for us to hold off until later? IIRC, there was some ambivalence about that when the question came up in internal discussion last year.

[Sesquipedalian]

FYI, the reason why I suspect that doing this correctly may require significant changes throughout the language files (as opposed to just adding a few arrays) is that the best practice for translation when dealing with plurals is to translate on the phrase or sentence level rather than translating individual words. This is because in most languages, changing the number of something in the sentence frequently changes the verb conjugation, noun and adjective declension, etc., throughout the sentence as a whole.

For example, an English speaker might think that handling plurals for the string "Delete 1 message?" only needs to substitute a construct like {%n_messages%} into the sentence, because in English the verb's conjugation is unaffected by the number of messages. In another language, however, the verb might very well be affected, which would therefore require a full {%delete_n_messages%} construct.

[Arantor]

This is not a new thing to note, it’s been known since at least I floated th previous iteration of my code in around 2013, and was understood amongst the translators even before that.

The problem is, just because list syntax is lower hanging fruit, it doesn’t change that it’s also significantly less visible than “delete 1 message” versus “delete 3 messages”.

The whole compound sentences problem is also not new and is only solveable by way of making new strings, and that’s something that’s worth doing sooner rather than later. Get the base support in, fix the worst cases in 2.1, fix the smaller cases in 2.2.

[Sesquipedalian]

and that’s something that’s worth doing sooner rather than later.

That's the (only) matter in question, and it is not one that can be decided by developers alone. We all agree on what the solution should be, but we need @frandominguez03, as the lead translator, to weigh in on whether sooner is better than later from the translation team's perspective.

My illustration of the scope of changes required was not an argument for or against any particular course of action. I was simply making sure that we are all equally aware of what will be involved.

[Arantor]

Were I in this situation, making this decision, my response would be to just do it and let the dust settle afterwards. I'm still bitter about the fact that the Q&A got upgraded the way it did (and backported to 2.0) to help mitigate the spam issues on sm.org and it never got actually used on the basis that getting any amount of translations was too hard, therefore why bother.

I am somewhat averse to waiting for others to make a decision at this point.

Though if I'm truly brutally honest, this whole debate is mostly academic for me since I have no plans to ship regular forums based on SMF any time soon, and I already started the groundwork for this in my realm which isn't regular forums anyway.

[Gwenwyfar]

I'm not sure I get what needs to be thought about it... Language strings are being changed and added all the time, and half the translation teams aren't active or don't even exist. The sooner you change it, the more likely anything will be translated when they feel like it/when they have time.

Besides, if the translators don't add anything, then the default that already exists will be used instead, as normal, no?

[frandominguez03]

My concern is only that I don't want to overburden our translators with more time consuming work when we are close to finishing 2.1 at last.

Get the base support in, fix the worst cases in 2.1, fix the smaller cases in 2.2.

I'm in favor of doing it. Thanks for all your input, but I think Arantor is right.