Weblate: Group + mark translations as reviewed on import

[rvanlaak]

Steps to reproduce

Import project .xlf translations from .yml format via ./manage.py import

Expected behaviour

We migrated our .ymlfiles to .xlf, the translations are imported okay but we would expect the translations to be marked as reviewed.

• So, maybe we can add an extra parameter to mark everything as reviewed on import? Or maybe this should be the default behavior?
• How would the Weblate import handle xliff groups? Our .yml files were nested, which is a really nice way of grouping, and if Weblate is able to handle these groups we could start using that to group the translations per page.

Actual behaviour

The translations are marked as "need review".

Server configuration

Latest master

[nijel]

You mean import_project?

Anyway in XLIFF you need to have approved="yes", otherwise the string is considered as for review.

[rvanlaak]

Okay great, then I'll add that to the xlf export script. See https://github.com/lexik/LexikTranslationBundle/pull/198 for that PR.

What about the grouping? This might be related to #263 ?

[nijel]

No that's grouping of components in Weblate. I think groups should work, but you probably can't use same IDs in different groups (I've never tested this, but I think this won't work properly).

[rvanlaak]

Okay that's clear, knowing that it never was tested already is a start :+1:

Maybe @phlax can clarify what the functionality of xliff grouping within Weblate could provide? Is grouping meant to group small sets of translations? Like having translations per page?

[pschneider]

Anyway in XLIFF you need to have approved="yes", otherwise the string is considered as for review.

Hello,
is there a way to disable this behaviour? (I've already tried disabling all quality checks for this component) In our company we do not need this flag but it would still be great to see "not translated" translations counting into translation progress of the component.
Unfortunately the Xcode XLIFF export does not respect the approved="yes" attributes but overrides the file completely instead. Unfortunately I haven't found a way to automatically merge a Weblate file containing the approved attributes with an Xcode merged file. :-1:

Any help appreciated! Thanks

[rvanlaak]

My guess would be that the XLIFF standard states that approved should be yes, and weblate (read: translation-toolkit) follows this. Would be great if more tools (same for PHPStorm) would start supporting these attributes.

[pschneider]

Yes, I definitely blame Xcode here too. I've managed to set up our Project using the strings files but it's way more effort since there are multiple and we need to set up a separate component for each file. Therefor XLIFF would be great if we could just see "not translated" percentage The trade-off not having quality checks would be fine for us (at least for now). Maybe it will take Apple years to actually implement XLIFF completely. ;/

This is why I thought this Issue would be a good place to ask if it is possible to deactivate just the "needs review" (fuzzy) mechanic. (mark every filled translation as reviewed on pull for example like it works when importing manually)

[nijel]

Probably we could add option to ignore this flag from translate-toolkit, however I'm really afraid having too much of such tweaks increases code complexity...

[rvanlaak]

Changing the behavior of the xliff format is something we shouldn't do? Maybe it's better to add bulk actions to mark all translations as 'approved'?

[nijel]

That's pretty much equivalent to processing the file on every change. I'm not sure if something what has to be done repeatedly and manually is a good solution...

[nijel]

You can now achieve this by adding skip-review-flag flag to component. It will then completely ignore this attribute when loading files.

[forceunwrap]

Hey @nijel. https://github.com/WeblateOrg/weblate/commit/56960fb11591d982382ccce5f40d365c1a968024 removed the skip-review-flag. Do I get this right that this is now the default behavior? If I understood it correctly you would now have to opt-in by using an addon (https://docs.weblate.org/en/latest/admin/addons.html?highlight=addon#flag-new-translations-to-need-edit) if you want new target translations to be marked as needing review, right?

[nijel]

It's not needed anymore since Weblate does properly handle approved state now.

[PowerKiKi]

Weblate does properly handle approved state now

Could you please elaborate how Weblate is expected to behave with XLIFF various flags ?

The docs says:

If the translation unit doesn't have approved="yes" it will be imported into Weblate as needing review (which matches XLIFF specification).

But I tested that in Weblate 3.1.1 and translate-toolkit 2.3.0 with this attached file (strip the ".txt" extension) and got the following results, where "null" mean the absence of attribute:

| <target> exists | approved attribute | state attribute | Weblate show as translated ? |
|-----------------|--------------------|--------------------------|:----------------------------:|
| FALSE | null | null | |
| FALSE | null | translate | |
| FALSE | null | approved | |
| FALSE | null | needs-translation | |
| FALSE | null | needs-review-translation | |
| FALSE | no | null | |
| FALSE | no | translate | |
| FALSE | no | approved | |
| FALSE | no | needs-translation | |
| FALSE | no | needs-review-translation | |
| FALSE | yes | null | |
| FALSE | yes | translate | |
| FALSE | yes | approved | |
| FALSE | yes | needs-translation | |
| FALSE | yes | needs-review-translation | |
| TRUE | null | null | 1 |
| TRUE | null | translate | 1 |
| TRUE | null | approved | 1 |
| TRUE | null | needs-translation | 1 |
| TRUE | null | needs-review-translation | 1 |
| TRUE | no | null | 1 |
| TRUE | no | translate | 1 |
| TRUE | no | approved | 1 |
| TRUE | no | needs-translation | 1 |
| TRUE | no | needs-review-translation | 1 |
| TRUE | yes | null | 1 |
| TRUE | yes | translate | 1 |
| TRUE | yes | approved | 1 |
| TRUE | yes | needs-translation | 1 |
| TRUE | yes | needs-review-translation | 1 |

From that table it is very obvious that a unit is considered translated (and not needing edition) if and only if a <target> element. Everything else is entirely ignored. AFAIK that goes against XLIFF specification, and more importantly it does not match Weblate docs.

So again, @nijel could you please elaborate if what I see is what you expect ? or is there a bug or misuse ?

Also, on a sidenote, I tried to solve this with addons, but I failed too. I can successfully configure the list of addons via WEBLATE_ADDONS. Remove them all, or add them all, change the list in the GUI. But whatever I do the GUI only show "Language consistency", "Statistics generator" and "Component discovery" and nothing else. SourceEditAddon and TargetEditAddon are never available. What am I doing wrong ?

[nijel]

Weblate will only get translated and approved states from xliff. Note that the approved state won't be used if you don't have enabled reviews on the project level, so you end up only with translated state.

The unit is approved iff there is approved="yes". Weblate does not support empty strings as being translated, so as long the target is missing, the string is considered not translated (I think there is issue for this). Also the approved state is not present in the specification see http://docs.oasis-open.org/xliff/v1.2/os/xliff-core.html#state.

There could certainly be some more fine grained mapping of states (probably most of the needs-* states could map to Weblate needs edit, possibly after making it more fine grained as described in https://github.com/WeblateOrg/weblate/issues/2240).

[PowerKiKi]

I realized that I made a mistake by replying on this thread, instead of crating a new one, because my issue is not when importing a file via the GUI, but when pulling from git. I believe your answer only apply to file import and not for git pull, is that correct ?

Could you please confirm what is supposed to happen when git pulling ?

PS: I was aware of the absence of a approved state in the spec. I was just desperate to try anything I could think of ;-)