Weblate: Support for Golang's i18n Formats

[panbanda]

Is your feature request related to a problem? Please describe.

I would love to see support for Golang's i18n support. I use hugo a lot and the yaml/json/toml i18n configs are great but they are unique to golang.

Describe the solution you'd like

Can we support at least go-i18n-json or yaml formats? It would be great to do toml formats as well but the low hanging fruit seems to be the json/yaml formats.

https://github.com/nicksnyder/go-i18n

- id: translation-key
  translation: translation goes here

[
  { 
    "id": "translation-key", 
    "translation": "translation goes here" 
  }
]

[translation-key]
other = "Translation goes {{.Here}}"

[nijel]

Preferably the support would be first integrated in the translate-toolkit, supporting it then in Weblate is easy.

Rant: Why does every toolchain come with own way of storing localization? There is already enough of them ;-).

[eighthave]

@panbanda does Hugo/Go really only support that one format? It looks like a bad format, without much support for very valuable things that PO and XLIFF support, like developer comments, character limits, etc. Seems to me the right solution here is to make Hugo/Go support a full featured, standard translation file format. Right now, it looks like they are just reinventing the wheel, and doing it poorly at that.

[eighthave]

@matrixcoffee and I will need to follow this, could you add it to the Linguine tracker?

[nijel]

I don't think it makes much difference implementing yaml or toml based format. And anything but toml doesn't seem to handle plurals, or am I missing something?

Actually is there some documentation for the formats? I was only able to find some examples, but really no specification...

[panbanda]

@nijel it does handle plurals but in a kind of (IMO) awkward way:

[readingTime]
one = "One minute read"
other = "{{.Count}} minutes read"

So if just defining something normally it would be "other" by default.

[panbanda]

@panbanda does Hugo/Go really only support that one format? It looks like a bad format, without much support for very valuable things that PO and XLIFF support, like developer comments, character limits, etc. Seems to me the right solution here is to make Hugo/Go support a full featured, standard translation file format. Right now, it looks like they are just reinventing the wheel, and doing it poorly at that.

I'm with you. I really enjoy Hugo for static sites but find the i18n stuff lacking. Maybe in the future I can build a lib that uses more standardized flow...

[nijel]

It's quite clear how plurals are handled in toml, my question was more about json/yaml, who are present only with id/translation in the examples.

[panbanda]

@nijel ah my bad yeah I understand. The test data there actually has the JSON and YAML formats if you're interested in seeing them. But TLDR: It's basically the same structured format if you can imagine turning the TOML into YAML.

https://github.com/nicksnyder/go-i18n/tree/master/goi18n/testdata

- id: person_greeting
  translation: "Hello {{.Person}}"

- id: my_height_in_meters
  translation:
    one: "I am {{.Count}} meter tall."
    other: "I am {{.Count}} meters tall."

# Also according to their testdata
person_greeting:
  other: "Hello {{.Person}}"

my_height_in_meters:
  one: "I am {{.Count}} meter tall."
  other: "I am {{.Count}} meters tall."

JSON:

{
  "person_greeting": {
    "other": "Hello {{.Person}}"
  },

  "my_height_in_meters": {
    "one":  "I am {{.Count}} meter tall.",
    "other":  "I am {{.Count}} meters tall."
  },
}

[eighthave]

FYI, the Hugo developers are considering supporting a standard format, so I nudged them in that direction: https://github.com/gohugoio/hugo/issues/5341#issuecomment-484286832 I think you all should too :-)

[uniqx]

@panbanda The example snippets you supplied seem incomplete. compare: https://godoc.org/github.com/nicksnyder/go-i18n/v2/internal/plural#Form

Is there any spec or understandable documentation about this format? The api-docs and examples in the project are hardly an ideal starting-point to figure out the requirements for implementing this format.

[Jibec]

Hi there,

as I doubt upstream will change any soon, I created a folder with the three file formats it exists for go-i18n: https://github.com/languages-in-floss/test

it does produce inconsistent results in weblate:

I currently consider Hugo to be the static website generator with the best multilingual support.
If something has to be done for golang, I suggest to go for toml support. It's the default in documentation and allows to preserve the current behavior of yaml and json parsers.

[nijel]

Adding support for YAML or JSON variants would be easier as these are formats already handled by translate-toolkit, it just needs to support additional variant of that. However, there is no TOML based format.

[Jibec]

Well, I'm not doing the job, anything you support is fine for me ;)

[nijel]

Is there some docs available for the JSON variant? That would be definitely easiest to implement for us (as in the end it seems to be just variant to Web Extension JSON). However what I've seen here are two completely different examples.

@panbanda in https://github.com/WeblateOrg/weblate/issues/2558#issuecomment-468923539:

{
  "person_greeting": {
    "other": "Hello {{.Person}}"
  },

  "my_height_in_meters": {
    "one":  "I am {{.Count}} meter tall.",
    "other":  "I am {{.Count}} meters tall."
  },
}

@Jibec in https://github.com/WeblateOrg/weblate/issues/2558#issuecomment-636429421:

[{
  "id": "tag",
  "description": "a piece or strip of strong paper, plastic, metal, leather, etc., for attaching by one end to something as a mark or label",
  "translation": {
    "one": "{{.count}} tag",
    "other": "{{.count}} tags"
  }
},
{
  "id": "table",
  "description": "an article of furniture consisting of a flat, slablike top supported on one or more legs or other supports",
  "translation": {
    "one": "Table",
    "other": "Tables"
  }
}]

[nijel]

I've made pull request based on my guess here: https://github.com/translate/translate/pull/4037. It would be useful to know if the used test data looks correct: https://github.com/translate/translate/pull/4037/files#diff-5b954663f244c2e1d32bce2961aa1104R41

[Jibec]

I am sure my format works as I tested it with hugo static generator. I'll check the other json format tonight.

I have no specific will to push one format or another, whatever works and have less impact in Weblate is fine for me.

[Jibec]

I confirm I couldn't make @panbanda 's json work with Hugo.

Error: add site dependencies: load resources: loading translations: "/home/jb/Projects/languages-in-floss.eu/site/themes/minimo/i18n/fr.json:1:1": failed to load translations: failed to unmarshal fr.json: invalid character '}' looking for beginning of object key string

And even if I change my json file to remove the translation key, it says:

Error: add site dependencies: load resources: loading translations: "/home/jb/Projects/languages-in-floss.eu/site/themes/minimo/i18n/fr.json:1:1": failed to load translations: unable to parse translation #0 because missing "translation" key
map[description:a piece or strip of strong paper, plastic, metal, leather, etc., for attaching by one end to something as a mark or label id:tag one:{{.count}} tag other:{{.count}} tags]

it looks like previous version of go-i18n supported @panbanda example in the past:
https://github.com/nicksnyder/go-i18n/commit/be3966fd39d744b0fce2bc71c79f42911cef7b7c#diff-96188962d1e3cc4dd2691f9bf6eef64d (I commented this commit to ask why example were removed, let's hope it doesn't mean they plan to remove support...)

the list of valid keys are the one from CLDR: "zero, one, two, few, many, other"
it also is part of goi18n tests: https://github.com/nicksnyder/go-i18n/blob/master/v2/goi18n/extract_command_test.go#L150

[nijel]

I've implemented the JSON support. In case somebody wants other variants as well, please open separate issue for that.

[github-actions[bot]]

Thank you for your report, the issue you have reported has just been fixed.

• In case you see a problem with the fix, please comment on this issue.
• In case you see a similar problem, please open a separate issue.
• If you are happy with the outcome, consider supporting Weblate by donating.