Mimesis: Check correctness of all data for all locales.

[lk-geimfari]

We have the support of 33 languages and it would be great if native-speakers of one's will check the correctness of data for his own language.

For example. I'm Russian and I'm sure of the correctness of the data for this language. But we also want to be sure of the correctness of German (de, de-ch), Italian (it) and other languages.

Checked locales:

• [ ] cs
• [ ] el
• [ ] es-mx
• [ ] es
• [ ] et
• [ ] fa
• [ ] fi
• [ ] hu
• [ ] is
• [ ] ja
• [ ] kk
• [ ] ko
• [ ] nl-be
• [ ] nl
• [ ] no
• [ ] sv
• [ ] zh

[ZeFifi]

Hi @lk-geimfari ! I'm confused: could you tell me which files need to be checked? I could help with the french ones ;)

[lk-geimfari]

@ZeFifi All files related to your locale, namely: mimesis/data/fr/.

[ZeFifi]

@lk-geimfari Ok cool, I'm gonna take a look at it, I saw some typos ;)

[lk-geimfari]

@ZeFifi Thank you very much!

[ZeFifi]

Everything looks OK now :)

Please not that in the text.json file there's this sentence, in english: "This is the first age that's ever paid much attention to the future, which is a little ironic since we may not have one.". Would you like me to translate it or is it meant to remain in english since it's a quote?

[lk-geimfari]

@ZeFifi In fr all data must be in French. If you can find more data in French then update it! It would be awesome!

[ZeFifi]

Sure, I'm gonna recheck all the JSON ;)

[ZeFifi]

@lk-geimfari It's OK for the french translations now ;) I submitted my last pull request! You can check the FR box ;) Have a great day, it was cool to contribute to your project!

[lk-geimfari]

@ZeFifi Thanks!

[ar4s]

@lk-geimfari I will check for pl

[ChloeAnn]

Hi, I would love to contribute, I will check en-gb.

[lk-geimfari]

@ChloeAnn Okay, thanks!

[wbkoetsier]

Hi, I started checking the Dutch data and I'm not sure how far I should go in correcting.

I translated some Belgium Dutch sentences to Dutch, consider for example the difference between 'Zijt gij al klaar?' (Belgium) and 'Bent u al klaar?' (Netherlands), or change 'tas koffie' into 'kopje koffie'. The sentences seem allright, I skimmed.

But then the words. I'm not sure where they came from, but I found many:

• offensive/rude words, many of them far beyond rude, I shall not give examples :anguished: :disappointed:
• English words
• English names
• abbreviations such as 'ls'

As for the 'bad words', I think it's okay to include some swear words, but there should be a limit to their offensiveness. A great deal of those are considered incredibly rude and offensive, even among those who like to use strong language. I'm all for making the 'bad words' a little more gentle. Or maybe we don't need a bad words section?

@lk-geimfari Can you advise?

Mean time, I didn't get passed text.json yet...

[lk-geimfari]

@wbkoetsier If you want to know why we have added swear words then I can explain. It's for testing comments, forums and i.e. Suppose you need to check whether your application skips comments with swear words.

You should not write bad words manually because mimesis will generate them. I understand that bad words sound very rude, but they sound so, only if they are addressed to someone what we do not do and did not plan.

[lk-geimfari]

@wbkoetsier Therefore, they should be most popular bad words in the current language.

[wbkoetsier]

I can see some massively offensive stuff in the EN 'bad words' as wel... OK, I'll leave them unchanged, I understand the purpose.

But what about the 'normal' words? Are there supposed to be rude words in there, and what about all the English words and names? Should I clean that up, or leave it as is?

[lk-geimfari]

@wbkoetsier There two types of words: normal and bad. What words there should be clear from the name.

[pimuzzo]

@lk-geimfari I can check for Italian

[lk-geimfari]

@pimuzzo It would be awesome! I have added it by myself and I'm not sure that all data is correct, because, unfortunately, I don't know Italian. Thank you!

[lucasmarcel]

@lk-geimfari , I just looked at the French person.json and the values for 'academic_degree' are incorrect, yet this issue is closed. I could help but I understand if the data is considered final at this point.

[lk-geimfari]

@lucasmarcel Can you fix it, please? It would be really helpful.

[lucasmarcel]

Will do but bear with me, it'll be my first PR, ever :)

[lk-geimfari]

@lucasmarcel It's easy, as you'll see.

[Anphisa]

Hi @lk-geimfari!
I started checking the German (de) files and have some questions as to what the output should look like exactly.

• In most of the lists, words and phrases occur more than once and are often repeated 2 or 3 times below each other. Should each entry only appear once?
• In e.g. states and weekdays, there are abbreviations and long-form versions of words. (Mon / Monday). Should abbreviations and long-form versions be in the same order?
• Should bad words only be in the text.json/bad_words or also in builtin.json? There are (few, but pretty bad) ones in builtin.json
• In person.json: Nationality/Language: Should nationality be the form in "A russian"? Or is "russian", as in an adjective, also okay? It's currently like an adjective in most cases.
• In person.json: Names are partially uncommon/more English than German? (e.g. Adelaide and Aleshire would be very uncommon in Germany)
• In person.json: There are two titles for women. Frau is normal, Fräulein was last used in the 1950s. Can I take it out?
• All of the word lists and especially the alphabet lists are sorted alphabetically. However, sorting is wrong by German standards. Umlaute and ß should not be at the end, but in the list. (so it would be A, Ä, B, C, D, ...). Should I change this?
• In address.json: Is an Adress = Street Name + Suffix? Because suffixes are in the street names often also (suffix is e.g. "Straße" and a street name is "Schwedter Straße")
• There are a lot of sentences in text.json. I think they are scraped from news websites. A lot of them contain numbers in between and seem to be very technical in content, e.g. "Bildunterschrift noalk heute, 12:47 Uhr Die Bildunterschrift verweist auf die falsche Person.", means "Caption noalc today, 12:47 PM The caption is referring to the wrong person." I just looked at the Italian and Russian text.json and they seem much more concise and normal-language like than the German one. What should be the content of text?

Sorry for the many questions. Let me know which issues are important!

[lk-geimfari]

@Anphisa First of all thanks for your interest.

Let me answer your questions:

1. They should not repeat, repeated words must be deleted.
2. An order does not really matter, what is matter is correctness of abbreviations.
3. The bad words absolutely not acceptable in builtins.
4. It must be something that can be used in a context like this Adam is a German, Vadim is a Russian (of course in German)

Much clear example:

>>> from mimesis import Person
>>> person = Person('de')
>>> person.full_name()
'Immanuel Kant'
>>> person.nationality() # returns nationality of Kant.

# How it works in English:
>>> person = Person('en')
person.full_name()
'Loe Tolstoy'
>>> person.nationality()
'Russian'

In Russian it looks like this:

>>> person.nationality(gender=Gender.MALE)
'Немец'

>>> person.nationality(gender=Gender.FEMALE)
'Немка'

1. It's a good idea to delete all non-German names and add German names.
2. Yeah, you can change it to Frau
3. It would be awesome to order the alphabet correctly.
4. As you can see here and here we are using a shortened format of address for locale de, because if common cases streets contain Straße. You can change to be correct if right now it is not.
5. Context really does not matter. This text must be useful as a placeholder. You can change it too.

And finally thanks again for interest. This adorable language is really important for us. And it is important for us that the data be as accurate and useful as possible.

[Anphisa]

@lk-geimfari Thanks for your fast and helpful answer! I now made those changes.

1. I deleted the repeated words in de and noticed that there are some repeats in the Russian files as well (at least in the professions). I guess it doesn't matter because they each occur exactly twice, so the probabilities to get one of them won't be affected. But maybe all of the files should be checked for duplicates.
2. Okay!
3. I removed the bad words from builtin.
4. I changed the nationality from Deutsch to Deutscher/Deutsche (so from неме́цкий to не́мец/не́мка)
5. I removed the very non-German names. I didn't add any, because there are still a lot left.
6. Okay!
7. I changed the alphabet order to reflect "a" = "ä", one possible correct order.
8. I left the street and suffix as is, as the suffixes are not used. They are correct.
9. I will not touch the many lines in text.json for now :)

And what an adorable language it is. I found a lot of very interesting names in the person.json and feel very inspired now by all the Germans I can generate on my command line.

[lk-geimfari]

@Anphisa Gut! Looking forward to your PR.

[lk-geimfari]

@ceccoemi Hey! As I can judge by your profile you are from Italy. Is it right? If you are, what do you think about the correctness of data for locale it? It would be really awesome to hear your opinion about it. Maybe you can improve data for locale it somehow.

[lk-geimfari]

@ceccoemi Friendly 🏓

[ceccoemi]

@lk-geimfari I'm sorry I didn't see the first message. I will work on it locale as soon as I can.

[tangram]

Hi there. I've had a look at the Norwegian (no) locale, and noticed lots of things that are outright wrong or could be improved. I'd be happy to work on it.

[lk-geimfari]

@tangram It would be awesome! Thank you very much! I'll wait for your PR.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

Closing this issue automatically because it has not had any activity since it has been marked as stale. If you think it is still relevant and should be addressed, feel free to open a new one.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

Closing this issue automatically because it has not had any activity since it has been marked as stale. If you think it is still relevant and should be addressed, feel free to open a new one.