The-turing-way: Guidelines for translating the Turing Way into other languages

[KirstieJane]

Summary

I'm chatting with @minnieho1115 & @sparkler0323 about translating some of the content in the Turing Way in to Chinese. It makes sense for us to capture a few guidelines about what that would look like!

• Might need to ask questions about the content in order to translate it effectively - that is a feature not a bug - might need to improve the english language version before translating

What needs to be done?

• [ ] Where would the files live? How would we have a translated version? Different branch? Different repository (prefer not)? Different folder?

Who can help?

• Anyone who wants to help translate the Turing Way into other languages 🌏 🌍 🌎 💖

---

Updates

No updates so far

[tonyyzy]

Would be happy to help with Chinese translation 😊

[KirstieJane]

Awesome @tonyyzy! That would be so great!

I need to have a chat with @choldgraf and maybe @emdupre about structing having translated versions of the book but once we've got the location figured out then it would be amazing to have translated pages!

Is there a page you'd like to start with?

[tonyyzy]

Hi @KirstieJane!

Yes, I guess structuring different language versions efficiently is very important. It would be nice to have a system notifying us when the English version's paragraph has changed then ask us to review/modify the translated version. Or even some plugins that map the original paragraph to the translated paragraph to reduce duplication of files...just some ideas.

I have never done any formal translation work so I think some guidelines regard wording in addition to the contributing guidelines would be really helpful! I will start with Introduction and Reproducibility chapters. Can't make any promise of progress but holiday season should give me more time to work on this 😀

[choldgraf]

I know that Jupyter has used Transifex for this before: https://www.transifex.com/ maybe that's worth looking into? I'm not sure how it works though...

For Jupyter Book and translations, you could either have top-level folders in your content folder that correspond to languages (e.g. /en/, /fr/, /es/) and have multiple copies of content inside. Or, you could host different languages in different branches, and then configure your domain to host each branch at a different sub-domain corresponding to different languages

[tonyyzy]

@choldgraf

I know that Jupyter has used Transifex for this before: https://www.transifex.com/ maybe that's worth looking into? I'm not sure how it works though...

Just saw that Electron uses Crowdin as their internationalisation solution https://crowdin.com/. Looks very similar to Transifex. Although they both seem to offer free open-source plan, don't know how open-source friendly they are.

[sparkler0323]

The transifex and the crowdin all look fine, I will be very glad to translate some chapters of the Turing Way into Chinese when you decide one.

[tonyyzy]

Some notes from collaboration cafe 18/12/2019 with @KirstieJane:

• The Carpentries does an excellent job, we can follow their practices
  
  
  
  • https://carpentries.org/blog/2019/11/translations-in-japanese/
  
  
  • https://carpentries.org/blog/2019/01/python-ecology-es/
  
  
• The Carpentries' Japanese translation effort: https://github.com/swcarpentry-ja/i18n
  
  
  
  • They use GNU PO-files to store the translated text
  
  
  • Their toolchain works fine with CJK characters
  
  
  • This allows easy tracking of translation text and modification when the original text is updated
  
  
• @dpshelio helped the carpentries setup the process, maybe worth having a chat with him
• May still look into more 'commercial' solutions like https://www.transifex.com/ (Jupyter uses) and https://crowdin.com/ (Electron uses)
  
  
  
  • They offer free account for open-source projects but don't have experience with how (well) it works
  
  
  • @sgibson91: crowdin is also used by the all-contributors bot that _The Turing Way_ uses as well: https://allcontributors.org/docs/en/project/contribute#translations
  
  

[sparkler0323]

Hello @tonyyzy , it is amazing that you have listed many solutions that enable us to translate the Turing Way. @minnieho1115 and I are trying to translate this book and have initially finished four chapters as a start (please see: https://github.com/sparkler0323/The-Turing-Way-Chinese-translation).

Maybe we could determine a platform that we can keep working on it?

[KirstieJane]

Hi @sparkler0323 - could you add a little explanation about why you're working in a new repository rather than a fork of the Turing Way? And why you've chosen a word document rather than markdown?

[tonyyzy]

新年好 @sparkler0323 @minnieho1115!

Apologise for slacking 😛 I'm leaning towards the PO-file solution like the carpentries one (open source for the win!). A quick search shows that VSCode has plugins for PO file though I haven't tried them out (https://marketplace.visualstudio.com/items?itemName=mrorz.language-gettext)

I'll have a play with the automation toolchains and report back by next week (feel free to have a try as well!) My goal is to set up the basics before the book dash then mostly do the translation work during the book dash.

[sparkler0323]

Hi @KirstieJane , this is a temporary repository and it's a good idea to folk the Turing Way. Yini and I are collaborating using word and we will be very glad to transform the work version into the Markdown once we have established the platform with @tonyyzy .

[sparkler0323]

新年好，很高兴认识你 @tonyyzy

Sounds a really nice solution! I'm not familiar with the VSCode but I will be happy to learn your pipeline next week 😄. I think it's very exciting to use automation toolchains instead of our current manual translation.

Once setting up the basics, we are also looking forward to knowing your suggestions on the translation style we have finished.

[sparkler0323]

Hi @KirstieJane @tonyyzy , I have deleted the previous repository and provisionally store a Markdown version in the following place: https://github.com/sparkler0323/the-turing-way/blob/master/book/Translation/Chinese_Chapter1_4.md.

@minnieho1115 and I will transform the word version into the Markdown version later.

[minnieho1115]

@tonyyzy :smiley:春节快乐！Thank you very much for your suggestions. Look forward to learning your pipeline next week. It would be suuuuper great to have automated tools.
Hi @KirstieJane :wink:, Ang @sparkler0323 and I will continue our translation work this week. We will go over the details of the translation again, especially the terminology.

[tonyyzy]

I'll track my progress in this cell:

1. po4gitbook works well for a flat structure (where all the markdown files are in one directory). Our content has sections for some chapters hence need to modify the scripts.
   
   
   
   • [x] [po4jupyterbook](https://github.com/tonyyzy/po4jupyterbook) is the new fork and will be updated to be suitable for this project.
   
   
2. Basic workflow:
   
   
   
   • po4jupyterbook update to update the pot files when a new commit is made (CI)
   
   
   • Translate by using the PO files (integrate with Transfix)
   
   
   • After translation and quality control, Transfix submits PR to update PO files with translation
   
   
3. Site generation changes:
   
   
   
   • Firstly compile the PO files to localised markdown files which inherit the /book/content folder structure
   
   -
   

[malvikasharan]

Hi everyone, Super glad to see so many interests in this part of the project.

I wanted to share The Carpentries English-Spanish translation GitHub repo, in case you haven't looked at it yet.

A group of people worked on it remotely by using a project management feature of the GitHub and maintaining a progress table. Rayna Harris has explained this process in her poster in this tweet thread

In their first-round, they did the following:

• A glossary and guidelines document for collaborators
• A style guide and template for translated lessons
• An example translated document

For the French translation, a different group used another process which I have requested the repo and more info.

Another example for the translated materials I really admire is by the A Byte of Python project that has been translated in many many languages and exists on GitHub. I did not find their how-to guide very helpful, but maybe their process exists in details somewhere. Here is a translation in chinese: https://bop.mol.uno/.

[tonyyzy]

Hi @malvikasharan, super pumped to have you here!

Thanks for sharing these, the guidelines will be very useful to get us started!

I guess the difference between our book and most of the other projects I've seen is our content is evolving much faster. So I think it should be a conscious decision to keep the translations in the same repo as the English version.

I've figured out most of the kinks in my fork https://github.com/tonyyzy/the-turing-way
I'll break down the process here, it helps me to get all the thoughts together too.

• po4jupyterbook converts all the Markdown files in the book/content directory into POT files. POT file contains the sentences in all the Markdown files within each directory. An example can be seen here
• Translation begins by copying all the POT files to PO files and append the language code to the filenames of the PO files. Then fill in the new PO files with translations. As shown here
• multilingual_make.sh is the script that forms part of the Netlify's Make process. The translations are found by adding the language code to the url. Here's my test example.
  original: https://competent-nightingale-4c1d36.netlify.com/binderhub/binderhub.html
  Chinese: https://competent-nightingale-4c1d36.netlify.com/zh_CN/binderhub/binderhub
  
  
  
  • The make script first calls po4jupyter and compile the translated markdown files from the PO files.
  
  
  • Make a new Table of Content (toc.yml) file by adding the language code to the beginning of the urls
  
  
  • Build the translated website with Jekyll
  
  
• When the content is updated, the POT files are updated with po4jupyter's update.sh. This script will merge the old PO files with the newly added content as well.
  
  
  
  • My thought is not to commit pot files to the repo, instead treat them as transient files and make the update step be part of the CI pipeline. This way PO files will keep track of the newest markdown files and if the PO files need to be updated, Travis can commit the changes directly.
  
  
  • However, this is not feasible if we want to use Transifix as an online editor which I will outline below.
  
  
• You mentioned establishing a glossary for collaborators which I think is crucial and Transfix helps us by indexing the translated phrases (if I understand correctly). So the next time the same technical term pop up it would remind you of the previous effort.
  
  
  
  • For Transfix to work, it needs to know the location of the POT files in the repo.
  
  
  • It then submits a PR once a certain level of translation is done. See example here
  
  
  • We can still do CI to keep POT files up-to-date but I haven't tested how Transfix would respond to that yet.
  
  
• If we keep Transfix out of the equation then I believe everything works nicely together now. The next step is the actual translation work and establishes the work choices.

[malvikasharan]

@tonyyzy this is amazing.

We should think of a way to record the process and quirks that you have mentioned and will appear on the way of setting this infrastructure. Maybe a standard change log or a document within your forked repo (or may be even the main repo) for process development update.

Where do you think I and others can help you with at this point?

Yini and Ang will be in the best position to help develop the glossaries that are very Turing Way specific (for consistency and translation purposes) and can add to the process development update from the translation side.

[camachoreina]

Hello! @crangelsmith and I would be interested to help with the Spanish translation. We will start looking at the documentation and guidelines already shared in this issue. Let us know if you have recommendations about what to prioritize and where to start!

[malvikasharan]

@camachoreina That would be awesome! What kind of support can we provide you to get started?

[tonyyzy]

@malvikasharan for sure, I will write documentation and a quick starter.
I've merged my commits into the main repo, under a new branch translation. I guess we will test things with this branch for a bit before merging into the master branch. I also made a request to install Transifex's GitHub integration on this repo, can you or @KirstieJane approve that when you see this message? ❤️

@camachoreina @crangelsmith awesome to have you on board, I will share the Transifex link with you once it's setup then you can join as collaborators.

[malvikasharan]

@tonyyzy Can you please walk me through the process of integration in one of the coming days over a call? I don't really know the Transifex integration and can use your help to understand this system better.

[tonyyzy]

Edit: Just saw the message on Gitter, will wait for organisation admin to approve then

@malvikasharan definitely, we can talk over it during the book dash? It's very new to me as well and I'm still figuring it out. I requested the Transifex integration to be installed on this repo, could you approve the request please?

[tonyyzy]

@camachoreina @crangelsmith @sparkler0323 @minnieho1115 Here we go! Please go to here then click "Help Translate". You can login with your desired account or register a new one. Then it will show you the project overview and you can select a language and start translating! Let me know if anything is unclear. I'm new to the interface too. Please restrain from reviewing your own changes – if you click "Save Changes" then let others do the reviewing and click "Review".

@KirstieJane @malvikasharan I've invited you two as I have your emails.

[malvikasharan]

@tonyyzy thank you sooo much for moving ahead with this. Learning so much from this process and your work already. 🙌

[sparkler0323]

@tonyyzy Looks great now! I just send a request to join the Chinese translation group. Looking forward to using this!

[tonyyzy]

@malvikasharan here's the quick starter guide. I'll add the implementation details when I have a bit more time :)
https://github.com/alan-turing-institute/the-turing-way/blob/translation/translation_guide.md

[malvikasharan]

You are a super start @tonyyzy <3 so excited to add this development in the report.

[dpshelio]

* @dpshelio helped the carpentries setup the process, maybe worth having a chat with him

Sorry I've not been able to get to this notification till now. I'm glad you are using pot files, as that would ease the work to keep it up to date. I'm seeing also that you are using Transifex, which is what we are also using on the Carpentries. However, besides Crowdin I'm also testing other platforms like weblate (open source) or zanata (OpenSource and supported by RedHat). The main differences are how you can benefit from translations from other projects that may have similar strings, but since all accept pot files you can move them from one to another without much difficulty (except that peopel will have to create accounts on them and familiarise with a different user interface).

Well done @tonyyzy for that guide! Feel free to ask me anything. I'm keeping all the info across different repositories in https://github.com/carpentries-i18n . I'm planning to document it all better and have a translations handbook.

[tonyyzy]

@dpshelio Thanks! Your carpentries i18n stuff was very helpful to get us started. Transifex works so far so good, we will test its PR function very soon. Kirstie talked about turning the translation guide into a book chapter for the Turing way once it's more comprehensive. I'm happy to contribute to your handbook too!

[psubias]

Hi @camachoreina and @crangelsmith, I could help out in the Spanish translation :-).

[crangelsmith]

Hi @psubias , thanks this is great. Please join the Spanish team in Transifex https://www.transifex.com/theturingway/theturingway/dashboard/, and maybe we can have a catch up at some point soon?. We are starting with the translation of the first this week.