Kivy: Rendering Hindi(Indian) Language Wrong in Kivy for Android App

[kriyanation]

Software Versions

• Python:3.7
• OS:Android
• Kivy:1.11
• Kivy installation method: P4A

When I render the following text with the google NotoSansfont for Devanagari script in kivy android, I get it rendered wrongly. I have seen similar messages in SO for other languages and it says use pango text provider but it also mentions it is not available for android. Is there a possibility to render the font correctly in android in a kivy App. This will be a make or break for using kivy as the app development platform for us as we need to support Indian languages in our mobile app.

Example Text Expected output ->

Example Text Rendered Output ->

This happens in all the controls - Label and TextInputs and we have tried different ttf files - NotoSans, Akshar, atleast 10 other font files.

The only font which works correctly is GNU Unifont for us but that is because it has all the glyphs within and the problem we have is it does not look good and readable although that is now our current and only option.

Can we get some help here please?

example code:
from kivy.app import App
from kivy.uix.image import Image
from kivy.uix.label import Label
from kivy.uix.boxlayout import BoxLayout
from kivy.uix.button import Button

class MainApp(App):
def build(self):
label = Label(text="अंतरिक्ष", font_name="notosans.ttf", size_hint=(0.5,0.5),pos_hint={'center_x':.5,'center_y':.5})
layout = BoxLayout(padding =10)
layout.add_widget(label)
return layout

if __name__=='__main__':
app = MainApp()
app.run()

Font file link : https://fonts.google.com/specimen/Noto+Sans?subset=devanagari

[trivedigaurav]

Let me know if I understand it correctly, the notosans font has the 'क्ष' glyph but is not displayed when you use it for the kivy label? I will try looking at this issue if that is the case.

[kriyanation]

That is a minor problem or a problem we can live with as the 'क्ष' glyph is not used very frequently in words. The major problem is the रि being rendered out of sequence as र and the ि glyph coming after the letter. The sequence is the problem. This is happening for other Indian languages as well like Tamil and Bengali . Example issue -> https://stackoverflow.com/questions/63047020/kivy-isnt-showing-bengali-unicode-character-properly

[trivedigaurav]

Thanks, that stack overflow question is helpful. There is long discussion on this here: https://github.com/kivy/kivy/issues/2669. This issue is still open.

A possible solution is here: https://github.com/kivy/kivy/issues/4902

[kriyanation]

According to #4902 , it suggests using pango textprovider which is possible in linux but I do not think it is possible with Android or the kivy packaging provides a means to use the renderer in Android using Python for Android. So do we conclude the Kivy does not support Indian languages in Android and move ahead?

[akshayaurora]

@kriyanation is your problem specific to Hindi ?

[kriyanation]

From an immediate requirement point of view yes, but the problem of wrong sequencing of glyphs is happening in many of the Indian languages as mentioned above.

[akshayaurora]

Can you try Arabic text reshaper ? https://github.com/mpcabd/python-arabic-reshaper I think I tried that once and it seemed to solve a lot of issues for me with Hindi too. Can not remember of it works with re specifically. Trying it out now to see if it works

[akshayaurora]

If arabic text reshaper works then you can simply include it as is cause both arabic text reshaper and python-bidi are pure python and should work as is if you include them in your app dir

[kriyanation]

I tried your suggestion . Here is the code with the Arabic reshaper. It seems to render the text in the same way. Hope I used it in the right way, please check from your side as well.

from kivy.app import App
from kivy.uix.image import Image
from kivy.uix.label import Label
from kivy.uix.boxlayout import  BoxLayout
from kivy.uix.button import Button
import arabic_reshaper

class MainApp(App):
    def build(self):
        text = "अंतरिक्ष"
        reshaped_text = arabic_reshaper.reshape(text)
        label = Label(text=reshaped_text, font_name="notosans.ttf", size_hint=(0.5,0.5),pos_hint={'center_x':.5,'center_y':.5})
        layout = BoxLayout(padding =10)
        layout.add_widget(label)
        return layout



if __name__=='__main__':
    app = MainApp()
    app.run()

[kriyanation]

btw the issue is not only with "re" it is with all alphabets with the ि extension. Examples Words below - In effect the language becomes unreadable overall because of this problem.
कवि | तकिया | मिलन
कविता | तिनका | मिला
कितना | तिल | मिलाना

[akshayaurora]

ok, I tried

import kivy.app
import kivy.uix.label
import arabic_reshaper
import bidi.algorithm
class TestApp(kivy.app.App):
    def build(self):
        reshaped_text = arabic_reshaper.reshape("अंतरिक्ष")
        bidi_text = bidi.algorithm.get_display(reshaped_text)
        return kivy.uix.label.Label(text=bidi_text, font_name="/Users/quanon/Library/Android/sdk/platforms/android-28/data/fonts/NotoSansDevanagari-Bold.ttf")

testApp = TestApp()
testApp.run()

Got the output:

From what I understand the problem is of wrong shape/positioning rather than of font fallback, correct?
If so, we should be able to adapt arabic text reshaper or python-bidi for our purpose?
Looking into it a bit ...

[kriyanation]

Yes, positioning primarily is wrong. If that can be done it would mean the world to us in terms of moving forward.

[akshayaurora]

I am looking at https://github.com/python-pillow/Pillow/pull/2576/files this seems to have added support for reshaping to pillow directly which would mean we should be able to just use pillow text provider, though just using KIVY_TEXT=pil does not seem to fix anything for me… needs further investigation.

Update: related issue for reference https://github.com/python-pillow/Pillow/issues/1089

[kriyanation]

Yes, I tried the pil textprovider as well on Android, the text provider changes succesfully but the rendering remains the same.

[kriyanation]

While I am totally a noob with respect to font rendering aspect and just an application developer, I read in other posts that harfbuzz has capability to render complex fonts.
I see when the p4a build is running for android there is now a freetype dependency for my application after I added the arabic-shaper and in the build log I see something like compiling freetype with harfbuzz=No . Would changing it somehow to harfbuzz=yes help us? Just a random observation, you can ignore it if it does not make sense.

[akshayaurora]

some more reference https://forums.libsdl.org/viewtopic.php?p=48243
This patch seems like a good thing to integrate in p4a sdl2_ttf as a patch….

Update, p4a already includes harfbuzz. So we should just need to update sdl2_ttf to latest and to enable harfbuzz at compile time and have font reshaping. Theoretically this should then be able to be enabled by default on all platforms as sdl2 is the default text provider.

[trivedigaurav]

This is a good discussion. I am also leaving a reference to Pillow issue about this: https://github.com/python-pillow/Pillow/issues/3191

[kriyanation]

Any suggestions for a quick fix? Can I do something in my dev system which is ubuntu where my p4a build is running to try things out? To me p4a is a blackbox - so if you can help me with some instructions I can try it out in my system.

[kriyanation]

Any update on this please? We would need to make a decision based on this by end this week. Kivy meets all our requirements and is a pleasure to develop on, but this can be a blocker from our business perspective. So any help and exploration ideas will be useful.

[trivedigaurav]

I asked on the Sanskrit programmers Google group: https://groups.google.com/g/sanskrit-programmers/c/I57qkxqT-q0/m/EQ69f_TKAAAJ. Hoping to get some more response there.

[kriyanation]

Looks like this is not priority for the Kivy support or development. We shall be moving to native development this week. Multi Language support is now a de-facto requirement for software frameworks these days. Would be good if you can prioritize it in the future for developers to use it more.

[matham]

@kriyanation it's important to keep in mind that none of the kivy developers are paid to work on Kivy and that we work on Kivy in our free time and according to our own interests. So generally it's considered rude to push the developers, who are gifting you their time, to meet any business deadlines as it will probably be counter productive.

If a feature has not been implemented, it means no one has so far found the time or interest to implement it, whether devs or users. The fastest way to get a feature, especially if you need quickly it for business purposes is to either implement it yourself and find a core-dev also interested in the issue to help you get it in, or to wait if a core-dev shows interest. But if that doesn't work for you then open source software may not match your requirements.

[kriyanation]

Fair Enough.Thanks.