Qksms: Improve Strip Unicode feature

[sledgehammer999]

DESCRIPTION

I am using greek language. I am stuck with the 2.7.3 version because the 3.x.x series don't have the "strip unicode" option.
This option replaces unicode chars with their GSM equivalents so we can still send long texts while typing non-ascii characters.
It works fine for me on the 2.7.3 version. Why did you decide to hide it with commit https://github.com/moezbhatti/qksms/commit/ddf1cc169f5433e6fc643d3d220ac1cd4220a771 ?
It is possible to re-enable/re-introduce it?

This is a critical feature for me. It was the initial reason why I started using QKSMS instead of my rom's stock app.

Thank you.

[moezbhatti]

The strip unicode feature is implemented in the current release of the app (3.2.0)

[sledgehammer999]

@moezbhatti I am sorry but no. I just installed 3.2.0 from f-droid.I don't see an option for this in the settings. Do you mean "Strip accents"? I have enabled it but my texts are still split after ~60 chars.
Could it be that it is just an UI bug?

PS: I didn't actually sent the text. I just typed random greek chars until the counter appeared and for the 2nd part it started at 63 chars.

[sm4rk0]

I've just tested this scenario (3.2.0 from f-droid, too). It's sent without accents and the oparator counted it as one SMS. Seems the bug is only in presentation.

[moezbhatti]

Not a bug :) The app will save the text with the accents locally on your phone, it will only strip them from the text that's sent. I understand why this might be confusing though, I'll think about how I might be able to make this less confusing in the future

[sledgehammer999]

I should see exactly what the other person will see. At the very least show the real text right after it is send.
Also the character counter should be fixed too.
Finally, IMO, stripping the accents isn't enough. The greek alphabet in the GSM 7-bit alphabet is represented by only the capital letters. The small letters don't have an equivalent.

[moezbhatti]

I'll keep a checklist of things to do here

• [x] Store the same message that is sent, when stripping unicode
• [ ] Fix character counter inaccuracy when feature is enabled

[moezbhatti]

Finally, IMO, stripping the accents isn't enough. The greek alphabet in the GSM 7-bit alphabet is represented by only the capital letters. The small letters don't have an equivalent.

What do you think an appropriate solution for this would be?

[sledgehammer999]

What do you think an appropriate solution for this would be?

Since you're already manipulating the text by removing the accents, then go one step further and convert to uppercase.
In case you didn't know, wikipedia has a nice table of the 7-bit GSM alphabet here

[sm4rk0]

I believe it's not so trivial. Greek alphabet is written in Unicode, but GSM alphabet uses Latin letters for common ones and special characters for those that are unique for Greek alphabet. For example, when you type a lowercase alpha: α and convert it to uppercase, it would still be a special Unicode character: Α. There would have to be a translation table that would go from α and Α to A, etc.
Did you try what happens if you send a message with all uppercase letters longer than 100 chars? Does you operator charge one SMS or two?

[sledgehammer999]

@sm4rk0 I understand what you're saying.
I still have the old QKSMS 2.7.3 which somehow manages to do it. Does it use a translation/conversion table?
Unfortunately I can't tell by my operator whether a specific sms has been charged as 2, until I get the bill at the end of month...

[sm4rk0]

I've searched this repo for "Greek" in order to find a clue in code, but found this issue: #815. I believe we should close this one as a duplicate and switch to the older one.

[sledgehammer999]

Any possible progress in this? (I am still using QKSMS 2.7.3)

[moezbhatti]

Sadly no, not yet. I'll move this up to the backlog

[moezbhatti]

Some additional resources sent by another user via email

I like that only the "big" accents are removed. I find that about the reason https://stackoverflow.com/questions/7103990/sms-messages-non-ascii-characters-encoding
Maybe we can find a complete list, I will search more tomorrow.

I find the list of standard characters: https://en.m.wikipedia.org/wiki/GSM_03.38

If your option "remove accents" could become "remove non 7-bit accents", this would be awesome! And I know no other app doing this. :-)

[maverick74]

Hum, I just figure this feature only removes the accents if you need the space.

Pretty clever!!!
I was thinking it would always remove them, but it turns out it is actually more clever than a thought.