Coteditor: Word count is not accurate for non-English languages

[liuhantang]

This is a great editor, I really love it.
However, the word count for non-English languages seems to be not correct, e.g., Chinese.
For example, "加入中文字符" should be 6 words (or 6字 by Chinese) while the editor shows 4.
Will we have a fix in the future?

[1024jp]

Let me know your macOS version, CotEditor's version, as well as your language setting.

As for code, CotEditor lets a macOS framework count words by assumpting the text is written in the current system language.
On my current setting (macOS 10.15.7, CotEditor 4.0.0, English) at least, it counts "加入中文字符" as 3 words, namely 加入, 中文字, and 符.
Furthermore, I guess it is not wrong though I'm not sure since I'm not a native Chinese speaker.

IMPO, counting words in the languages that don't break words with space is difficult for a computer in general.
It is also helpful for debugging if you give me examples of macOS apps that count Chinese correctly.

[liuhantang]

Thank you for your reply. Here's my setting:
macOS version: Catalina 10.15.7
CotEditor's version: 4.0.0
language setting: simplified Chinese.
Currently, CotEditor correctly counts the number of characters. The problem seems to be how they define a 'word'.
In my opinion, one single Chinese character should be treated as a 'word' (in the sense of English words). So "加入中文字符" should be "加 入 中 文 字 符" 6 words. macOS's counting words framework seems to count phrases (which can consists of multiple characters) instead of words. They may treat '中文' as one word instead of 2. I tried to find a clue of how they do segmentation, yet the result seems to be somewhat random.
The goal is actually simple, just count each Chinese character as a single word. As far as I know, Typora does this job perfectly, even when you mix input English and Chinese.

[1024jp]

Thank you for your environment info and explanation.
I'll look for how it can be improved.
Let me have time.

[michelf]

To add another data point: Apple's Pages app seem to agree this text is 6 words.

Apple frameworks follow Unicode segmentation rules, using the ICU library underneath, which does split this text into 3 "words". You can try it here. On that demo page, if you choose "line" instead (for places where the line can be wrapped) it appears to work mostly as desired, except for empty lines that count as a word.

(Edit: I don't know Chinese and have no opinion on what should be counted as a word; this is only a technical suggestion on how to achieve the requested result.)