Language: Support grapheme cluster aware operations via a package

[mit-mit]

Several issues indicate a need for grapheme cluster aware string operations in Dart. The current support is insufficient as covered in https://github.com/dart-lang/language/issues/34; summarised here:

  String hi = 'Hi 🇩🇰';
  print('String.length: ${hi.length}');
  // Prints 7; would expect 4

  print('The string ends with: ${hi.substring(hi.length - 1)}');
  // Prints ���; would expect 🇩🇰

  print('Substring -1: "${hi.substring(0, hi.length - 1)}"');
  // Prints "Hi 🇩���"; would expect "Hi "

The present feature tracks solving this via a new package supporting common string operations in a grapheme cluster aware way. This has the downside that developers would have to "opt in" to use this API, but then also has the huge upside that this would be entirely non-breaking (as opposed to changing the current String API).

[mit-mit]

An experimental version of this is available in https://pub.dev/packages/characters. Sample APIs for the code listed in the previous comment:

  String hi = 'Hi 🇩🇰';
  print(hi.characters.length);
  // Prints 4

  print(hi.characters.last);
  // Prints 🇩🇰

  print(hi.characters.skipLast(1));
  // Prints "Hi "

[marcglasberg]

We're having serious problems with emojis, that this is suppose to fix. While non-breaking is usually good, in the real world all user input have to use this package. It's not only that developers would have to opt in, they would have to opt in always.

[mit-mit]

@marcglasberg we hear you. I'm not saying here that we are not considering breaking API changes on strings, I'm just saying that we're staging it, and the package (with its extension onto String) is phase 1. Can you give the package a try, and let us know how well those APIs work?

[ramsestom]

Thanks for this package. Haven't tested it yet but seems promising.
Just a semantic remark. You chose to call graphene clusters characters but it might be misleading to developpers as it could be interpreted as "Unicode characters" (=runes in Dart). Maybe the name glyphs would have been preferable

[mit-mit]

Marking "done" as the initial package shipped alongside Dart 2.7.

We'll continue to increment on the package. For feedback on the package, please use this issue tracker: https://github.com/dart-lang/characters/issues