Godot-proposals: Complex text layout/shaping API structure

Created on 7 May 2019 · 6Comments · Source: godotengine/godot-proposals

Since CTL proposal was approved and before any GSoC coding begins, lets finalize appropriate CTL APIs structure and breaking changes,

1. Godot `String` API:

Current implementation:

Uses wchar_t type.
32-bit on *nix platforms (works as UTF-32).
16-bit on Windows, half-broken UTF-16 (without any support for surrogate pairs).

Proposed implementation:

UTF-16 (char16_t) on all platforms (primary because ICU API is UTF-16 based)
Encoding conversion and other missing or partially implemented stuff (case matching and conversion, localization related function like date/number formatting) can be implemented on the ICU base.

2. Text rendering API structure and abstraction level:

Current implementation:

Draw (draw_text, draw_char) and string size functions (get_string_size) in both Font and CanvasItem

Pros:
Simple to use for plain text.

Cons:
Bad for performance, shaping is costly and results of shaping are required for controls (caret movement, selection), and other text processing functions (justification, line breaking).
It's impossible to use functions linked to the single font with rich text (RichTextLabel), to archive correct result whole paragraph of the text should be shaped as single entity.
Font ascent and descent are used directly, since shaped glyphs can be displaced from the baseline these values should be corrected in the process of shaping.

Proposed/`libgdtl` implementation:

Uses additional objects to store shaped text.
TLShapedString immutable class to store shaping results, and provide function to perform measurements (string metrics, caret, selection) and shaped text processing (justification, line breaking).
TLAttributedShapedString spannable constructed on top of TLShapedString for the Rich text support.
TLParagraph simple wrapper to handle multiple lines of text in the more convenient way (optional, may be part of a multiline control instead of full-blown class)

Pros:
Shaping results can be reused in the controls without costly shaping.
Uses same interface for both plain and rich text.

Cons:
Extra abstraction layer (but it's easy to make wrappers for backward-compatibility).

3. Font fallback:

Current implementation:

BitmapFont uses reference to the next fallback, in the manner of single-linked list.
DynamicFont uses main DynamicFontData and a list of fallback DynamicFontData references.

Pros:

None ?

Cons:
Inconsistent, doesn't provide means to select fallback based on text language or script.
Since shaped text is font and content specific, for rendering purpose it should be associated directly with DynamicFontDataForSize selected during shaping, current implementation doesn't have similar object for BitmapFont.

Proposed/`libgdtl` implementation:

TLFontFamily holding separate TLFontFace reference lists for scripts/languages (supported scripts can be derived during TTF/OTF font loading, without additional user interaction).

Pros:
Ability to quickly find required font without trying to shape text with unsupported fonts.
Consistent with font structure. Font with wide language coverage are usually shipped as multiple files, divided by script/language.

Cons:
Breaks compatibility with current fonts.

4. Use of ICU library:

In the libgdtl provides APIs for BiDi, break-iterators (for justification, line breaking), and UCDN (used for shaping) and encoding conversion.

Pros:

Provides multiple useful APIs for CTL/shaping, some can be replaced with separate smaller implementation (BiDi, UCDN).
Potentially useful for other internationalization related stuff.

Cons:
ICU is quite big. Break-iterators are even bigger, but optional and can be included as part of the project if required.

archived core gui

Source

bruvzg

👍5

All 6 comments

My comments about this are:

~~I think we should use 32 bits everywhere, because users seem to like using font emojis.. 😩~~ as @bruvzg mentioned on irc, using 16 bits with surrogate pairs should be fine.
Current implementation must be kept. This is a hard requirement. For 99% of controls drawing an un-formatted line of text is enough, so these functions are perfect. Performance wise, Godot caches the rendered text, and we can use an LRU with hashing to improve get_string_size() in the worst case. Ascent is also fine because it gives you the baseline, but if we need an extra function to more correctly request the height of the string, we can add it. Of course for LineEdit/TextEdit/RichTextLabe/Label, the approach for rendering must be different.
There really is not much reason to keep BitmapFont to be honest, we do because some people likes it (works for custom, colored pixel art fonts), but I don't think anyone uses that will use advanced shaping or internalization, so I guess we should let it be. For DynamicFont, just implement whatever fallback mechanism works best.
Agreed that we definitely need to use ICU, but we should investigate if this can be added via GDNative if this is too big (so users don't have to recompile Godot), or maybe simply provide separate export templates with this enabled or disabled?

reduz on 7 May 2019

Agreed that we definitely need to use ICU, but we should investigate if this can be added via GDNative if this is too big (so users don't have to recompile Godot), or maybe simply provide separate export templates with this enabled or disabled?

It depends on what's considered big.

HarfBuzz is: 2.5 MB of sources, optimized static lib is 4 MB
ICU (common) is: 8 MB of sources, optimized static lib 3.5 MB
gdtl gdnative library is about 5 - 7 MB (depending on platform) total and incudes statically linked ICU, HB, FreeType, libpng and zlib.

ICU data 5 to 30 MB (it's only used for dictionary based line breaking and loaded from file or request).

There really is not much reason to keep BitmapFont to be honest

I guess that's reasonable, while it's possible to do some sort of shaping results are far from perfect.

bruvzg on 7 May 2019

To summarize I have drafted following minimal API changes:

`String`

Change it to use UTF-16 (should be straightforward, pretty much everything required is implemented by defines in the ICU's utf16.h header).
UTF-16 String Implementation Draft - adds Char32String (UTF-32 copy of CharString), and parse_utf32, to_utf32 functions, fixes *utf8() to handle surrogates, adds surrogate check to substr.
[Optional] Use ICU for UTF-8/UTF-32 and case conversion.
[Optional, not directly related to the topic] Add support to other ICU interfaces, like message formatting (some people need this stuff - https://github.com/godotengine/godot/pull/28710) or maybe this should go to TranslationServer?
Add static get_char_direction(char) -> direction simple ICU UCDN based function to get last input direction form keyboard events.

`Spannable` (or something similar).

For the rich text rendering, can be immutable.
get/set_text(text).
set_span(type, object, start, end), should detect span type base on object type or have it specified as the separate parameter:
- font (DynamicFont).
- language (String).
- [?] open_type_features (String), since OpenType feature tags already support string ranges this one can be safely omitted, single ot_features string passed to shape_xxx functions is sufficient.
- embedded image (Image), embedded image is subject to BiDi reordering and should be represented as a character in the string (usually it's U+FFFC but can be any).
- text/outline color (Color)
- [?] underline/strikeout/overline (??), color, width and style? Or simply bool (currently it's toggle only in RichTextLabel).
- meta (Variant), for RichTextLabel hyperlinks and other user defined stuff.
Bidirectional span iterator.

`BitmapFont`

Keep it as is.

`DynamicFontData`

Add supported_scripts() function for fallback sorting 1.
Add draw_glyph(rid, pos, index, color, outline) takes full UTF-32 char, same as existing draw_char but with FT_Load_Glyph instead of FT_Load_Char.
Add hb_font_t * interface.

`DynamicFont`

Add separate fallback lists for script/language.
add_fallback should automatically append DynamicFontData to appropriate list(s).
Add add_fallback_for_script/language or optional arguments to add_fallback for manually adding of the less common/non auth-detectable scripts. And probably some inspector plug-in for convenient edition of fallbacks.
Add shape_string(string, base_dir=AUTO, lang="", ot_features="") -> ShapedString
Add shape_spannable(spannable, base_dir=AUTO, lang="", ot_features="") -> ShapedSpannable
- BiDi/Shape: BiDi init, Script init, BiDi iterator, Script iterator, Font/OT feature/language iterator (for Spannable), Shape
- Return shaped string.
draw(rid, pos, string, color, base_dir=AUTO, lang="", ot_features="") add optional argument for better control, or just use current locale.
get_string_size(str, base_dir=AUTO, lang="", ot_features="")
- Shape.
- Cache shaped string if requited.
- Draw/return size.

`ShapedString` (shouldn't be as complex as `TLShapeString` as it does not need to reimplement half of the `String`).

get_size()
get_ascent/descent()
get_base_direction() for L/R alignment.
draw(rid, pos, color, outline)
break_lines(width, option_flags) -> Vector<ShapedString>
- Find break points
- Reshape each substring
get_start/end()-> int for lines after breaking
extend_to_width(width, option_flags) -> ShapedString or do it in place instead of creating new instance, Kashidas and spaces, Ligatures, Glyph width stretching (change up to 20% should be OK).
get_highlighted_shapes(start, end) -> Vector<Rect2> for selection 1.
get_cursor_positions(pos, last_input_dir) -> Vector<float> for cursor control 1.
hit_test(pos) -> int to get cursor position form mouse clicks 1.
Direct access to glyphs and clusters is not required.

`ShapedSpannable` subclass of `ShapedString`

hit_test_meta/span(pos) -> Variant, for RichTextLabel hyperlinks and stuff like this.

bruvzg on 7 May 2019

A bit of additional size info, here are symbol sizes in the debug Linux binary with gdtl module measured using Bloaty tool.

ICU (≈512.4 KiB)

     VM SIZE                                                                                        FILE SIZE
 --------------                                                                                  --------------
   0.0%  29.5Ki modules/godot_tl/subprojects/icu4c/source/common/normalizer2impl.cpp              29.5Ki   0.0%
   0.0%  20.1Ki modules/godot_tl/subprojects/icu4c/source/common/uresbund.cpp                     20.1Ki   0.0%
   0.0%  18.2Ki modules/godot_tl/subprojects/icu4c/source/common/ubidi.cpp                        18.2Ki   0.0%
   0.0%  16.4Ki modules/godot_tl/subprojects/icu4c/source/common/unistr.cpp                       16.4Ki   0.0%
   0.0%  16.3Ki modules/godot_tl/subprojects/icu4c/source/common/dictbe.cpp                       16.3Ki   0.0%
   0.0%  15.4Ki modules/godot_tl/subprojects/icu4c/source/common/uniset.cpp                       15.4Ki   0.0%
   0.0%  14.7Ki modules/godot_tl/subprojects/icu4c/source/common/uloc.cpp                         14.7Ki   0.0%
   0.0%  14.4Ki modules/godot_tl/subprojects/icu4c/source/common/uloc_tag.cpp                     14.4Ki   0.0%
   0.0%  14.3Ki modules/godot_tl/subprojects/icu4c/source/common/utext.cpp                        14.3Ki   0.0%
   0.0%  13.0Ki modules/godot_tl/subprojects/icu4c/source/common/locid.cpp                        13.0Ki   0.0%
   0.0%  11.6Ki modules/godot_tl/subprojects/icu4c/source/common/unames.cpp                       11.6Ki   0.0%
   0.0%  11.4Ki modules/godot_tl/subprojects/icu4c/source/common/unisetspan.cpp                   11.4Ki   0.0%
   0.0%  11.3Ki modules/godot_tl/subprojects/icu4c/source/common/ustrcase.cpp                     11.3Ki   0.0%
   0.0%  10.2Ki modules/godot_tl/subprojects/icu4c/source/common/rbbitblb.cpp                     10.2Ki   0.0%
   0.0%  10.1Ki modules/godot_tl/subprojects/icu4c/source/common/umutablecptrie.cpp               10.1Ki   0.0%
   0.0%  10.1Ki modules/godot_tl/subprojects/icu4c/source/common/normalizer2.cpp                  10.1Ki   0.0%
   0.0%  10.0Ki modules/godot_tl/subprojects/icu4c/source/common/ustrtrns.cpp                     10.0Ki   0.0%
   0.0%  9.12Ki modules/godot_tl/subprojects/icu4c/source/common/ucase.cpp                        9.12Ki   0.0%
   0.0%  8.90Ki modules/godot_tl/subprojects/icu4c/source/common/uresdata.cpp                     8.90Ki   0.0%
   0.0%  8.83Ki modules/godot_tl/subprojects/icu4c/source/common/serv.cpp                         8.83Ki   0.0%
   0.0%  8.77Ki modules/godot_tl/subprojects/icu4c/source/common/uniset_props.cpp                 8.77Ki   0.0%
   0.0%  8.58Ki modules/godot_tl/subprojects/icu4c/source/common/ucnv_io.cpp                      8.58Ki   0.0%
   0.0%  8.53Ki modules/godot_tl/subprojects/icu4c/source/common/rbbiscan.cpp                     8.53Ki   0.0%
   0.0%  8.03Ki modules/godot_tl/subprojects/icu4c/source/common/rbbi.cpp                         8.03Ki   0.0%
   0.0%  7.73Ki modules/godot_tl/subprojects/icu4c/source/common/udata.cpp                        7.73Ki   0.0%
   0.0%  7.61Ki modules/godot_tl/subprojects/icu4c/source/common/ustring.cpp                      7.61Ki   0.0%
   0.0%  7.44Ki modules/godot_tl/subprojects/icu4c/source/common/utrie2_builder.cpp               7.44Ki   0.0%
   0.0%  7.26Ki modules/godot_tl/subprojects/icu4c/source/common/ubidiln.cpp                      7.26Ki   0.0%
   0.0%  6.93Ki modules/godot_tl/subprojects/icu4c/source/common/uchar.cpp                        6.93Ki   0.0%
   0.0%  6.46Ki modules/godot_tl/subprojects/icu4c/source/common/filteredbrk.cpp                  6.46Ki   0.0%
   0.0%  6.46Ki modules/godot_tl/subprojects/icu4c/source/common/utrie.cpp                        6.46Ki   0.0%
   0.0%  6.30Ki modules/godot_tl/subprojects/icu4c/source/common/putil.cpp                        6.30Ki   0.0%
   0.0%  5.74Ki modules/godot_tl/subprojects/icu4c/source/common/locdispnames.cpp                 5.74Ki   0.0%
   0.0%  5.70Ki modules/godot_tl/subprojects/icu4c/source/common/edits.cpp                        5.70Ki   0.0%
   0.0%  5.63Ki modules/godot_tl/subprojects/icu4c/source/common/stringtriebuilder.cpp            5.63Ki   0.0%
   0.0%  5.62Ki modules/godot_tl/subprojects/icu4c/source/common/rbbi_cache.cpp                   5.62Ki   0.0%
   0.0%  5.03Ki modules/godot_tl/subprojects/icu4c/source/common/bmpset.cpp                       5.03Ki   0.0%
   0.0%  4.82Ki modules/godot_tl/subprojects/icu4c/source/common/ucharstriebuilder.cpp            4.82Ki   0.0%
   0.0%  4.72Ki modules/godot_tl/subprojects/icu4c/source/common/loclikely.cpp                    4.72Ki   0.0%
   0.0%  4.59Ki modules/godot_tl/subprojects/icu4c/source/common/uloc_keytype.cpp                 4.59Ki   0.0%
   0.0%  4.47Ki modules/godot_tl/subprojects/icu4c/source/common/utrie2.cpp                       4.47Ki   0.0%
   0.0%  3.93Ki modules/godot_tl/subprojects/icu4c/source/common/ubidiwrt.cpp                     3.93Ki   0.0%
   0.0%  3.86Ki modules/godot_tl/subprojects/icu4c/source/common/uprops.cpp                       3.86Ki   0.0%
   0.0%  3.80Ki modules/godot_tl/subprojects/icu4c/source/common/uhash.cpp                        3.80Ki   0.0%
   0.0%  3.75Ki modules/godot_tl/subprojects/icu4c/source/common/utrace.cpp                       3.75Ki   0.0%
   0.0%  3.62Ki modules/godot_tl/subprojects/icu4c/source/common/uvector.cpp                      3.62Ki   0.0%
   0.0%  3.46Ki modules/godot_tl/subprojects/icu4c/source/common/ucptrie.cpp                      3.46Ki   0.0%
   0.0%  3.31Ki modules/godot_tl/subprojects/icu4c/source/common/brkiter.cpp                      3.31Ki   0.0%
   0.0%  3.07Ki modules/godot_tl/subprojects/icu4c/source/common/uchriter.cpp                     3.07Ki   0.0%
   0.0%  3.00Ki modules/godot_tl/subprojects/icu4c/source/common/ucol_swp.cpp                     3.00Ki   0.0%
   0.0%  2.95Ki modules/godot_tl/subprojects/icu4c/source/common/bytestrie.cpp                    2.95Ki   0.0%
   0.0%  2.74Ki modules/godot_tl/subprojects/icu4c/source/common/loadednormalizer2impl.cpp        2.74Ki   0.0%
   0.0%  2.71Ki modules/godot_tl/subprojects/icu4c/source/common/ucharstrie.cpp                   2.71Ki   0.0%
   0.0%  2.67Ki modules/godot_tl/subprojects/icu4c/source/common/util.cpp                         2.67Ki   0.0%
   0.0%  2.61Ki modules/godot_tl/subprojects/icu4c/source/common/uinvchar.cpp                     2.61Ki   0.0%
   0.0%  2.52Ki modules/godot_tl/subprojects/icu4c/source/common/udataswp.cpp                     2.52Ki   0.0%
   0.0%  2.42Ki modules/godot_tl/subprojects/icu4c/source/common/characterproperties.cpp          2.42Ki   0.0%
   0.0%  2.40Ki modules/godot_tl/subprojects/icu4c/source/common/locutil.cpp                      2.40Ki   0.0%
   0.0%  2.37Ki modules/godot_tl/subprojects/icu4c/source/common/brkeng.cpp                       2.37Ki   0.0%
   0.0%  2.34Ki modules/godot_tl/subprojects/icu4c/source/common/rbbisetb.cpp                     2.34Ki   0.0%
   0.0%  2.30Ki modules/godot_tl/subprojects/icu4c/source/common/unistr_case.cpp                  2.30Ki   0.0%
   0.0%  2.25Ki modules/godot_tl/subprojects/icu4c/source/common/servls.cpp                       2.25Ki   0.0%
   0.0%  2.24Ki modules/godot_tl/subprojects/icu4c/source/common/ustrenum.cpp                     2.24Ki   0.0%
   0.0%  2.20Ki modules/godot_tl/subprojects/icu4c/source/common/uvectr32.cpp                     2.20Ki   0.0%
   0.0%  2.18Ki modules/godot_tl/subprojects/icu4c/source/common/utf_impl.cpp                     2.18Ki   0.0%
   0.0%  2.14Ki modules/godot_tl/subprojects/icu4c/source/common/resbund.cpp                      2.14Ki   0.0%
   0.0%  2.05Ki modules/godot_tl/subprojects/icu4c/source/common/rbbirb.cpp                       2.05Ki   0.0%
   0.0%  1.96Ki modules/godot_tl/subprojects/icu4c/source/common/dictionarydata.cpp               1.96Ki   0.0%
   0.0%  1.86Ki modules/godot_tl/subprojects/icu4c/source/common/rbbidata.cpp                     1.86Ki   0.0%
   0.0%  1.83Ki modules/godot_tl/subprojects/icu4c/source/common/ubidi_props.cpp                  1.83Ki   0.0%
   0.0%  1.81Ki modules/godot_tl/subprojects/icu4c/source/common/charstr.cpp                      1.81Ki   0.0%
   0.0%  1.79Ki modules/godot_tl/subprojects/icu4c/source/common/propname.cpp                     1.79Ki   0.0%
   0.0%  1.74Ki modules/godot_tl/subprojects/icu4c/source/common/servlk.cpp                       1.74Ki   0.0%
   0.0%  1.72Ki modules/godot_tl/subprojects/icu4c/source/common/uarrsort.cpp                     1.72Ki   0.0%
   0.0%  1.70Ki modules/godot_tl/subprojects/icu4c/source/common/utrie_swap.cpp                   1.70Ki   0.0%
   0.0%  1.49Ki modules/godot_tl/subprojects/icu4c/source/common/rbbinode.cpp                     1.49Ki   0.0%
   0.0%  1.47Ki modules/godot_tl/subprojects/icu4c/source/common/schriter.cpp                     1.47Ki   0.0%
   0.0%  1.46Ki modules/godot_tl/subprojects/icu4c/source/common/ubrk.cpp                         1.46Ki   0.0%
   0.0%  1.32Ki modules/godot_tl/subprojects/icu4c/source/common/ucmndata.cpp                     1.32Ki   0.0%
   0.0%  1.31Ki modules/godot_tl/subprojects/icu4c/source/common/bytesinkutil.cpp                 1.31Ki   0.0%
   0.0%  1.23Ki modules/godot_tl/subprojects/icu4c/source/common/locavailable.cpp                 1.23Ki   0.0%
   0.0%  1.09Ki modules/godot_tl/subprojects/icu4c/source/common/ruleiter.cpp                     1.09Ki   0.0%
   0.0%  1.03Ki modules/godot_tl/subprojects/icu4c/source/common/servlkf.cpp                      1.03Ki   0.0%
   0.0%  1.02Ki modules/godot_tl/subprojects/icu4c/source/common/locmap.cpp                       1.02Ki   0.0%
   0.0%    1007 modules/godot_tl/subprojects/icu4c/source/common/locresdata.cpp                     1007   0.0%
   0.0%    1003 modules/godot_tl/subprojects/icu4c/source/common/rbbistbl.cpp                       1003   0.0%
   0.0%     989 modules/godot_tl/subprojects/icu4c/source/common/cstring.cpp                         989   0.0%
   0.0%     886 modules/godot_tl/subprojects/icu4c/source/common/servnotf.cpp                        886   0.0%
   0.0%     870 modules/godot_tl/subprojects/icu4c/source/common/patternprops.cpp                    870   0.0%
   0.0%     784 modules/godot_tl/subprojects/icu4c/source/common/servslkf.cpp                        784   0.0%
   0.0%     727 modules/godot_tl/subprojects/icu4c/source/common/uenum.cpp                           727   0.0%
   0.0%     540 modules/godot_tl/subprojects/icu4c/source/common/chariter.cpp                        540   0.0%
   0.0%     473 modules/godot_tl/subprojects/icu4c/source/common/udatamem.cpp                        473   0.0%
   0.0%     456 modules/godot_tl/subprojects/icu4c/source/common/servrbf.cpp                         456   0.0%
   0.0%     434 modules/godot_tl/subprojects/icu4c/source/common/cmemory.cpp                         434   0.0%
   0.0%     406 modules/godot_tl/subprojects/icu4c/source/common/unifilt.cpp                         406   0.0%
   0.0%     392 modules/godot_tl/subprojects/icu4c/source/common/bytestream.cpp                      392   0.0%
   0.0%     382 modules/godot_tl/subprojects/icu4c/source/common/uscript_props.cpp                   382   0.0%
   0.0%     360 modules/godot_tl/subprojects/icu4c/source/common/ustack.cpp                          360   0.0%
   0.0%     359 modules/godot_tl/subprojects/icu4c/source/common/umutex.cpp                          359   0.0%
   0.0%     300 modules/godot_tl/subprojects/icu4c/source/common/appendable.cpp                      300   0.0%
   0.0%     293 modules/godot_tl/subprojects/icu4c/source/common/ucln_cmn.cpp                        293   0.0%
   0.0%     267 modules/godot_tl/subprojects/icu4c/source/common/locbased.cpp                        267   0.0%
   0.0%     241 modules/godot_tl/subprojects/icu4c/source/common/umapfile.cpp                        241   0.0%
   0.0%     241 modules/godot_tl/subprojects/icu4c/source/common/ustrfmt.cpp                         241   0.0%
   0.0%     238 modules/godot_tl/subprojects/icu4c/source/common/utypes.cpp                          238   0.0%
   0.0%     234 modules/godot_tl/subprojects/icu4c/source/common/uobject.cpp                         234   0.0%
   0.0%     227 modules/godot_tl/subprojects/icu4c/source/common/stringpiece.cpp                     227   0.0%
   0.0%     142 modules/godot_tl/subprojects/icu4c/source/common/uinit.cpp                           142   0.0%
   0.0%     123 modules/godot_tl/subprojects/icu4c/source/common/parsepos.cpp                        123   0.0%
   0.0%     122 modules/godot_tl/subprojects/icu4c/source/common/ustr_wcs.cpp                        122   0.0%
   0.0%      92 modules/godot_tl/subprojects/icu4c/source/common/resource.cpp                         92   0.0%
   0.0%      92 modules/godot_tl/subprojects/icu4c/source/common/unifunct.cpp                         92   0.0%
   0.0%      33 modules/godot_tl/subprojects/icu4c/source/common/uhash_us.cpp                         33   0.0%
   0.0%      24 modules/godot_tl/subprojects/icu4c/source/common/umath.cpp                            24   0.0%

HarfBuzz (≈531.9 KiB)

     VM SIZE                                                                                        FILE SIZE
 --------------                                                                                  --------------
   0.2%   131Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-layout.cc                          131Ki   0.2%
   0.1%  75.0Ki modules/godot_tl/subprojects/harfbuzz/src/hb-aat-layout.cc                        75.0Ki   0.1%
   0.1%  47.6Ki modules/godot_tl/subprojects/harfbuzz/src/hb-face.cc                              47.6Ki   0.1%
   0.1%  39.5Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-font.cc                           39.5Ki   0.1%
   0.0%  29.4Ki modules/godot_tl/subprojects/harfbuzz/src/hb-set.cc                               29.4Ki   0.0%
   0.0%  20.4Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-normalize.cc                20.4Ki   0.0%
   0.0%  20.3Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-arabic.cc           20.3Ki   0.0%
   0.0%  19.9Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-var.cc                            19.9Ki   0.0%
   0.0%  18.4Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-indic.cc            18.4Ki   0.0%
   0.0%  18.3Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape.cc                          18.3Ki   0.0%
   0.0%  14.6Ki modules/godot_tl/subprojects/harfbuzz/src/hb-font.cc                              14.6Ki   0.0%
   0.0%  14.4Ki modules/godot_tl/subprojects/harfbuzz/src/hb-buffer.cc                            14.4Ki   0.0%
   0.0%  10.3Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-tag.cc                            10.3Ki   0.0%
   0.0%  8.96Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-fallback.cc                 8.96Ki   0.0%
   0.0%  7.24Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-map.cc                            7.24Ki   0.0%
   0.0%  7.09Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-use.cc              7.09Ki   0.0%
   0.0%  5.68Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-khmer.cc            5.68Ki   0.0%
   0.0%  5.60Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-vowel-constraints.  5.60Ki   0.0%
   0.0%  5.43Ki modules/godot_tl/subprojects/harfbuzz/src/hb-common.cc                            5.43Ki   0.0%
   0.0%  4.49Ki modules/godot_tl/subprojects/harfbuzz/src/hb-shape-plan.cc                        4.49Ki   0.0%
   0.0%  3.88Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ft.cc                                3.88Ki   0.0%
   0.0%  3.65Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-myanmar.cc          3.65Ki   0.0%
   0.0%  3.50Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-hangul.cc           3.50Ki   0.0%
   0.0%  3.16Ki modules/godot_tl/subprojects/harfbuzz/src/hb-graphite2.cc                         3.16Ki   0.0%
   0.0%  3.00Ki modules/godot_tl/subprojects/harfbuzz/src/hb-unicode.cc                           3.00Ki   0.0%
   0.0%  2.03Ki modules/godot_tl/subprojects/harfbuzz/src/hb-blob.cc                              2.03Ki   0.0%
   0.0%  1.68Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-thai.cc             1.68Ki   0.0%
   0.0%  1.54Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-face.cc                           1.54Ki   0.0%
   0.0%  1.22Ki modules/godot_tl/subprojects/harfbuzz/src/hb-aat-map.cc                           1.22Ki   0.0%
   0.0%  1.09Ki modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-use-table.cc        1.09Ki   0.0%
   0.0%  1.03Ki modules/godot_tl/subprojects/harfbuzz/src/hb-icu.cc                               1.03Ki   0.0%
   0.0%     736 modules/godot_tl/subprojects/harfbuzz/src/hb-static.cc                               736   0.0%
   0.0%     605 modules/godot_tl/subprojects/harfbuzz/src/hb-shaper.cc                               605   0.0%
   0.0%     588 modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-hebrew.cc              588   0.0%
   0.0%     498 modules/godot_tl/subprojects/harfbuzz/src/hb-ot-shape-complex-indic-table.cc         498   0.0%
   0.0%     261 modules/godot_tl/subprojects/harfbuzz/src/hb-shape.cc                                261   0.0%

Graphite (≈100.6 KiB)

     VM SIZE                                                                                        FILE SIZE
 --------------                                                                                  --------------
   0.0%  23.9Ki modules/godot_tl/subprojects/graphite2/src/Collider.cpp                           23.9Ki   0.0%
   0.0%  11.5Ki modules/godot_tl/subprojects/graphite2/src/Pass.cpp                               11.5Ki   0.0%
   0.0%  7.59Ki modules/godot_tl/subprojects/graphite2/src/Slot.cpp                               7.59Ki   0.0%
   0.0%  6.84Ki modules/godot_tl/subprojects/graphite2/src/Segment.cpp                            6.84Ki   0.0%
   0.0%  6.53Ki modules/godot_tl/subprojects/graphite2/src/Code.cpp                               6.53Ki   0.0%
   0.0%  5.68Ki modules/godot_tl/subprojects/graphite2/src/direct_machine.cpp                     5.68Ki   0.0%
   0.0%  5.37Ki modules/godot_tl/subprojects/graphite2/src/GlyphCache.cpp                         5.37Ki   0.0%
   0.0%  4.32Ki modules/godot_tl/subprojects/graphite2/src/Silf.cpp                               4.32Ki   0.0%
   0.0%  4.31Ki modules/godot_tl/subprojects/graphite2/src/TtfUtil.cpp                            4.31Ki   0.0%
   0.0%  4.28Ki modules/godot_tl/subprojects/graphite2/src/Intervals.cpp                          4.28Ki   0.0%
   0.0%  3.32Ki modules/godot_tl/subprojects/graphite2/src/NameTable.cpp                          3.32Ki   0.0%
   0.0%  3.26Ki modules/godot_tl/subprojects/graphite2/src/FeatureMap.cpp                         3.26Ki   0.0%
   0.0%  2.87Ki modules/godot_tl/subprojects/graphite2/src/Justifier.cpp                          2.87Ki   0.0%
   0.0%  2.82Ki modules/godot_tl/subprojects/graphite2/src/Face.cpp                               2.82Ki   0.0%
   0.0%  2.38Ki modules/godot_tl/subprojects/graphite2/src/gr_segment.cpp                         2.38Ki   0.0%
   0.0%  1.44Ki modules/godot_tl/subprojects/graphite2/src/CmapCache.cpp                          1.44Ki   0.0%
   0.0%  1.14Ki modules/godot_tl/subprojects/graphite2/src/gr_face.cpp                            1.14Ki   0.0%
   0.0%  1.06Ki modules/godot_tl/subprojects/graphite2/src/gr_slot.cpp                            1.06Ki   0.0%
   0.0%     817 modules/godot_tl/subprojects/graphite2/src/gr_features.cpp                           817   0.0%
   0.0%     700 modules/godot_tl/subprojects/graphite2/src/Decompressor.cpp                          700   0.0%
   0.0%     387 modules/godot_tl/subprojects/graphite2/src/Sparse.cpp                                387   0.0%
   0.0%     169 modules/godot_tl/subprojects/graphite2/src/GlyphFace.cpp                             169   0.0%

Rest of the modeule code (≈633.5 KiB)

     VM SIZE                                                                                        FILE SIZE
 --------------                                                                                  --------------
   0.2%   156Ki modules/godot_tl/src/resources/tl_shaped_string.cpp                                156Ki   0.2%
   0.2%   109Ki modules/godot_tl/src/controls/tl_proto_control.cpp                                 109Ki   0.2%
   0.2%   101Ki modules/godot_tl/src/resources/tl_shaped_attributed_string.cpp                     101Ki   0.2%
   0.1%  61.5Ki modules/godot_tl/src/controls/tl_line_edit.cpp                                    61.5Ki   0.1%
   0.1%  40.6Ki modules/godot_tl/src/resources/tl_font_family.cpp                                 40.6Ki   0.1%
   0.1%  34.8Ki modules/godot_tl/src/resources/tl_dynamic_font.cpp                                34.8Ki   0.1%
   0.0%  30.7Ki modules/godot_tl/src/controls/tl_label.cpp                                        30.7Ki   0.0%
   0.0%  26.7Ki modules/godot_tl/src/resources/tl_font.cpp                                        26.7Ki   0.0%
   0.0%  23.7Ki modules/godot_tl/src/resources/tl_shaped_paragraph.cpp                            23.7Ki   0.0%
   0.0%  23.0Ki modules/godot_tl/src/resources/tl_bitmap_font.cpp                                 23.0Ki   0.0%
   0.0%  13.3Ki modules/godot_tl/src/tl_core.cpp                                                  13.3Ki   0.0%
   0.0%  6.75Ki modules/godot_tl/src/tools/tl_editor_node.cpp                                     6.75Ki   0.0%
   0.0%  6.48Ki modules/godot_tl/src/resources/tl_icu_data_loader.cpp                             6.48Ki   0.0%

bruvzg on 28 May 2019

Since this stuff is quite heavy, OSs like Windows, macOS and Android have their own shaping engines, shaping engines in general depends on font implementation internals and some users desire to use system fonts, it might be better to move both font handling and shaper to the single module/plugin. Also, using system shapers and font libraries will give UI more native look and feel.

Keep BitmapFont and dummy shaper (LTR only, pass thought - maps chars/graphemes one-to-one) as fallback, have FreeType and ICU/HarfBuzz (or whatever will be used) as optional (and default for the editor) module, and allow custom implementations (e.g. based on DirectWrite/CoreText shapers) via GDNative plugins.

A quick mock-up of the backend interface that should cover all use cases.

class TextServer : public Object {
    GDCLASS(TextServer, Object);

public:
    enum TextDirection {
        TEXT_DIRECTION_AUTO, // Detects text direction based on string content and specific locale
        TEXT_DIRECTION_LTR, // Left-to-right.
        TEXT_DIRECTION_RTL // Right-to-left.
    };

    enum TextJustification {
        TEXT_JUSTIFICATION_NONE = 0,
        TEXT_JUSTIFICATION_KASHIDA = 1 << 1, // Change width or add/remove kashidas (ــــ).
        TEXT_JUSTIFICATION_WORD_BOUND = 1 << 2, // Adds/removes extra space between the words (for some languages, should add spaces even if there were non in the original string, using dictionary).
        TEXT_JUSTIFICATION_GRAPHEME_BOUND = 1 << 3, // Adds/removes extra space in between all non-joining graphemes.
        TEXT_JUSTIFICATION_GRAPHEME_WIDTH = 1 << 4 // Adjusts width of the graphemes visually (if supported by font), 10-15% of change should be OK in general.
    };

    enum TextBreak {
        TEXT_BREAK_NONE = 0,
        TEXT_BREAK_MANDATORY = 1 << 1, // Breaks line at the explicit line break characters ("\n" etc).
        TEXT_BREAK_WORD_BOUND = 1 << 2, // Breaks line between the words.
        TEXT_BREAK_GRAPHEME_BOUND = 1 << 3 // Breaks line between any graphemes (in general it's OK to break line anywhere, as long as it isn't reshaped after).
    };

    struct Grapheme {
        struct Glyph {
            uint32_t glyph_index = 0; // Glyph index is internal value of the font and can't be reused with other fonts.
            Vector2 offset; // Offset from the origin of the glyph.
        };

        Vector<Glyph> glyphs;
        Vecotor2i range; // Range in the original string this grapheme corresponds to.
        float advance = 0.f; // Horizontal advance to the next grapheme.
        bool rtl = false; // Direction of the grapheme, can be used to display cursor on the correct side of it.
        RID font;
    };

    struct TextFormat {
        Vector2i range; // Range in the original string to apply formatting to.

        RID font;
        String features; // List of OpenType feature tags, for advanced typography, see https://docs.microsoft.com/en-us/typography/opentype/spec/features_ae
        String locale; // ISO language tag.

        Variant inline_object; // Inline object id. Can be used to add images/tables etc., for the shaping engine only size of the object and it's position in the string is relevant, usually it's represented as special placeholder character (U+FFFC).
        Size2 inline_object_size; // Inline object size.
        VAlign inline_object_align; // Inline object offset from the base line.
    };

protected:
    //......//

public:
    virtual void free(RID p_rid) = 0;

    // Font API.
    virtual RID create_font_from_name(const String &p_name, float p_font_size) = 0; // Loads OS defualt font by name (if supported).
    virtual RID create_font_from_res(const String &p_filename, float p_font_size) = 0; // Loads custom font from "res://" "file".
    virtual RID create_font(const Vector<uint8_t> &p_data, float p_font_size) = 0; // Loads custom font from memory.

    virtual RID font_get_fallback(RID p_font) const = 0;
    virtual void font_set_fallback(RID p_font, RID p_fallback) = 0;

    virtual float font_get_height(RID p_font) const = 0;
    virtual float font_get_ascent(RID p_font) const = 0;
    virtual float font_get_descent(RID p_font) const = 0;
    virtual float font_get_underline_position(RID p_font) const = 0;
    virtual float font_get_underline_thickness(RID p_font) const = 0;

    virtual bool font_has_outline(RID p_font) const = 0;

    virtual void font_draw_glyph(RID p_canvas, const Vector2 &p_pos, uint32_t p_index, RID p_font, const Color &p_color);
    virtual void font_draw_glyph_outline(RID p_canvas, const Vector2 &p_pos, uint32_t p_index, RID p_font, const Color &p_color);

    // Shaped text API.
    virtual RID shape_plain_text(const String &p_text, RID p_font, const String &p_features, const String &p_locale, TextDirection p_direction) = 0; // Performs BiDi reordering and shaping as a single line.
    virtual RID shape_rich_text(const String &p_text, const Vector<TextFormat> &p_formatting, TextDirection p_direction) = 0;

    virtual Vector<Grapheme> shaped_get_graphemes(RID p_shaped, const Vector2i &p_range) const = 0; // Returns graphemes as is or BiDi reorders them for the line if range is specified. Graphemes returned in visual (LTR) order. Returned graphems should be usable in the place of characters for the most UI use cases, without massive code changes.
    virtual TextDirection shaped_get_direction(RID p_shaped) const = 0; // Returns detected base direction of the string if it was shaped with AUTO direction.
    virtual Vector2 shaped_get_inline_object_position(RID p_shaped, const Variant &p_inline_object, const Vector2i &p_range) const = 0; // Returns position of the inline object after shaping (position in specific line if range is specified).

    virtual Vector<Vector2i> shaped_get_line_breaks(RID p_shaped, float p_width, TextBreak p_break_mode) const = 0; // Returns line ranges, ranges can be directly used with get_graphemes function to render multiline text.
    virtual Vector<Vector2i> shaped_get_word_breaks(RID p_shaped, float p_width) const = 0;

    virtual Size2 shaped_get_size(RID p_shaped) const = 0;
    virtual float shaped_get_ascent(RID p_shaped) const = 0; // For some languages, graphemes can be offset from the base line significantly, these functions should return maximum ascent and descent, though for most cases using font ascent/descent is OK.
    virtual float shaped_get_descent(RID p_shaped) const = 0; // Also, can include size of inline objects.

    virtual float shaped_fit_to_width(RID p_shaped, float p_width, TextJustification p_justification_mode) const = 0; // Adjusts spaces and elongations in the line to fit it to the specified width, returns line width after adjustment.
};

VARIANT_ENUM_CAST(TextServer::TextDirection);
VARIANT_ENUM_CAST(TextServer::TextBreak);
VARIANT_ENUM_CAST(TextServer::TextJustification);

// Text and cursors drawing sample.

RID sh = TS->shape_plain_text(text, font, "", "", AUTO);
Vector<Grapheme> gr = TS->shaped_get_graphemes(sh, Vector2i());

...

Vecotr2 offset;
for (int i = 0; i < gr.size(); i++) {
    // Draw glyphs
    for (int j = 0; j < gr[i].glpyhs.size(); j++) {
        TS->font_draw_glyph(canvas, offset + gr[i].glyphs[j].offset, gr[i].glyphs[j].index, gr[i].font, color);
    }
    // Draw cursors (In most cases moving cursor only to the grapheme edges is OK. Cursor in the middle of grapheme can be useful, but calculating its position require support from the font which is absent absolute majority of the fonts)
    if (gr[i].range.x /*start*/ == cursor_pos) {
        if (gr[i].rtl) {
            // draw cursor @ offset.x + gr[i].advance
        } else {
            // draw cursor @ offset.x
        }
    }
    if (gr[i].range.y /*end*/ == cursor_pos) {
        if (gr[i].rtl) {
            // draw cursor @ offset.x
        } else {
            // draw cursor @ offset.x + gr[i].advance
        }
    }
    offset.x += gr[i].advance;
}

bruvzg on 1 Jun 2020

👀4

Superseded by #1180, #1181, #1182, #1183

bruvzg on 10 Jul 2020

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Differentiate Image's get_data() and Texture's get_data()

rainlizard · 3Comments

Add 'default scene path' to Project Settings

Dorifor · 3Comments

Sprite Overlays

SpyrexDE · 3Comments

Add support for 3D Level of Detail

lupoDharkael · 3Comments

Display enum name next to its value in the Remote Inspector

PLyczkowski · 3Comments

Godot-proposals: Complex text layout/shaping API structure

1. Godot String API:

Current implementation:

Proposed implementation:

2. Text rendering API structure and abstraction level:

Current implementation:

Pros:

Cons:

Proposed/libgdtl implementation:

Pros:

Cons:

3. Font fallback:

Current implementation:

Pros:

Cons:

Proposed/libgdtl implementation:

Pros:

Cons:

4. Use of ICU library:

Pros:

Cons:

All 6 comments

String

Spannable (or something similar).

BitmapFont

DynamicFontData

DynamicFont

ShapedString (shouldn't be as complex as TLShapeString as it does not need to reimplement half of the String).

ShapedSpannable subclass of ShapedString

Related issues

1. Godot `String` API:

Proposed/`libgdtl` implementation:

Proposed/`libgdtl` implementation:

`String`

`Spannable` (or something similar).

`BitmapFont`

`DynamicFontData`

`DynamicFont`

`ShapedString` (shouldn't be as complex as `TLShapeString` as it does not need to reimplement half of the `String`).

`ShapedSpannable` subclass of `ShapedString`