Mapbox-gl-native: Allow using system fonts or fonts bundled in the app

[ericrwolfe]

This topic was discussed back in 2014 (https://github.com/mapbox/mapbox-gl-native/issues/260) and ought to be revisited now that we've expanded the SDK.

In many cases, developers who implement custom styles or change fonts via runtime-styling are going to want to use the same fonts in their maps as they do in the UI of their app, whether that's using the system fonts or bundling in their own.

Currently, fonts/glyphs are downloaded by Mapbox even if they might already exist on the device. This is particularly heavyweight for offline: https://github.com/mapbox/mapbox-gl-native/issues/4069.

Additionally, instead of relying on a version of Arial Unicode MS to be downloaded with the style for fallback glyph rendering, we should consider falling back to the default system font instead. I'm not sure what the implications are here for complex text rendering: https://github.com/mapbox/mapbox-gl-native/issues/7774

[1ec5]

In many cases, developers who implement custom styles or change fonts via runtime-styling are going to want to use the same fonts in their maps as they do in the UI of their app, whether that's using the system fonts or bundling in their own.

Agreed. From the perspective of an iOS developer, specifying custom fonts in Studio is more of a bug than a feature; they’d expect the SDK to use system or locally bundled fonts for any custom font styling.

I'm not sure what the implications are here for complex text rendering: #7774

mbgl uses SDF glyphs rather than a standard font format, so one idea for #7774 and #7528 would be to use a system text layout/rendering library like Core Text (via HarfBuzz) that could use local fonts out of the box.

/cc @ChrisLoer

[ChrisLoer]

mbgl uses SDF glyphs rather than a standard font format, so one idea for #7774 and #7528 would be to use a system text layout/rendering library like Core Text (via HarfBuzz) that could use local fonts out of the box.

That should be pretty easy once HarfBuzz is in place. We'd also have to include FreeType to do the Glyph->SDF rendering on the client side instead of in node-fontnik. After talking with @yhahn, I've actually been thinking about starting the HarfBuzz implementation this way so we can get a working client implementation before we start trying to define a new font server API.

[ChrisLoer]

I've got a crude version of this working for iOS on the cloer_harfbuzz branch. Although I implemented it using FreeType and HarfBuzz, only the FreeType part is necessary for most languages, so if we think there's good value here we could start with a much smaller PR.

[ChrisLoer]

PR #10522 is motivated by the narrower goal of making CJK glyphs load faster, but if we end up with Android/iOS implementations that can extract glyph metrics, it could turn into a complete solution for this issue, if not #7774.

Freetype

• Renders directly from vector -> SDF :+1:
• Understands font-specific glyph IDs necessary for doing shaping with Harfbuzz :+1:
• Requires bundling Freetype code into Mapbox SDKs, increasing bundle size :-1:

TinySDF

• Renders vector -> raster -> SDF, which theoretically makes it more expensive and increases lossiness. After actually using it, though, we don't really see any meaningful difference in performance or fidelity. :+1:
• Depends on a Unicode codepoint based interface, so can't be used for complex text shaping/Harfbuzz :-1:
• Tiny! No noticeable change in SDK size. :+1:

/cc @mourner

[1ec5]

@ChrisLoer and I paired on the coretext-hack branch to achieve the same functionality as the cloer_harfbuzz branch mentioned in https://github.com/mapbox/mapbox-gl-native/issues/7862#issuecomment-282851879, but using TinySDF and Core Text instead of Harfbuzz and FreeType to perform complex text shaping using system fonts:

[1ec5]

As of e3645ecd1d6420c73f0ca5bc53def60af9a235da on that branch, text-font is set to Zapfino, based on the selection in the font picker. Latin text uses the Zapfino installed on the system, while non-Latin text uses fallback fonts according to the system’s default cascade list:

If a font bundled with the application is specified in text-font, then the map would automatically pick up that font instead of a system font.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[friedbunny]

This should still happen.

[stale[bot]]

This issue has been automatically detected as stale because it has not had recent activity and will be archived. Thank you for your contributions.

[1ec5]

By way of an update, #10572 long ago added the ability to render glyphs from local or system fonts, but it was enabled only for a limited range of Chinese and Japanese (and later Korean) characters, which we could reasonably expect to be monospaced. This was largely because the approach we took required hard-coding glyph metrics instead of getting them from the font.

16253 overhauls the local glyph rendering code on iOS and macOS, giving developers more flexibility in specifying local fonts but also trying harder to match the intended fonts by default. https://github.com/mapbox/mapbox-gl-native-ios/issues/105#issuecomment-593069163 has some screenshots of local font selection in action.

There’s still code in mbgl::util::i18n::allowsFixedWidthGlyphGeneration() that enables the feature only for the CJK Unified Ideographs, Hangul Syllables, Katakana, and Hiragana blocks in Unicode. However, modifying LocalGlyphRasterizer::canRasterizeGlyph() to ignore that check, as in the 1ec5-coretext-no-remote-7862 branch, shows that #16253 does a good enough job rendering text in general that we might be able to support it for all scripts (but off by default so that styles don’t unexpectedly fall back to Helvetica). Some internationalization issues remain, particularly the treatment of combining Indic/Brahmic characters.

Existing remote glyph rasterization (note missing Amharic in Ethiopia but intact Bengali in Bangledesh):

16253 rigged for local glyph rasterization of all scripts (note dotted circle ◌ on combining characters):

Bloopers

Passing a font-from-font-descriptor into CTFontDrawGlyphs() instead of a font-from-run-attribute:

[1ec5]

More discussion specific to iOS and macOS regarding the existing Core Text/TinySDF approach:

The gist of the coretext-hack branch in https://github.com/mapbox/mapbox-gl-native/issues/7862#issuecomment-362798522 is that the following code in mbgl::style::SymbolLayout’s constructor and shapeLines() need to avoid conflating codepoints with glyph IDs and instead consult platform-specific code to map codepoints to glyph IDs.

https://github.com/mapbox/mapbox-gl-native/blob/66ac55445998718b28a6fbd4a4094a10426627a9/src/mbgl/layout/symbol_layout.cpp#L179-L182 https://github.com/mapbox/mapbox-gl-native/blob/4da54d61cc7e61325e23440fc4578fefd34f52d3/src/mbgl/text/shaping.cpp#L425-L428

This mapping needs to take place when gathering glyph dependencies, not later when rasterizing glyphs as LocalGlyphRasterizer currently does. At that early stage, the font descriptor can be created and resolved just once for an entire run of text instead of per-glyph. This not only improves performance but also respects ligatures and other complex text.

Since glyph IDs differ from one font to the next, the font information needs to be stored alongside each glyph ID. The good news is that glyph dependencies are already bucketed by font stack as of #12624. So instead of adding to the glyph dependencies of the font stack as specified by the style, SymbolLayout would associate these dependencies with a synthetic font stack that contains only the resolved font name determined by Core Text’s CTRunGetAttributes(). Later, when rasterizing glyphs, LocalGlyphRasterizer can simply get the font by font name instead of building a complex font descriptor for each character as in #16253.

Going beyond the basic requirements, Core Text provides a number of text layout functionality that would be useful as overrides to mbgl’s built-in shaping code. For example:

• We can specify kCTFontOrientationVertical as the font descriptor’s kCTFontOrientationAttribute and pass that orientation into CTFontGetBoundingRectsForGlyphs() to get glyphs and metrics appropriate for vertical text, which would allow us to short-circuit the homegrown verticalizing code in the SymbolLayer constructor and i18n.cpp that’s overly reliant on Unicode compatibility forms. The same is likely true of much of the right-to-left layout code.
• By the time SymbolLayout gathers the glyph dependencies, the text has already been reordered by the BiDi code. Ideally, we’d rely on Core Text to do any BiDi processing too.

We could move the homegrown code into platform/default/ implementations and omit them from iOS and macOS builds, reducing the binary size.

The 1ec5-coretext-shaping-7862 has some hacky code that revives and modernizes the old coretext-hack branch based on the approach outlined here. Instead of redefining glyph dependencies across the board, it completely circumvents mbgl’s text shaping in favor of a new Core Text–based implementation, with an eye toward eliminating the need for a local glyph rasterizer.

This branch currently renders the wrong glyphs, making a mess of things, because each feature’s formattedText contains the original string instead of the glyphs that have been rendered by LocalGlyphRasterizer. It would be trivial for getGlyphDependencies() to also convert the string to a series of glyph IDs at the same time, but that would make the original string inaccessible to the code that later performs shaping. It’s unfortunate that we perform shaping only after deciding which glyphs to show – that would seem to defeat the purpose of performing actual text shaping.

[kikothemaster]

Is this gonna be ever possible?