Inter: Inter is not automatically embedded in PDF export in Pages

[HarshilShah]

Hi there, thanks for the work on this font!

I prepared a document in Pages using Inter, and upon export to PDF while it looked fine when opened locally, some other font entirely was being rendered on devices where Inter wasn't installed locally.

At first I thought Pages just didn't embed fonts at all, however when I tried using another font, Tofino, the embed worked just fine (there was even a tangible file size increase with it).

I can't find much information on this, but according to this thread Pages will embed fonts where the license allows it: https://forums.createspace.com/en/community/thread/48558

Would really appreciate if you could see if the Inter license could be modified to allow for embedding in Pages, thanks!

As a demo, I've attached a tiny Pages document below which uses both Inter and Tofino and the PDF export for the same. You can see that in the PDF, Inter will render correctly on devices where it is installed, whereas Tofino will render correctly always: InterEmbed.zip

[jakerockland]

@rsms I believe this is the same issue that I messaged you on Twitter about. I am running macOS High Sierra (10.13.4) and have encountered the issue using the latest release of InterUI both first with the OTF and then also the same issue with the TTF.

Are you sure that you are not able to replicate the issue locally? I didn't notice the issue until trying to move an exported PDF from my computer to another without InterUI installed, but have noted that I'm able to replicate the issue locally by exporting in Pages and then opening the PDF in Adobe Reader.

[jakerockland]

Here is a .zip of the TTF files that I tried.

InterUI.zip

[rsms]

Digging into this, the data in a font signalling how (if at all) it can be embedded and transferred is the fsType (or fst) in the OS/2 font table (applies to both OTF and TTF.) This value is zero (0x000) for all Inter UI font files, which means "do whatever you want". This is the same value that the Roboto font files has set.

Theories:

a. When Apple's Preview.app sees fsType==0 it attempts to embed the actual font file into the PDF instead of subsetting the glyphs, but either fails or writes it in such a way that the PDF viewer used in your case is unable to read and use the font file. To try: Set fsType=4 with the idea that Preview.app in this case will subset and embed the glyphs instead of the entire font file.

b. Preview.app (or rather macOS's PDF system) has a size limit for embedded fonts that Inter UI surpasses. To try: Make a document using a pre-subsetted version of just a single style/weight of Inter UI that is very small.

c. There's some other metadata in the Inter UI font files that is an issue for the macOS's PDF system, causing it to abort the process of embedding the font and instead just references it by name.

Feel free to investigate any/all of these! I'll try to find time maybe next weekend to look into this more.

[jakerockland]

Interesting... @rsms It seems that it is some variation of option (b), as I am able to get the following to work correctly.

Test.zip

Strangely enough, after exporting the test files included here, I was able to export my original troubled file without issue this time... 😓

Seems like this is not an issue with InterUI then in the end but rather _something_ else and that this issue can be closed?

[rsms]

What follows is a log that I kept as I've been investigating this issue.

TL;DR:

• There's a bug in Apple's PDF system causing some fonts's data to never be written into the PDF.
• This bug is triggered by some still-unknown combination of metrics, metadata and glyph data.
• The solution was a complete rewrite of the build toolchain and the source files

Here's a brand new build:
Inter-UI-2.6-582504497e.zip
@jakerockland @HarshilShah I'd appreciate it if you guys could try this out and let me know if this does it for you. I did manage to repro it by writing a simple mac app (details below), and am pretty confident this solves it, but your help confirming would be greatly appreciated!

---

PDF embedding issue notes

Reproduction steps: (with OTF files installed)

1. Create a new RTF document in Apple TextEdit on macOS and write something set in Inter UI Regular.
2. Use the "Export as PDF..." menu item to create a PDF of the RTF document.
3. Open this PDF in Chrome (drag it onto the tab strip) or some other PDF viewer (like https://mozilla.github.io/pdf.js/web/viewer.html) that doesn't access the file system and thus can't load the font from the system.

Note that exporting a PDF from Adobe Illustrator (and several other applications) yields correct, working PDFs.

Analysis of the broken PDF file

First step is to use pdf-parser to inspect the broken PDF and a correct PDF generated by Adobe Illustrator.

Exporting from Preview uses /FontFile3 13 though _it never actually writes object #13_ which is supposed to be a data object of /Subtype /Type1C containing the font data.

Exporting from Adobe Illustrator does yield a correct PDF and Illustrator does manage to write the data object with the font data.

Running the OTF files through OTFCC seems to fix the issue.

otfccdump InterUI-Regular.otf | otfccbuild -O0 -o InterUI-Regular-otfcc.otf

Now, why is this that Apple's PDF subsystem fails to embed the subset font in the PDF?

We've already made sure that the font has the proper fsType set in the OS/2 table to allow embedding.

Theory 1: Somehow the kerning data and/or other font data is incorrect.

Theory 2: There's a bug in ufo2ft which is used to compile the OTF files. We're using an older version of ufo2ft (and it's hard to upgrade, as the more recent versions has a different API that would potentially require a lot of work on our end.)

Before investigating these theories deeper, we first run the ots OpenType Sanitizer, used by Chrome and Firefox among others to validate font files.

Note: It appears the binaries distributed via Homebrew are somehow broken. ots was therefore compiled from source.

Both ots-idempotent and ots-sanitize ran clean without any remarks on the OTF file produced by our current build chain. We assume the OTF files are therefore correct as far as ots is concerned.

Investigating Theory 1 — incorrect font metadata

Step 1: Use the more recent RoboFont 3 to convert the UFO2 source files to UFO3. RoboFont 3 has the ability to detect some extent of invalid kerning data and recursive component usage. Indeed there was one invalid kerning entry which was fixed in the UFO2 files.

Note: We are not currently able to move to UFO3; see the ufo3 and python3 for ongoing work to convert the project to UFO3.

Compiling OTF files after making these changes and replaying the reproduction steps showed this did not make a difference.

Step 2

Since we discovered that recompiling the font with otfcc fixes the issue, we can compare the output from otfccbuild to the input font file. Wrote a simple kerning difference script misc/kerndiff and looked at the difference of kerning between the font produced by ufo2ft (used by our current tool chain) and the font produced by otfccbuild. However, it showed no difference in kerning data.

Step 3

Use sfddiff to look at more detailed differences between the ufo2ft font file and the otfccbuild font file.

This yielded some interesting results which are unfortunately hard to interpret. There are differences in glyph substitution, specifically in ccmp and calt.

Went through all calt by hand and found a few minor issues that would cause ordering differences. Fixed that and calt differenced went away.

Narrowed the ccmp differences down to some rules inside ccmp02. It appears there's either a misspelled glyph in the @ccmp_acc glyph set and/or a glyph that appears in @ccmp_acc which also matches another rule in ccmp02 (i.e. may appear alone on left or right in some combination incompatible with other rules.)

After a couple of hours of testing and editing, it's clear that otfcc either has a bug or that there's some yet-understood-by-me issue with some particular metrics as transcoding an OTF file produced by ufo2ft with otfcc yields a font file that macOS is unable to read (however, both RoboFont and Glyphs are able to read the resulting font file.) Additionally, ots-idempotent fails on the OTF file from otfcc but this is due to ots-idempotent asking macOS to read the file, which it fails to do. ots-sanitize passes cleanly though.

I also ran the OTF file through a recent version of FreeType 2 and Harfbuzz, both which happily read the font and Harfbuzz was able to correctly do layout.

The conclusion here is that there's some sort of issue with macOS's font system, since all other font interpreters tested correctly loads the font.

Step 4

The font UPM were converted to 2048, a very common UPM for OpenType fonts. The theory here is that macOS has some bug related to CFF data in coordinates systems that don't use a UPM of 2048 or 1000.

Interestingly, this seems to have promise.

First off, the font coming out from fontTools and ufo2ft is still not being embedded by Preview, but loads fine in macOS.

Transcoding this font with otfcc now finally yields an OTF file that is both being embedded by Preview _and_ is readable by macOS.

This may be a viable short-term solution — convert the font to the common 2048 UPM and transcode it with otfcc in the build chain. However, the price is that metrics no longer have nice properties like power of two, which is a big deal for day-to-day editing of the fonts (keyboard increments, grid snapping, etc.) Additionally, a slightly longer build process.

Note: A simple macOS program misc/fontsample was written to generate PDFs using specific font files to speed up testing. Instead of reinstalling a font file, restarting TextEdit or Pages and manually exporting a PDF, simply run fontsample.

Investigating Theory 2 — Bugs in ufo2ft?

This path ended after only a few hours as recent versions of ufo2ft only supports UFO3.

I backtracked reading the source code of various versions of ufo2ft and found the last commit that both retains the API used by our toolchain and supports UFO2 which is almost two years old.

This version yields the exact same OTF data as the previous version used.

Converting the project to UFO3

This is a major task as misc/fontbuild and a large portion of the build chain is deeply dependent on UFO2.

• Weight interpolation
• Oblique "italicization"
• Diacritic glyph generation
• Shape overlap elimination
• Build coordination

Converting to UFO3 means finding new ways to do these things, or rewrite the existing code to use the UFO3 format and API.

A bit of brainstorming:

• Ditching all or most of misc/fontbuild
• Start by running Builder once for each master:
  
  
  
  • without overlap elimination
  
  
  • making sure generateGlyphs() generates glyphs with components
  
  
  • Remember to copy the features file into each UFO2 file first as RoboFont seems to be unable to read & resolve the include directives in the feature files.
  
  
• Taking the resulting UFO files in build/tmp and converting them
  with RoboFont to UFO3
• Perhaps merging Regular and Black into one file, with separate
  layers?
  
  
  
  • Investigate best practices for multiple-master UFO3.
  
  
• Big question mark: mixglyph — how do we create interpolations?
  
  
  
  • Maybe DesignSpace document is the answer?
    
    https://github.com/LettError/designSpaceDocument
  
  

Okay, so after three long days (Labor Day weekend, yay) of pretty intensive work the project has been converted to:

• UFO3 master files
• Glyphs master source file
• Brand new toolchain that is a lot simpler
  
  
  
  • old fontbuild is gone entirely (all of misc/pylib, in fact)
  
  
  • new fontbuild is much easier to use
  
  
  • latest version of fontmake
  
  
  • Recent versions of defcon, fontTools, mutatorMath, ufo2ft, ufolib
  
  
  • Recent versions of numpy and scipy
  
  
  • Font products sanitized by ots (the OpenType sanitizer used by several large web browsers)
  
  
• Test suite based on ots as an extra level of saftety, alongside fontsample, to ensure high-quality solid distributions.

Summary

This concludes a pretty epic journey that started with trying quickly fix what seemed like a small issue—avoiding a bug in Apple's PDF system for font embedding—and ended in an almost complete rewrite of the toolchain and source files.

Now what remains is to test the products carefully and compare them with the 2.5 branch (currently the master branch as of writing), and when confidence level is high, we'll merge glyphs-ufo3 into master and release 2.6.

[HarshilShah]

This is some great sleuthing! And the fix seems to work; documents are correctly exported on my end with this new build!

_{Sent with GitHawk}

[jakerockland]

Awesome, this is working correctly on our end as well! 🎊 Thank you @rsms! 🙏

[rsms]

Thank you for confirming!

[rsms]

Changes have landed in master.