Fonttools: ttx cant dump font (In my case TSIV table)

[0xff00ff]

Hello, i have a problem with some fonts.

I use ttx for dumping a lot of different fonts, some time i have problems with dumping different tables.

For example i will use this font: https://www.dafont.com/sansation.font
ttx cant dump TSIV table of this font.

I tryed to play with this error, and changed this file /usr/lib/python3/dist-packages/fontTools/misc/py23.py (trace stack from screen)
I changed function tounicode from
def tounicode(s, encoding="ascii", errors="strict"):
to
def tounicode(s, encoding="latin", errors="strict"):
and after that changes font was dumped without any error

I use ttx 4.16.1

can i somehow set encoding when use ttx? or maybe ignore this character, etc

[anthrotype]

TSIV is a private table used by VTT (MS Visual TrueType editor for TrueType hinting instructions). FontTools assumes that the table only contains ASCII but somehow that's not true for that particular font. We need somebody that has access to VTT on Windows that can confirm whether TSIV table can contain characters other than ASCII.
I believe for other VTT tables (e.g. TSI1) we are using UTF-8 as the text encoding, maybe we should do the same for all of them, including TSIV /cc @moyogo @madig

@0xff00ff are you actually interested in decompiling this particular TSIV table? If not, you can just exclude it from the ttx dump (-x option). The font should have not been published with that table, it is only needed if you are developing the font's TrueType hinting, but for releases you want to strip that as it's not used for rendering the font.

[0xff00ff]

@anthrotype thanks, i think i will skip this table

also i found new error, i think its not connected to any table, should i create new task for this fonts or we can discuss them here?
https://fonts.google.com/specimen/Noto+Sans+JP?query=noto+sans+
https://fonts.google.com/specimen/Noto+Serif+JP?query=noto+serif

[jenskutilek]

TSIV is the MS VOLT source code, and in this case it seems to contain an fl ligature in a lookup name ;) There seems to be nothing in VOLT that prevents the user from entering non-ASCII characters.

[anthrotype]

ah! thank you Jens, I thought all TSI* tables were for VTT, but you know better.

seems to contain an fl ligature

does VOLT complain about the fl ligature? How is that fl ligature encoded, as UTF-8? If so, we can use that too

should i create new task for this fonts or we can discuss them here?

@0xff00ff it's a different issue (very weird though)

[jenskutilek]

Ah, it's not an fl ligature, that's just what's shown when I copy the TSIV code from a hex editor ;) It's actually supposed to be an ß.

In the raw TSIV data the character ß is encoded as 0xDF. When I open Sansation in VOLT, the lookup name is "Capital ß". VOLT seems fine with it.

[anthrotype]

hm so it _is_ "latin1" after all.. UTF-8 for 'ß' is not 0xDF

>>> hex(ord('ß'))
'0xdf'
>>> 'ß'.encode('latin1')
b'\xdf'
>>> 'ß'.encode('utf-8')
b'\xc3\x9f'

[moyogo]

If I remember correctly, there's no way to know what encoding is being used. When testing, there were fonts that had different encoding for unknown reasons.