Fontbakery: Font Bakery can't be used from within Glyphs

[graphicore]

We recently had a switch to support Python 3.6+ only, dropping support for Python 2.7. I like Py3 as a language, but, I feel like we should discuss the downsides of this move, also because I see occasionally cheering about this and I feel it caused a lot of trouble as well (e.g. it created work downstream and in other teams). Here's my biggest regret so far.

Font Bakery can't be used from within Glyphs (And maybe other editors, what about FontLab and Robofont?) at the moment. Py3 will come to Glyphs eventually, but until then this is just no option anymore and I think we're missing out a great use case.

This is actually a thing I wanted to do and that I had already discussed with @schriftgestalt. The plan was roughly to run source file checks using the same specification from within Glyphs, e.g. as a designer and just plain from the shell or the Dashboard with glyphsLib.

[chrissimpkins]

Can you pipe stdout/stderr from system installed Py 3 based FB to the user through Py 2 based scripts on the font editor side? Were you planning to package and distribute all of the macOS executables required by FB as part of the script or plugin in order to make self-contained editor specific releases?

[graphicore]

Can you pipe stdout/stderr from system installed Py 3

Maybe. But that could not use the glyphs API directly. You would have to save the file and open it from the external FB. I'm not sure though, how the glyphsLib implementation from Glyphs would handle this, could also just create a copy.

Were you planning to package and distribute all of the macOS executables required by FB

No, this wouldn't use the specifications that we have right now, but rather use a special spec to check Glyphs sources. Thus, the executables that we require for googlefonts/open type specs would not be required at all.

[chrissimpkins]

I see. You need to use the Glyphs API to write the tests. I missed that.

How difficult would it be to support a data interchange format that provides FB with the test results for pass/fail interpretation and reporting? This might help to address the 2/3 transition and support new test application development, existing open apps outside of the Python language without the need to parse stdout / err from test binaries.

[felipesanches]

I am not sure what you mean. But maybe what you are asking for is our JSON report which you can get by passing --json report_filename.json.

[chrissimpkins]

I mean the reverse. Tool external to FB provides FB with something like JSON data that includes the results of the tests, formatted in a way that FB can read and understand the data because you spec the format for talk between tools.

[felipesanches]

I think we won't ever get back to python 2 unless there's a very strong reason to do so. I think that at this point, the need/wish of having fontbakery-based checks inside Glyphs may be seen as one more reason to upgrade Glyphs to python 3.

[felipesanches]

And I guess that glyphs-specific checks that would only be possible to run within glyphs itself can be written without the fontbakery checkrunner. Also those would be out of scope for fontbakery itself.

Something different would be fontbakery checks that validate glyphs project source files without requiring to run GlyphsApp. That would be within scope for fontbakery, in my opinion.

[graphicore]

the need/wish of having fontbakery-based checks inside Glyphs may be seen as one more reason to upgrade Glyphs to python 3.

But that's not in our power.

And I guess that glyphs-specific checks that would only be possible to run within glyphs itself can be written without the fontbakery checkrunner.

Every check could be written without FB. This is a none argument.

Also those would be out of scope for fontbakery itself.

I don't see why you draw the line there. The model is pretty much what FontQA was for FobtLab. Using a good framework to write and run checks is a good idea and the reason for FB.

Something different would be fontbakery checks that validate glyphs project source files without requiring to run GlyphsApp. That would be within scope for fontbakery, in my opinion.

I want both, within Glyphs and from the command line. The reason for within Glyphs would be that it's super comfortable for designers, who often don't like to use the command line. Then the same spec can be used to do CI in an automated build pipeline.

[chrissimpkins]

Wouldn't you need a headless Glyphs.app to support automated CI with the same approach that you use in the editor? As far as I know that doesn't exist. If you can use the open source glyphsLib for the source checks why not just develop it all based upon that and use the tests within and outside of editor?

[anthrotype]

From informally chatting with Georg at ATypI, I heard that Python 3 support will land in the next major version of Glyphs 3, though I don’t know what the exact timing of that will be.

[chrissimpkins]

@anthrotype the ball is rolling! :)

[felipesanches]

sure. My point is very much aligned to what Chris Simpkins is saying. If GlyphsApp's API is not in our power, then we should not rely on that as a dependency. That's why I draw the line there.

I am not opposed to having fontbakery checks running form within GlyphsApp. Actually I think it would be pretty useful to increase the user-base. But I suggest we find a way of doing it that allows the same checks to be run without the glyphsapp runtime environment (such as with vanilla fontbakery running on a CI server).

[felipesanches]

what does the internal glyphsapp api provide that a free software python library cannot yet get from glyphs project files?

[graphicore]

glyphsLib: that's the plan or maybe the upcoming https://github.com/robofab-developers/fontParts API. I don't know if Glyphs will support glyphsLib or another API by itself. But we can't run FB within the editor now, that's the issue I'm raising. I understand that you guys like the idea of just developing for py3 but it breaks at least some opportunities we had before, it shuts doors that were open, so to say. I don't share the enthusiasm fully here.

If GlyphsApp's API is not in our power, then we should not rely on that as a dependency.

We wouldn't. How did you get this impression. It's totally possible to write a fontbakery spec that's downstream, not living in the fontbakery repo. You just import fontbakery.

Actually I think it would be pretty useful to increase the user-base. But I suggest we find a way of doing it that allows the same checks to be run without the glyphsapp runtime environment (such as with vanilla fontbakery running on a CI server).

My point.

[graphicore]

I heard that Python 3 support will land in the next major version

Yeah, I have the same info.

[felipesanches]

if you don't need to rely on the internal glyphsapp api, then you probably can spawn a python 3 subprocess from within glyphsapp python 2 environment as someone suggested earlier in this thread. (Sounds super hacky, I know...)

[schriftgestalt]

glyphsLibs object model is similar to the python API in Glyphs. So scripts should work with both.

[graphicore]

It's also not the nice thing you pitch to some designer who's just starting to look into automating stuff with python in their workflow, it's for nerds like you and me ;-)

[graphicore]

glyphsLibs object model is similar to the python API in Glyphs. So scripts should work with both.

See, I really like this :-D Thanks for the info.

[anthrotype]

glyphsLibs object model is similar to the python API in Glyphs.

yes, which means you shouldn't need to have Glyphs.app in order to run fontbakery checks written on top of glyphsLib.
The only argument that was raised in support of running from within Glyphs.app is that "designers don't like to use command line". I think the fontbakery's dashboard is meant to cover that use case, no?

[felipesanches]

OK. Here's an important consideration.

(1) We can put some extra effort to refactor the codebase to get rid of the py3 niceties & cleanup in order to be once again compatible with the old python 2 interpreters.
(2) Or we can do other useful things (like dashboard improvements, or actual fontbakery-core improvements) while we wait for the upcoming python-3 capable glyphsapp.

Which would be better? I vouch for the second option.

[chrissimpkins]

The only argument...

@anthrotype I think that there is a good argument for _source_ checks in the Glyphs editor (and other editors that support Glyphs source files) UI. That is where designers are working on source and it eliminates steps for source checks that include conversion to a web browser, upload of source file(s) for execution rather than a menu selection inside the editor with the entire source path argument and testing process automated for them. As I understand it, that is what Lasse is trying to do here. From a usability standpoint this is the ideal approach for that audience IMO. If the target user is not the general design community but rather developers familiar with the command line who will not necessarily be touching the source for edits, then the argument is exactly the opposite and this adds an unnecessary step of launching the editor to run tests.

[graphicore]

It's (2) obviously. I'm just ranting about that "python 3 vandalism" you guys pulled off. We must discuss this stuff, so we don't act that careless again.

I think the fontbakery's dashboard is meant to cover that use case, no?

No, that's oversimplifying it.

The only argument that was raised in support of running from within Glyphs.app is that "designers don't like to use command line".

That's an example, not an argument. The argument is that the library got less versatile. I don't know all examples, I only know that we lost some options.

[chrissimpkins]

Another point is that you are trying to harness a CLI reporting approach in a GUI environment which may not be ideal nor necessary. If you are going to use the UI, then take advantage of it for your reporting and notifications to users. This likely warrants taking the entire testing approach out of FB source and implementing in a different fashion so that you can do this. Otherwise you will find that you are just creating command line to UI workarounds for tests that have yet to be implemented. Having the same code base for scripted / batch and UI testing argues against this point but it may be easier to make two different projects if the main goal is to harness the UI rather than behave as a CLI app inside a GUI application.

[davelab6]

>

The plan was roughly to run source file checks using the same
specification from within Glyphs, e.g. as a designer and just plain from
the shell or the Dashboard with glyphsLib.

I'm quite alright with source checks specific to .glyphs files to be
written in https://github.com/googlefonts/gf-glyphs-scripts until GlyphsApp
v3 is released, and then do the (pretty small, I expect) amount of work to
port them into the FontBakery specification style and have a "true"
FontBakery GlyphsApp Extension.

We have higher priorities at least until G v3 is out :)

[madig]

I feel it caused a lot of trouble as well (e.g. it created work downstream and in other teams).

Out of curiosity, what downstream and which teams?

Another point is that you are trying to harness a CLI reporting approach in a GUI environment which may not be ideal nor necessary.

This is the real kicker and the reason why I think the vandalism isn't as bad as it sounds. I'm all for in-editor-checkers but before that, we need to sketch out how that should work and look like. I imagine using FontBakery as is, adding some source tests using glyphsLib and slapping in a menu entry in Glyphs simply cuts out the terminal. I mean, we are just going to display a stream of text like on the terminal in the output window.

It could work if we

1. made tests return structured data with e.g. glyph name, contour and node information, etc. and
2. wrote an editor-specific adapter and front-end where people can select the tests and visually cycle through the results.

Think of a mix of "Red Arrows", "Glyph Nanny", "Font Audit" and FontForge's checking dialogues. I anticipate that this needs changes to how FontBakery operates and that the front-end is more important than running FontBakery directly inside Glyphs, etc., so one might as well run FontBakery externally and collect the JSON from stdin.

Maybe looking into "Red Arrows" and "Glyph Nanny" provides clues as to what is needed to make proper use of a GUI. Robofont 3 uses Python 3 and could be used as a test-bed.

[chrissimpkins]

I mean we are just going to display a stream of text...

How about a dialog window that displays results as though the implementation is in the editor rather than as a CLI application addon. Then you can build upon the macOS GUI framework to create something that is more than the typical free text firehose that comes with command line apps :)

The Glyphs plugin implementation is simple enough that even I was able to do it. Have a look at the glyphs filter list manager plugin in our organization account to see how you implement menu list items with Python source in the editor. https://github.com/source-foundry/FilterListManager

Happy to lend a hand if I can be of any help with any part of it that I understand. Perhaps Rainer or Georg or someone else with Xcode expertise would be willing to help with GUI window(s) implementation? For the truly motivated out there, a full fledged independent desktop application for QA is begging to be built :)

[graphicore]

Out of curiosity, what downstream and which teams?

Fontbakery Dashboard and gftools.

I mean, we are just going to display a stream of text like on the terminal in the output window.

The reporting model (FontbakeryReporter) can handle any kind of reporting, we don't have to do this. We can implement a reporter that does GUI stuff. It's a good starting point though, to do the simple thing first.

1. made tests return structured data with e.g. glyph name, contour and node information, etc. and

We can do this, no problem, just yield Fail, MessageWithStructuredData(...) where class MessageWithStructuredData(Message): ... https://github.com/googlefonts/fontbakery/blob/master/Lib/fontbakery/message.py

1. wrote an editor-specific adapter and front-end where people can select the tests and visually cycle through the results.

Why not?

I anticipate that this needs changes to how FontBakery operates

How so?

and that the front-end is more important than running FontBakery directly inside Glyphs

Well, a thing would be to implement one reporter that streams events into a parseable structure, could be lines of JSON for example, that would run outside of Glyphs. When done right, we can just read live from the pipe as the events happen, parse them and output them directly with a generator to a GlyphsFontbakeryReporter. That way, also, once we can run FB from inside of Glyphs it should be easy to switch.

[chrissimpkins]

Same structured data interchange format could be used across your open, non-Python dependencies in order to eliminate parsing exit status codes/stdout/stderr from these command line tools to assess test results. The spec for the data interchange format that you decide upon could support tests across Py2/3 and from any testing tool via piping to FB stdin vs file i/o vs remote web application development with http requests for JSON responses. I think maybe we are thinking along the same lines?

[chrissimpkins]

Then host all dependencies remotely and let client side FB executable, including any GUI implementations, act as a client to the application. This eliminates the x-platform install issues for all if Google is willing to host it :)

Not trying to generate work for you. Just exploring ideas as they come up... sorry

[madig]

wrote an editor-specific adapter and front-end where people can select the tests and visually cycle through the results.

Why not?

Hm?

I anticipate that this needs changes to how FontBakery operates

How so?

I don't know, but no one does until we actually implement something and find things we didn't anticipate. Or not. :)

I'll opine though that before we care about the backend, we need to imagine a frontend.

[felipesanches]

I'll opine though that before we care about the backend, we need to imagine a frontend.

This is just a mockup at the moment, but it sounds promising:
https://github.com/eliheuer/fontbakery-desktop

It is a side-project that @eliheuer has been working on his free time on weekends. It was born after a video-chat between me and Eli right after ATypI, inspired by the feedback we got from users who attended the FontBakery workshop.