Hack: Test Python 2 against Python 3 builds

[chrissimpkins]

Continuation of the conversation in https://github.com/source-foundry/Hack/issues/397

• [x] compare SHA1 hashes for Py2 vs Py3 builds
• [x] compare differences in ttx dump between builds with different versions of the Python interpreters

Question: does the font compile output differ between Python interpreter versions? If so, how?

cc:@anthrotype

[chrissimpkins]

Compiled in virtualenv with fresh installs of fontTools and fontmake, same versions of ttfautohint and ttfautohint build dependencies:

SHA1 hashes Python 2 builds

$ python --version
Python 2.7.14

SHA1 (Hack-Bold.ttf) :
d1f5acacb37be1c7c9d9ce7fad82a6f7a7989c04
SHA1 (Hack-BoldItalic.ttf) :
1bebf8f2397582fc35e66851ee009d292a5e1806
SHA1 (Hack-Italic.ttf) :
7210bf97911f3844ddc3fc4543dcdd2539e2846c
SHA1 (Hack-Regular.ttf) :
d48737263e9b3adc70e9ea25d776e0e88ecb0f23

SHA1 hashes Python 3 builds

$ python --version
Python 3.6.4

SHA1 (Hack-Bold.ttf) :
712782074605b6979369c9b4643d994a11a4e332
SHA1 (Hack-BoldItalic.ttf) :
972cefb92c7614d16962409fe39231a6221cf0f3
SHA1 (Hack-Italic.ttf) :
f51e25716eefd4a2ccd7b4f931c2dfc62cb29082
SHA1 (Hack-Regular.ttf) :
0648da83fbd3bb0d72e79e80550f92acba00f1f6

Different

[anthrotype]

use ttx to dump the fonts and then compare them with a diff tool.

here's a simple shell function that does that (you can add it to your .bashrc or similar)
https://gist.github.com/anthrotype/6c3dce176314ec6ffb116b0f8561bee0

[chrissimpkins]

Thanks Cosimo. Found a bug in ttFont method when I tried to dump separate tables with ttx so that I could review them. Filed an issue report and PR with fix.

Thanks for this script! Will give it a go and see what we find. Will post when I have more information.

[anthrotype]

sorry about that. Behdad recently refactored code around in ttLib to add support for TTCollections and that splitTables features wasn't tested so he didn't realize he had broken it. :disappointed:

you don't need to split tables to compare them, btw. I'm really curious to know what are the differences.

[chrissimpkins]

will run script this afternoon and let you know

[chrissimpkins]

that splitTables features wasn't tested so he didn't realize he had broken it

and no worries! Edge case and glad to contribute back :)

[chrissimpkins]

Had to modify the script slightly as I was getting errors on macOS. This is what I came up with:

# compare fonts with ttx
if [ "$#" -lt 2 ]
then
    echo "Usage: ttdiff FONT1.ttf FONT2.ttf [tables ...]"
    exit 1
fi
first="$1"
if [ ! -f "$first" ]; then
    echo "File $first not found"
    exit 1
fi
second="$2"
if [ ! -f "$second" ]; then
    echo "File $second not found"
    exit 1
fi
tables=""
for i in "${@:3}"
do
    if [ ! -z "$i" ]
    then
        table="-t "
        if [ ${#i} -eq 3 ]; then
            # add trailing space to pad tag to four chars
            table+="'"$i" '"
        else
            table+=$i
        fi
        tables+="$table "
    fi
done
ttx -q -f -o first.ttx $tables "$first"
ttx -q -f -o second.ttx $tables "$second"

# diff -u first.ttx second.ttx | less -R
git diff first.ttx second.ttx

[chrissimpkins]

and here are the results :)

diff --git a/first.ttx b/second.ttx
index 3b05f8d..2dc586e 100644
--- a/first.ttx
+++ b/second.ttx
@@ -1582,12 +1582,12 @@
     <!-- Most of this table will be recalculated by the compiler -->
     <tableVersion value="1.0"/>
     <fontRevision value="3.002"/>
-    <checkSumAdjustment value="0xe4aa7082"/>
+    <checkSumAdjustment value="0xe4aa6cc8"/>
     <magicNumber value="0x5f0f3cf5"/>
     <flags value="00000000 00000110"/>
     <unitsPerEm value="2048"/>
     <created value="Mon Oct 23 16:00:00 2017"/>
-    <modified value="Wed Feb 21 15:57:02 2018"/>
+    <modified value="Wed Feb 21 16:04:59 2018"/>
     <xMin value="-954"/>
     <yMin value="-605"/>
     <xMax value="1355"/>

[anthrotype]

YAY! no differences :dancer:

[anthrotype]

I was getting errors on macOS

probably because you don't have colordiff instaled. brew install colordiff

[chrissimpkins]

Thinking through my build process to make sure that this was the correct approach. I did the following:

• created separated Py2 and Py3 virtualenv
• git cloned Hack into separate directories for each
• activated Py2 venv
• pip installed fonttools and fontmake
• built Hack ttf fonts with make = calls fontmake executable in script
• deactivated Py2 venv
• activated Py3 venv
• pip installed fontools and fontmake
• built Hack ttf fonts with make = calls fontmake executable in script

Do I need to change the path to the venv installed version (i.e. an absolute path) of fontmake or should the fontmake call be picked up by the proper install within the venv?

[anthrotype]

you don't need to split tables to compare them, btw

forgot to mention, that ttdiff function can also compare specific tables instead of the whole font; simply list the table tags you want to compare: e.g. ttdiff font1.ttf font2.ttf OS/2 hhea

[anthrotype]

should the fontmake call be picked up by the proper install within the venv?

oopps pressed "update comment" too soon

[anthrotype]

if you activated the venv, and just run fontmake, it should pick up the right one, yeah

[chrissimpkins]

alright, this is valid then!

[chrissimpkins]

These were fonttools and fontmake installs into Py2 and Py3 isolated venv's.

[anthrotype]

you don;t need to install fonttools separately, it's a dependency of fontmake

[anthrotype]

if you export a SOURCE_DATE_EPOCH variable, the build will be reproducible and also the timestamp and checksum will be identical

[anthrotype]

on unix, you can get the seconds from epoch to now with date +%s

[chrissimpkins]

Are either of those values used on systems to determine changes in the fonts (i.e. to eliminate a cached version on update)? Any reason not to maintain them as different across builds?

[anthrotype]

I'm thinking, maybe you could have so that the build script gets the git commit's date, converts it into seconds since the Unix epoch, and exports SOURCE_DATE_EPOCH so that every time one builds from a commit the output is identical, no matter _when_ one runs the build...

[chrissimpkins]

Ah that is an interesting idea. Let me look into that. I really like that

[anthrotype]

I mean, the produced fonts are entirely determined by the git commit they were compiled from, and the version of the toolchain (which you've pinned down to specific versions with a requirements.txt, haven't you? If not, you should).

Then, setting SOURCE_DATE_EPOCH to the date of the commit will ensure that timestamp will be deterministic, and will always produce the same result given the state of the source files (the git repository at a given commit) and the version of the requirements (and yeah, the version of the python runtime.. but for that we are betting no differences ;).

[chrissimpkins]

cc: @paride Paride, we went through the testing to review Py2 vs. Py3 builds with the fontmake compiler and fonttools dependencies. As of current release versions of both, there is no significant (i.e., that will make a difference in your renders) change in the diff across the files. See https://github.com/source-foundry/Hack/issues/398#issuecomment-367427666 for the raw diff. It is simply timestamp values that differ based upon the time of build. These do not influence anything in your fonts display to users.

[anthrotype]

actually the implementation of SOURCE_DATE_EPOCH in ufo2ft is buggy because it assumes local time instead of UTC so the seconds since epoch may be interpreted differently based on where you run the script...

https://github.com/googlei18n/ufo2ft/commit/54b635298861ec32837aaac1e66cade47fe97420

I actually discovered this where I was in San Francisco, because you know... London is GMT :stuck_out_tongue:

It's fixed on master, and the next version should be coming sometime this week or earlier next.

[chrissimpkins]

I mean, the produced fonts are entirely determined by the git commit they were compiled from, and the version of the toolchain (which you've pinned down to specific versions with a requirements.txt, haven't you? If not, you should).

Have not. Agree with you, probably best to eliminate the pip install [X] and replace with pip install -r requirements.txt

with requirements.txt:

fontmake==1.4.0
fontTools==3.22.0

then bump versions when/if necessary.

[chrissimpkins]

This will have implications for our downstream packages though. Be interested in @paride @spstarr thoughts about pinning these versions? They will be rolling on Linux distros and I think that they like to install from their packaged versions?

[anthrotype]

ehrm.. to be bullet proof you want to pin _all_ requirements, including recursive dependencies of dependencies.

You can do a fresh virtual environment and dump the output of pip freeze to a requirements.txt file, like they do in Ubuntu for example:
https://github.com/daltonmaag/ubuntu/blob/master/tools/update-requirements.sh

Or there are tools to do this automatically, like pip-compile from https://github.com/jazzband/pip-tools

[chrissimpkins]

@anthrotype Will wait on the release to give the SOURCE_DATE_EPOCH issue a try. Will keep an eye out for it.

[chrissimpkins]

to be bullet proof you want to pin all requirements, including recursive dependencies of dependencies.

I really like this. I have a sense that packagers on Linux distros will not like this. Let's see what Paride and Shawn have to say. They are going to periodically bump these dependency versions and will likely want to build with their validated packages for each. If they are building off of our scripts, then this will cause problems. Not sure how to solve that issue and will need feedback.

[anthrotype]

hey I'm not saying anything new I hope..
Fontmake does all its doings by using other libraries. When you pip install fontmake you get an environment which satisfy the minimum installation requirements defined in fontmake's setup.py, but these are open-ended >= and the concrete version for each package will be the one currently on the PyPI index, it can be newer, it may have fixed bugs or introduced new features, hopefully hasn't broken backward-compatibility, etc.

Using a requirements.txt means you freezes the versions with exact specifiers as in == and you will always create the same virtual environment, hence the same output fonts, from a given commit.

[chrissimpkins]

Ideally, we should be able to rely on SemVer across the build tooling, though there are so many moving parts across so many libraries it is difficult this far up the chain to determine what is and is not backwards incompatible.

[chrissimpkins]

No, no not new. We I just have been lax about applying pinned versions for Python deps.

[anthrotype]

I have a sense that packagers on Linux distros will not like this

but that's what downstream packagers do all the time. They record this kind of things very thoroughly and are very much concerned about reproducibility (much more than regular font devs, I reckon)

[anthrotype]

We just have been lax about applying pinned versions

you shouldn't worry too much, on the other end, because I tend to always "bump" the minimum requirements of fontmake every time I release a new fontmake itself, and changes that happen in the dependencies are usually "for the best", rather than "for the worse". But it's good to pin dependencies in production and CI environments.

[chrissimpkins]

but that's what downstream packagers do all the time.

Yeah, I agree and understand. Let me get some feedback from Paride (Debian) and Shawn (Fedora). They are the only two distros that are building fonts from source as far as I know. Unclear whether they are modifying the build scripts in this repo to point to Debian/Fedora specific Python executables/libraries based upon their testing/vetting process but I think that this is the way that it will work. I suspect that they will be defining their own set of pinned versions based upon what is available on the respective distro at any given point in time and those definitions are likely driven by this project and others that require those dependencies. If we begin to define the dependency versions here, this will be something new for them to watch for as our current approach suggests to them that within these minor version releases of fontmake we are ok without version pinning. It also means that they either decide to use what they have available to build new font versions when these dependencies change here or they bump to what we define here, await the vetting process to make those available on the distro, and then release the new version of the Hack fonts.

you shouldn't worry too much, on the other end, because I tend to always "bump" the minimum requirements of fontmake every time I release a new fontmake itself

👍

Your definitions will likely drive the definitions here.

[chrissimpkins]

@anthrotype You willing to apply a license to that modified version of the ttx diff script that you posted in the gist so that we can include it in this repo? I can add it or you are welcome to PR in to the tools/scripts directory if you have the time/interest.

[anthrotype]

I don’t mind you using or modifying it as you like. I hereby place it in the public domain. Is that good enough?

[chrissimpkins]

@davelab6 has cautioned us against public domain. It sounds free/open but there are international legal issues that arise with its use. Possible to apply an OSI approved license like MIT or Apache?

[anthrotype]

Ok then, MIT is fine.
But really this is quick&dirty shell scripting i don’t even want to claim ownership of

[chrissimpkins]

Ok then, MIT is fine.

:)

# Copyright 2018 The Ether
# MIT License

# compare fonts with ttx
if [ "$#" -lt 2 ]
then
    echo "Usage: ttdiff FONT1.ttf FONT2.ttf [tables ...]"
    exit 1
fi
first="$1"
if [ ! -f "$first" ]; then
    echo "File $first not found"
    exit 1
fi
second="$2"
if [ ! -f "$second" ]; then
    echo "File $second not found"
    exit 1
fi
tables=""
for i in "${@:3}"
do
    if [ ! -z "$i" ]
    then
        table="-t "
        if [ ${#i} -eq 3 ]; then
            # add trailing space to pad tag to four chars
            table+="'"$i" '"
        else
            table+=$i
        fi
        tables+="$table "
    fi
done
ttx -q -f -o first.ttx $tables "$first"
ttx -q -f -o second.ttx $tables "$second"

# diff -u first.ttx second.ttx | less -R
git diff first.ttx second.ttx

[chrissimpkins]

Will just use Source Foundry Authors like our main license if you don't want your name associated with it.

[chrissimpkins]

i don’t even want to claim ownership of

No worries. I always take full blame for contents of this repo ;)

[anthrotype]

Sgtm

[paride]

@chrissimpkins technically it is possible to build-depend on specific versions, but this means that the build breaks when the dependencies are updated, as the older versions aren't available anymore. It's not a solution I'd like. Also, if these strong dependencies gets in the way, packagers could decide to just remove the version check.

"Open" restrictions are less problematic, like fonttools >= 3.21.2, but this means that when a version gets released you should check the build against it...

[anthrotype]

the older versions aren't available anymore

why? all the previous versions are available on PyPI

[chrissimpkins]

it is possible to build-depend on specific versions, but this means that the build breaks when the dependencies are updated, as the older versions aren't available anymore

This was my concern.

[chrissimpkins]

@anthrotype I don't think that they are pulling from PyPI. They package their own releases of the projects. When one rolls to a new version, the previous version (in this case the Debian specific packaged version) is not available for use by projects. That is my understanding. Is this correct Paride?

[paride]

@anthrotype in Debian all the dependencies have to be packaged, the distribution is fully self-contained (and source-based).

[chrissimpkins]

This has to lead to tremendous headaches between all of the interdependencies...

[chrissimpkins]

@paride for other forms of software, the package maintainer tests, or some form of automated testing/check happens, against all planned dependency updates before these updates are released broadly to Debian users? Is this part of the standard testing approach across all forms of software released on Debian/Ubuntu?

[anthrotype]

You can maintain two requirements file: one open ended with only top level requirements (fontmake alone, and not all its dependency) and minimum version only; this will be used by downstream maintainers to resolve deps. and another concrete one with pinned versions that is derived from the former (using pip-compile or the Ubuntu fresh venv trick) that you use to run the CI.

[anthrotype]

if you have a file called requirements.in in the current folder and run pip-compile (from pip-tools package), it will resolve all deps and write out a new requirements.txt next to it.
The *.in file only contains the abstract top level requirements, optionally with a minimum version

[chrissimpkins]

The issue for us is one of automation of the testing to a simple examination of test exit status codes. It is possible to CI test for certain issues but for fonts there is a good deal of manual testing of the render that can't be automated.

[paride]

@chrissimpkins strict dependencies are not so common, take for example the file listing the build-deps for a complex package, firefox:

https://sources.debian.org/src/firefox/58.0.1-1/debian/control/

for a few deps a minimum version is required, for the others any (packaged) version will do.

[chrissimpkins]

@paride how do you handle SemVer major version changes in the dependencies? Wait for all dependent software to resolve issues before you push a backwards incompatible update?

[paride]

@chrisclauss that's a possibility. For major backwards-incompatible changes, it is possible to change the name of the package (e.g., libgtk2 and libgtk3).

[chrissimpkins]

Ah I see

[chrissimpkins]

Thoughts about Cosimo's suggestions above Paride?

[paride]

As long as you keep the build scripts/targets with the open-ended dependencies, I think it would work fine...

[anthrotype]

You can’t kill two birds with one requirements.txt.
Running your own build pipeline in a reproducible way is not the same thing as allowing downstream maintainers flexibility to resolve dependencies for their own builds.
You need two set: one abstract, with min and possibly even max versions (e.g. fontmake>=1.4,<2) to avoid breaks when a major semver update comes out; another concrete but derived from the former, for continuos integration and reproducible builds

[paride]

I see. Anyway the Debian build procedure will never call pip, so in practice one requirements.txt should be enough. Just make sure there is a build script/target that assumes the right deps are already there...

[chrissimpkins]

@anthrotype Confirming that the SOURCE_DATE_EPOCH calculation that you referred to in https://github.com/source-foundry/Hack/issues/398#issuecomment-367431523 is coming from ufo2ft? I want to look at the source to get an idea how they are calculating this so that I can figure out an approach to set the value from git commit times.

[chrissimpkins]

@anthrotype also, do you happen to know where/how these timestamp values are being used? Are they necessary for anything out there or would it be possible to simply make the value constant in the fonts across all builds?

[anthrotype]

read the reproducible builds spec to understand what SOURCE_DATE_EPOCH in general means:
https://reproducible-builds.org/specs/source-date-epoch/

As for font files, in the OpenType head table there are two places that contain a timestamp. The head.created and the head.modified:
https://docs.microsoft.com/en-us/typography/opentype/spec/head

The UFO spec allows to set the head.created with the fontinfo.plist field called openTypeHeadCreated:
http://unifiedfontobject.org/versions/ufo3/fontinfo.plist/

If the above field is not present in the UFO (and it's by no means required), ufo2ft.fontInfoData computes a fallback value for it:
https://github.com/googlei18n/ufo2ft/blob/773761be39243b78454ba29b286a221e766cc3ef/Lib/ufo2ft/fontInfoData.py#L59-L68

The openTypeHeadCreatedFallback uses the "SOURCE_DATE_EPOCH" variable when it's present in the environment, or if that's missing it uses the current time (damn!!! I just realised as I'm writing to you that the function dateStringForNow is calling time.localtime() whereas it should have used time.gmtime() in UTC -- well, I didn't write that code :stuck_out_tongue_closed_eyes:.. anyway. I'll fix that later).

So far so good (almost). Now, the UFO spec does not have a corresponding fontinfo attribute for the modified field of the head table. That's traditionally recalculated by the font compiler (fonttools) every time you do TTFont.save().

However, for the sake of reproducible builds, fonttools itself will honour the SOURCE_DATE_EPOCH when calculating the modified timestamp in the head table (else fall back to the current time).
https://github.com/fonttools/fonttools/blob/0740f899a1175888dabc0ff780dce53f8b559f2a/Lib/fontTools/misc/timeTools.py#L50-L55

where/how these timestamp values are being used? Are they necessary...

They are necessary in the sense that the head table struct has a fixed size so you must put something in there. They are not used anywhere by font engines, it's only for informational or archeological purpose, mostly for the developers to know when a font was created and modified.

[chrissimpkins]

This is very, very helpful. Thanks for the detailed response. Let me do some research into it to see what approach we should use here. We don't build with every commit but do attach the git commit SHA1 hash in the version string of the fonts with every build so it may be easiest to simply set the UFO openTypeHeadValue at each bump of the target release version number so that it remains consistent across WIP to reach the release target and then bumps in the source immediately prior to WIP for the next release. If it isn't used anywhere (and I can confirm that I don't ever use it to determine build times, we use git commit SHA1 writes to the name ID 5 version string to link back to source at build) it may not even be necessary to do this but this is simple and I am modifying the fontinfo.plist for our next version target in our source files anyways...

[chrissimpkins]

We could run into issues for work that is intended for one version but winds up in a release further down the road but this rarely takes place in this project and this would only be an issue for builds that are intended for testing, not for release. And the testing phase is when we need this to remain consistent so that we can review the OT tables with diffs...

[chrissimpkins]

Missed the head modified point when I first read through it. Will give this a detailed read and review the links over the weekend. It sounds like the tables still differ unless SOURCE_DATE_EPOCH set but there are still some time stamp calculation issues at play that may require a fix before any approach works as suggested.

[anthrotype]

yes, you can't freeze the head.modified with the UFO alone (only the head.created). You have to set the SOURCE_DATE_EPOCH to ensure that head.modified doesn't change depending on when you compile the fonts.

[chrissimpkins]

The most useful approach from a developer standpoint is to achieve this:

• Font file OpenType tables that demonstrate changes to the UFO source as work progresses
• Font file OpenType tables that demonstrate changes to the post compile hints where write takes place in post compile phase and is not reflected in source file (text) changes
• Font file OpenType tables that do not differ based upon any other parameters that are irrelevant to renders and validation of the files, including the time dependent nature of compiler writes to the head table.

All through a routine text diff of the ttx dump of the binaries.

[chrissimpkins]

The above would allow us to review these diffs as we bump dependency versions for instance (or change to Python 3 build defaults as we have reviewed) with assurance that we are not causing unintended, unknown, and difficult to evaluate / interpret problems in the font files.

[chrissimpkins]

It might also permit us to be more lax about dependency pinning, including across downstream packages / downstream repositories. If we have a solid approach to diff the binaries for changes, the dependency version does not matter. If diff fails, it requires review before a release. I suppose the diff simply needs manual review before releases. There won't be a way to diff to an exit status code of 0 in order to automate this. Even with this (which would be a new approach here), elimination of things that are not important to the render / validity of the font file for installs/caching etc should ideally be eliminated from the diff.

[anthrotype]

I think you're over-thinking :)
A timestamp is just a timestamp. It only tells you, well, the time. Nothing more, nothing less.
If, and only if, you're interested in reproducible builds, then you want to control the timestamp in the output binaries so that they only change when the sources change, not every time you run a build from the same sources.

If you're interested in comparing two fonts for various reasons, you can (and should) exclude the timestamp (and related checksum) differences from your diff, because they don't make _any_ difference, apart of course from the time. That's it.

[spstarr]

@chrissimpkins currently Fedora is targeting python-fontmake-1.4.0 once the package is approved, we have fonttools-3.22.0 currently, so those versions are fine.

The actual hack RPM will depend on the distro installed versions of these tools.

[chrissimpkins]

@spstarr thank you very much Shawn!

[chrissimpkins]

@anthrotype I am interested in trying to support reproducible builds for downstream compilers so that it offloads the burden of testing on their end if they change build dependencies from the upstream. It sounds like this is bound to happen and I haven't come up with a good approach to address it. We could dump ttx XML files and let them diff against that but they are massive and we have repository size restrictions to use our web font CDN's so that will cause problems. Ultimately it may just require visual proofing by package maintainers but I am hoping that this is not the case. I do this with each release and it is both time consuming and painful. Many of the problems that arise are not systematic in a way that you can appreciate them with a small set of glyphs on visual inspection. Trying to make this a more "software'ish" approach to development but I think that there will always be some manual visual testing involved.

[chrissimpkins]

@spstarr we are trying to work through an approach to help establish consistent (defined as both visual render and valid font binaries) builds with different moving parts in the upstream and downstream packages. Debian group told us that they will be advancing build dependency versions independent of those defined in this upstream repo using packaged versions as you are doing on Fedora. I'd be interested in your feedback on how pinning versions of our build dependencies upstream will affect your downstream builds and how you QA the font binaries. The only assurance that we can provide about valid files cross-platform with expected renders will be with the tooling versions that we specify here. We rely on SemVer for our build tools but have found that with many interdependent moving parts in the build toolchain, SemVer may not apply across all tooling.

[anthrotype]

It's just a matter of exporting an environment variable, and it already works now (well, minus a few bugs with localtime vs UTC that were fixed in the past week).

But I think you can't expect that the fonts you build "upstream" will be exactly identical binary-wise, like in diffing the blobs in a hex editor, to the ones that will be produced downstream by the linux distros.
This is bound to fail, and it's not even desirable.
You want the two sets of fonts to be visually and functionally equivalent, not identical.

[spstarr]

@chrissimpkins It's a good question, I can ask the Fedora QA folks, there is some font testing but I think it's mostly along the lines of installation vs font hinting/quality/anti-aliasing etc. If the dependency is within an RPM we can set build versions that are supported for example, if we say 3.0.0 onwards we can guarantee X v1.0 and Y v2.0 tools will build but come the release of 3.0.5 the RPM can enforce requirement that it will not build without X being >= v1.1 and Y being >= v2.1 for example also.

It's a moving target so we don't freeze, but we also don't bump versions unless a security need or other users want new features not found in older tools.

[spstarr]

That said, X and Y RPM versions have to be updated, but once an RPM is in Fedora updating to a new version doesn't require approvals the package manager just pushes the changes and it gets into the next rawhide build or backported if requested.

[chrissimpkins]

You want the two sets of fonts to be visually and functionally equivalent, not identical.

@anthrotype Absolutely agree! But we are working within the confines of software tooling for diffs. Do you know of any projects to provide such testing in an automated fashion? I would love to see a "visual diff" tool emerge if it doesn't already exist. There should be a way to do this so that you can set some sort of rendering parameter against a prior font build and receive notification if there is a rendering discrepancy that is above that criterion glyph by glyph. It wouldn't apply to all platforms X renderers but it would at least be some automation of something that is currently very, very manual and not comprehensive.

[anthrotype]

https://github.com/googlei18n/fontdiff

[chrissimpkins]

Haha! I knew that it had to exist. Awesome. Will check it out.

[chrissimpkins]

Closing as resolved. As of the tested versions of Py2 and Py3 interpreters with current versions of Python build tooling, there are no observed differences in the compiled fonts.