Hack 🚀 - Update versioning approach

My suggestion, as summarized (and slightly extended) from my comments on Slack:

Version number follow the pattern: a.bbb, where a is the major version, and bbb is the release version.
We are currently working on major version 3, which will not change for a while. I consider this the same as the major distinction in semver.
The minor distinction in semver relates to our releases. Releases are _public_ and _stable_. They increase our release version by 1. Our v3.004 is equal to semver v3.4.0.
The patch distinction in semver relates to our test builds. They are essentially _private_ and _not stable_. They are not meant for distribution. To designate a test build, we add the (short version) SHA-1 hash of the latest pre-build commit to the version number it is based on. For instance: v3.004-483bf1f.

burodepeper on 31 Aug 2017

I like this and agree. We can't touch the actual version string but can use the following:

v3.004; 483bf1f

with a git SHA1 (short) stamped on every build in the version string. We can opt to have it not occur on builds intended for release, though I suggest that we stamp every build. This will help us to distinguish something that was built from commits in the Hack repository from something that was not since we are planning to eliminate the reserved font name. This will add a git Python library dependency to the release automation, though, this could be optional and only stamp commits if the library is installed and default to no stamp (in which case it was not a release from this upstream) if it is not. The dependency will cause problems for the Linux packagers (e.g. on Debian, all build tools must meet Debian Free Software Guidelines and be formally approved and accepted as Debian packages, as are all dependencies of the build tool...). Rather than deal with additional review processes to allow for redistribution on these platforms, I think that we keep this as an optional step since it is intended for our own development of the fonts (i.e. communication about what repository state the fonts were built from).

@texhex wanted to weigh in too. Let's see what his thoughts are and we will make a decision.

chrissimpkins on 31 Aug 2017

Let's add one more step then, to make a difference between releases and builds.

v3.004; 483bf1f-release
v3.004; 483bf1f-build (or -dev, or whatever)

burodepeper on 31 Aug 2017

Like it! Now to find a foolproof way to make sure that the "builder" applies it :)

chrissimpkins on 31 Aug 2017

$ make release-build

$ make dev-build

chrissimpkins on 31 Aug 2017

The other thing we can do with this is to use the sha1 as a 'cache breaker' string on the web font CSS.

chrissimpkins on 31 Aug 2017

👍1

$ make css

chrissimpkins on 31 Aug 2017

Web font CSS = #294

chrissimpkins on 31 Aug 2017

I agree with the above noted approach, but I'm against the proposed version schema of having leading zeros.

Although we can not fully match semver (as we do not have a build number), we should still try to respect the rules it defines. And §2 says MUST NOT contain leading zeroes. So instead of v3.004; 483bf1f-release we should use v3.4; 483bf1f-release which can be also be read as v3.4.0, so semver compatible.

I would also like to suggest that when a dev-build is done, the filenames should be "Hack-X-Dev.ttf" and the names should be "Hack X (Dev)". This would allow interested users to install this DEV build beside the latest release version of Hack to see if everything is still working. If not, they just change the configuration to use "Hack" again, no new installation hassle required.

texhex on 4 Sep 2017

@texhex

but I'm against the proposed version schema of having leading zeros.

Version numbers of fonts are in the x.xxx format. That's a fact, nothing to work around.

burodepeper on 4 Sep 2017

May be possible...

It seems the build tools have defaults but the spec itself defines it as any integer up to 65535. I just looked into this in more detail last week as we opened issue report.

Seems to indicate that you can use X.X and increment the minor version up to 65k times... Must terminate with an integer value so cannot add dev or release strings on this part of the version string. Supposed to use semicolons between blocks of other metadata in the string.

See OT docs here under nameID 5

https://www.microsoft.com/typography/otspec/name.htm#nameIDs

With the new font-v tool we will be able to convert the version string to anything that we want so we do not rely on how the fonts are compiled.

chrissimpkins on 4 Sep 2017

@chrissimpkins I wonder how font software will treat it. Perhaps x.xxx is not as official as we thought it was, but it has possibly become the de-facto standard?

burodepeper on 4 Sep 2017

For renaming of dev builds, we could approach this via either the source (change name during dev builds, then change back when ready for release) or by post compilation modification of the name tables which would require us to build another new tool. The latter approach is more optimal if we do this.

The new tool will take some work / effort though it shouldn't be too much trouble. I guess it depends on whether we expect a significant amount of this type of A/B testing on same system to take place in order to warrant it. Is this approach more ideal than fresh installs on Win to confirm a fix because of the font caching issue?

chrissimpkins on 4 Sep 2017

I wonder how font software will treat it. Perhaps x.xxx is not as official as we thought it was, but it has possibly become the de-facto standard?

It seems that above is the way that the compiles occur in the compilation tools in the font editors. Also with the command line tool that we are using. I looked through a lot of fonts and there are a broad range of approaches, including ones that do not fit spec. There are professional fonts out there that are versioned as @texhex suggested. We can change to that approach but should test across platforms to confirm that we don't generate new problems. It appears that versioning should be accepted by tools that render fonts because it does fall in the specification.

chrissimpkins on 4 Sep 2017

This tool is now available for modification of version strings in any of the OT name tables (i.e. across any of the platformID) included in fonts:

https://github.com/source-foundry/font-v

Adding tests and docs this week. We will be able to use this to both modify the version number (though shouldn't be necessary as this should change in source rather than be a post build change), stamp a SHA1 short hash string from the current repository commit, and add -dev and -release tags on the sha1 string.

chrissimpkins on 5 Sep 2017

Did we make a decision about the versioning approach here? I believe that by default, compilation from our source will yield X.XXX with fontmake. We will need to adjust it to X.X manually with the font-v tool (or script this) in order to modify this default.

chrissimpkins on 7 Sep 2017

From that thread, nothing has been settled yet. And me personally, I am fully for semver, since it’s easy. We need to perform extensive testing or field inspection in order to find out support for this version scheme. Too bad that fonts do not support semver x.y.z and x.y.z-data directly. For my projects, I use 2.9.3, 3.0.0-dev for development and 3.0.0-alpha.1 for prereleases.

mynetx on 27 Sep 2017

@jublo thoughts about how to approach semver within the confines of font version strings?

chrissimpkins on 27 Sep 2017

@chrissimpkins Well, what possibilities do we have exactly, within the official standards and within the allowances of software?

mynetx on 27 Sep 2017

what possibilities do we have exactly, within the official standards and within the allowances of software?

I wrote my thoughts on that @ https://github.com/googlefonts/gf-docs/blob/master/ProjectChecklist.md#versioning

davelab6 on 27 Sep 2017

@davelab6 ty!

semver.org is growing in popularity as a deeply considered way of versioning software that uses 3 version numbers, MAJOR.MINOR.PATCH, but it does not work well for fonts because the OpenType standard only allows binary font metadata to have one period separator.

it would be ideal to be able to replicate the semver approach... alas.

It would be good to have some note in the version string where possible like 'development version' that is removed when making a release build. Some systems will not accept fonts with a version number of 0.something so that can be good to use in development sources.

This is the approach that we have landed on (subject to other opinions) as of now. We are "pre-versioning" builds according to the target release version and tagging the version string with the git sha1 short commit hash (to indicate actual repository state at build) and a "release" or "dev" indicator to indicate whether the build (which we distinguish from a release) is intended for general consumption or not. This takes it one step beyond your description above to provide an explicit label for releases (in addition to development builds) and a label outside of git tags for repository state at build.

The format looks like this:

Version X.XXX; [sha1 string]-[(dev|release)]

because I am under the impression that it is not possible to modify the integer string preceding the semicolon with non-integer characters.

Notably, this versioning issue interferes with our web font CDN approach which is via npm. npm forces the semver issue and requires us to work around the font versioning format, something that is not completely resolved at the moment.

chrissimpkins on 27 Sep 2017

An alternative approach is the one that git itself uses with its own version strings when you build from the HEAD of the git repo. They have a format like this:

git version X.X.X-[number of commits since the last version tag]-[sha1 string]

@jublo and I discussed this and my feeling is that there isn't valuable information in the number of commits since last git tag beyond what one can ascertain from direct inspection of the git commit history via the sha1 that is in the version string. It seems more appropriate to define work beyond the git history of a previous release as associated with the next release IMO. It is just preliminary work in progress.

chrissimpkins on 27 Sep 2017

@davelab6 does an improperly specified version string lead to predictable failure behavior on the part of compilers, renderers, OS's or is this undefined and software dependent? i.e. does OpenType specify that compiles, installs, renders should fail when this format does not follow spec?

Wondering if we can just ignore the specification and use semver versioning without issues. Not something that I've tried. I know, heresy :)

chrissimpkins on 27 Sep 2017

@jublo to add to the documentation that Dave linked above, OpenType specifies that you can use up to a count of 65535 in the "minor" portion of font version strings. In theory you can iterate up to a count of something like the following:

Version 1.65535

for any major version. Trivial knowledge because one's head would explode if you require that number of minor releases for any given major version. However, that is the definition.

See nameID 5 here for complete definition: https://www.microsoft.com/typography/otspec/name.htm#nameRecord

chrissimpkins on 27 Sep 2017

Hmm, so 1.9 and 1.00009 will eval to the same?

On Sep 27, 2017 4:41 PM, "Chris Simpkins" notifications@github.com wrote:

@jublo https://github.com/jublo to add to the documentation that Dave
linked above, OpenType specifies that you can use up to a count of 65535 in
the "minor" portion of font version strings. In theory you can iterate up
to a count of something like the following:

Version 1.65535

for any major version. Trivial knowledge because one's head would explode
if you require that number of minor releases for any given major version.
However, that is the definition.

See nameID 5 here for complete definition: https://www.microsoft.com/
typography/otspec/name.htm#nameRecord

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/source-foundry/Hack/issues/293#issuecomment-332648640,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAP9y5dYoATyAVm77Jv4kJo-Ky4Pb0XPks5smrLogaJpZM4PISNs
.

davelab6 on 27 Sep 2017

Yes, believe so. I have seen some fonts that use no leading 0's, many that use a total of three integers irrespective of the length of the minor number. The latter seems to be the most common approach. It seems to be the common approach in commercial fonts at least though that may be a compiler dependent definition rather than spec'd by a development team?

chrissimpkins on 27 Sep 2017

I think that both halves of the version number are stored as unsigned 16 bit integers which is why that definition exists.

chrissimpkins on 27 Sep 2017

In ufo2ft (hence fontmake), we join UFO versionMajor and versionMinor with period, and pad versionMinor with zeros on the left, up to max length 3:
https://github.com/googlei18n/ufo2ft/blob/fe3befae77507786ef322630983e549e298dfebe/Lib/ufo2ft/fontInfoData.py#L83

The same does RoboFont.

anthrotype on 28 Sep 2017

@anthrotype Thanks Cosimo. To confirm, the returned string in that function is what you write to nameID 5 from the source version definition on compilation of ttf/otf files? This leading zero approach seems to be common across compilers.

It appears zfill only adds leading zeroes when the string is under the character limit. It does not restrict the length of strings to the character limit defined in the string method:

zfill

It is very unlikely from a practical standpoint that anyone would need it, but it is possible to increment the minor version over 999 using ufo2ft/fontmake. Also possible to include a period in the minor version string to mimic semver with these tools (not suggesting good idea) unless there is an integer test somewhere during the compile process.

_For my own future reference_:
builds with ufo2ft/fontmake will write up to maximum of two leading zeroes (e.g. 1.001) for version strings that are compiled from definitions in UFO source

chrissimpkins on 28 Sep 2017

@davelab6

To address, your question:

zfill2

No idea whether or how these are detected as different versions on OS's during installs or by applications dealing with caching issues.

chrissimpkins on 28 Sep 2017

is what you write to nameID 5

yes, we use versionMajor and versionMinor as a fallback when an openTypeNameVersion is not explicitly specified in the UFO fontinfo.plist.

zfill only adds leading zeroes when the string is under the character limit.

correct

it is possible to increment the minor version over 999 using ufo2ft/fontmake

yes

Also possible to include a period in the minor version string to mimic semver

No. versionMinor must be "non-negative integer" according to the UFO spec. But you could set openTypeNameVersion to an arbitrary string if you want.

anthrotype on 28 Sep 2017

👍1

(oops, I inadvertently clicked "Comment" before finishing)

I think 1.9 and 1.0009 do not eval the same as far as nameID=5 is concerned. I believe they are simply compared as strings, hence they are different. The 65535 limit I believe has to do with the head major and minor versions which are unsigned integers, and the intention of keeping the two version specifications in head and name table in sync with each other.

anthrotype on 28 Sep 2017

👍1

@anthrotype ty! that is very helpful information!

chrissimpkins on 28 Sep 2017

font-v version string modification currently leads to a bizarre string formatting issue when displayed in OTM_Light.

See https://github.com/source-foundry/font-v/issues/12

I can confirm that after font-v edits to the version string, the version string is not displayed in FontBook so I consider this a definite bug and suggest that we do not use this to modify release build version strings until there is a fix. Will go with version string as defined in fontmake compile from UFO source for the time being.

Can confirm that it does not influence installs/renders on OSX and will check whether this is the case on Linux and Windows. If it does not cause problems with installation, we can continue using it for development builds as a means to determine what build we are dealing with. The version string reporting with font-v displays the strings as intended in the terminal and can be used to evaluate the version string for these builds.

Will look into it.

chrissimpkins on 29 Sep 2017

font-v fixed https://github.com/source-foundry/font-v/issues/12 thanks to @anthrotype

Clear to use v0.3.1+ of the tool to label all builds again.

chrissimpkins on 3 Oct 2017

Do we have consensus here re: @burodepeper versioning approach to bridge the semver / non-semver divide across our build files and the CDN "releases" via NPM? The approach was described as follows in https://github.com/source-foundry/Hack/issues/293#issuecomment-326264190:

The minor distinction in semver relates to our releases. Releases are public and stable. They increase our release version by 1. Our v3.004 is equal to semver v3.4.0.

So we release font builds as v3.000 with a web font CDN (via package.json) version string of v3.0.0. v3.001 release then becomes CDN release v3.1.0... etc. It's not optimal, is a workaround for the inflexible font versioning vs. mandatory semver in NPM, but it remains consistent / predictable across releases.

chrissimpkins on 12 Oct 2017

@chrissimpkins: I believe we do. The test 'release' of today was just a test in that sense. v3 releases will follow proper version naming.

burodepeper on 12 Oct 2017

👍1

@burodepeper I checked jsDelivr and it never pushed because of the repo directory size but we tested with master which does not have the wonderfully parsimonious changes of the dev branch yet :)

Checked repo directory size and we are well within the 50MB limit now. Should be good once we merge to master.

chrissimpkins on 12 Oct 2017

@chrissimpkins Ah, that makes sense.

burodepeper on 12 Oct 2017

I'm reading this thread and the issue description has an URL referencing the master branch. That file is not on master anymore, so I looked up what the code looked like back in August 30th and I think this is a good reference to that file back then: https://github.com/source-foundry/Hack/blob/12806fd0f36a6611ef72dc4616eb8708402a7beb/postbuild_processing/posthinted_builds/dev-versioner.py

It may make life easier for others who may read this thread in the future ;-)

Please consider always using commit-specific repo-URLs instead of simply pointing to files on master branch (which is a moving target).