Hack: Proposal: Web font CDN

[chrissimpkins]

We are currently using jsDelivr as our CDN service for web fonts.

I'd be interested in hearing whether this is a valuable service and should be maintained. Updates to jsDelivr currently require manual updates which means this requires time. If it is valuable, is there anyone who would be interested in taking responsibility for the management of updates with this web font CDN service for new Hack releases (or developing an automated approach to push to jsDelivr or developing another CDN service for web fonts)?

Help here would be much appreciated.

TODO:

• [x] confirm that repository size is under 50MB to support automated release on jsDelivr (https://github.com/source-foundry/Hack/issues/284#issuecomment-324499270)
• [x] create approach to update package.json files with new releases
• [x] modify web font CSS directory to build/web path from build/web/css - supports jsDelivr CDN URL of https://cdn.jsdelivr.net/npm/hack-font/build/web/hack.min.css

[MartinKolarik]

jsDelivr switched to a new system and can serve all files from npm and GitHub now. The files would be available here but unfortunately your project is a little over our 50 MB package size limit. I believe some files could be removed from the package to reduce the file size and then everything would become fully automatic.

Btw, we also have usage statistics now, so I can tell you your projects gets ~3M requests/month.

[jimaek]

This would format would be prettier https://cdn.jsdelivr.net/npm/hack-font :)

[chrissimpkins]

@MartinKolarik @jimaek thank you both for your feedback!

your project is a little over our 50 MB package size limit.

This should be addressed when the deprecated source code goes away as of the upcoming v3.0 release. We will confirm. I believe that is the bulk of the 'fat' in the repo.

gets ~3M requests/month.

This is more than I expected. Thanks for passing this information along and also for providing this service to us and to those who are using it! It is very much appreciated!

Out of interest for our own development of the web fonts, are these data public? I would like to see the comparison between the use of the full set and the extended Latin subset. We have an open issue report (https://github.com/source-foundry/Hack/issues/285) to discuss subset use for the web fonts and it may inform the discussion there.

@jimaek :

This would format would be prettier https://cdn.jsdelivr.net/npm/hack-font :)

The npm package was created by an interested user and the package.json file was PR'd in. I haven't performed any maintenance on this file in the repo, nor do I have any significant experience in the npm area. @burodepeper is looking into this. Thanks for letting us know about this option. We will try to create a release directory structure that works with this approach. I am assuming that users would simply add (to HTML) the URL path to the CSS file in the repository using the above as the base URL? We will take a look through the docs to see if we need to tag releases in some fashion so that the new builds are cached on your end as new releases are pushed. The base URL indicates to me that users' web pages update to the current release version as soon as this is cached on your end (i.e. no longer necessary to indicate version number in the URL string)?

Thanks again!

[MartinKolarik]

Out of interest for our own development of the web fonts, are these data public? I would like to see if the comparison between the use of the full set and the extended Latin subset. We have an open issue report (#285) to discuss subset use for the web fonts and it may inform the discussion there.

Yes!
https://data.jsdelivr.com/v1/package/gh/source-foundry/Hack/stats
https://data.jsdelivr.com/v1/package/gh/source-foundry/[email protected]/stats
docs: https://github.com/jsdelivr/data.jsdelivr.com
(this is very new so we only have data for the past few days)

Our website will be also updated to show those stats next week.

We will try to create a release directory structure that works with this approach. I am assuming that users would simply add (to HTML) the URL path to the CSS file in the repository using the above as the base URL? We will take a look through the docs to see if we need to tag releases in some fashion so that the new builds are cached on your end as new releases are pushed. The base URL indicates to me that users' web pages update to the current release version as soon as this is cached on your end?

Both /npm/ and /gh/ work in a similar way, but npm packages provide better UX (for example, they'll be searchable on our new website.

If you use valid semver versions for your tags, then you can use "version aliasing" in the links to get automatic updates to your users E.g. https://cdn.jsdelivr.net/npm/hack-font for latest version, https://cdn.jsdelivr.net/npm/hack-font@1 for latest 1.x.x version and so on. If you don't want this, you can also specify exact versions: https://cdn.jsdelivr.net/npm/[email protected]/

[chrissimpkins]

If you use valid semver versions for your tags, then you can use "version aliasing" in the links to get automatic updates to your users E.g. https://cdn.jsdelivr.net/npm/hack-font for latest version, https://cdn.jsdelivr.net/npm/hack-font@1 for latest 1.x.x version and so on. If you don't want this, you can also specify exact versions: https://cdn.jsdelivr.net/npm/[email protected]/

👍 thank you!

[chrissimpkins]

@burodepeper adding myself to this IR too as the build directory structure will influence the resulting URL here. Will be working on the web font build scripts over the next few days. Let me know what you think about the above and whether you feel that there is a more optimal directory structure for the web fonts than what is in use at the moment. If we do decide to go woff only (https://github.com/source-foundry/Hack/issues/286), we can eliminate an unnecessary level of directories in the current master branch of the repo.

An alternative would be to push web font builds to the top level of a separate repository if we want the "cleanest" URL string. Adds work, maybe doesn't matter on the user end?

[chrissimpkins]

Will modify web font CSS with v3.0 release. See #294

Any updates here?

[burodepeper]

I think publishing via npm is a good approach for our webfonts. We rarely have releases and a manual npm publish minor shouldn't be a dealbreaker for the convenience it adds.

I would like to suggest simplifying our directory structure a little. The current URL with the new API would be https://cdn.jsdelivr.net/npm/hack-font/build/webfonts/css/hack.min.css. Something like https://cdn.jsdelivr.net/npm/hack-font/build/web/hack.min.css isn't a necessity, but less is more, right?

@MartinKolarik or @jimaek: Would that break old links to the CDN?

[chrissimpkins]

We rarely have releases

If this changes, is this still a viable approach? Part of the goal of the build transition is to make smaller frequent releases viable and acceptable (from a burden of release tasks standpoint). There is a fair amount of work to do on things here and I would favor a new release approach as of v3.0 that pushes small frequent releases with changes as they are available rather than large bundles of changes. These don't all have to be deemed "new features" to warrant a separate release, simply a change that is worth having in place.

An example is identification of several glyphs that require new manual hints. Is it really necessary to hold this until new "features" are added (e.g. ASCII set modifications, new glyphs to support expanded sets, etc) to justify merge to master? I would favor an approach that allows us to address things with greater frequency when felt to be needed.

Once this transition is complete, all pieces of the build process and testing will boil down to a couple of minutes of automated (via the new scripts) work. The bottleneck for us will be the manual review process that is required before a merge to master once in place. It seems to me that these manual reviews become easier with smaller chunks of changing pieces rather than lots of moving parts.

[chrissimpkins]

https://cdn.jsdelivr.net/npm/hack-font/build/web/hack.min.css

This is a simple change in the web font build scripts. Will wait for response from Martin or Dmitriy.

[burodepeper]

If this changes, is this still a viable approach?

Sure. The main link to the CDN will just be to the latest (stable) version. We could also have the main link point to the latest release in the 3.x.x range, so we can safely 'break' to 4.x.x if we need to.

[chrissimpkins]

npm will need to prompted (and by extension jsDelivr) to cache the updated font and CSS files with every merge to master though no?

[burodepeper]

@chrissimpkins Whenever we decide that we release a new version, we have to run npm publish minor. So, if every merge to master counts as a release, then yes.

[chrissimpkins]

@burodepeper Does the package.json file need to be updated in any fashion with this npm publish execution?

[burodepeper]

@chrissimpkins The npm command takes care of that.

[chrissimpkins]

@burodepeper ah, I see. So that command would iterate the patch or minor version based upon the type of release (assuming there is also a npm publish patch)? The person who PR'd the file in suggested use of patch version string as the last character of our font version string (i.e. v1.123 becomes v1.12.3) as a workaround for mandatory font versioning specs.

[burodepeper]

@chrissimpkins Hmm, that versioning approach seems a bit tedious to me. Makes you have to think whether to release it as patch or minor. If you screw up once, you end up having to manually fix it. I'm not a big fan of doing things manually.

[burodepeper]

@chrissimpkins So semver looks like release.minor.patch. When you run npm publish minor it will automatically increase the value of minor by one, and set all lower level version bits to zero, in this case patch. Since our font version only has two version bits, I think it is best to leave patch always at zero, and just keep increasing the minor version.

[chrissimpkins]

I'm not a big fan of doing things manually.

Agreed.

This discrepancy between font versioning and standard software semver is a bit of an issue.

Could:

1. do all of it manually and use the patch number workaround (manual approach not ideal at all)
2. only push a limited set of web font releases to CDN when felt to be of value, use npm to automate
3. not use npm and use the jsDelivr system directly with every release (would need feedback re: how to automate this). I previously did this with manual PR's each time we released, also not ideal.

[burodepeper]

The second approach is by far the simplest, and I don't think we need to limit ourselves in the number of releases. Realistically, we could limit ourselves to a monthly release, which I think is already quite optimistic on average. We can easily plan around those moments, which would result in only twelve releases in a year. That's acceptable I would like to assume ;)

[chrissimpkins]

Realistically, we could limit ourselves to a monthly release

monthly web font release you mean?

[burodepeper]

No, just a monthly release in general. Unless there's a breaking issue,
would it be an issue to cache some changes for a couple of weeks?

On Wed, 6 Sep 2017 at 22:03, Chris Simpkins [e-mail]
wrote:

Realistically, we could limit ourselves to a monthly release

monthly web font release you mean?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/source-foundry/Hack/issues/284#issuecomment-327597202,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADiC6ret90FEVmHGtpmfEutBOI7U4oEhks5sfvp3gaJpZM4PAu84
.

[chrissimpkins]

It's reasonable. I guess the question is why do this? Seems we let the work dictate the release schedule rather than the calendar. In some cases might be days, in some cases might be multiple months.

[chrissimpkins]

Ultimately boils down to a conversation that we should have around planning for releases and we should take that out of this thread. Let's have it though.

[MartinKolarik]

You can change the file structure however you like since this will be the first release at the /npm/ endpoint and that is completely separate from the old links. After the first release, you should only change file structure in major versions, i.e. 3.x.x -> 4.x.x.

As @burodepeper already said, publishing to npm (and therefore to jsDelivr) is as simple as running npm publish minor (you can always use minor if you don't want to think about minor/patch, and only use major when there are backward incompatible changes) and you can do it however often you need.

If you want to recommend links like https://cdn.jsdelivr.net/npm/hack-font@3, i.e. latest version in 3.x.x, you should know that the CDNs cache those for one week, so it can take up to a week for new changes to be available for everyone. In general, this isn't a problem, but if it was, we also have an API which allows you to purge the CDN cache at any time by making a simple http request - you can easily make it a part of your publish script if necessary.

[chrissimpkins]

@MartinKolarik

you can always use minor if you don't want to think about minor/patch

I assume that npm follows standard semver specs and only two digits are permitted in the minor version string? Fonts are a bit funny in that the OpenType spec does not permit patch version numbers. We are limited to a MAJOR.MINOR format and the MINOR portion of the version string can be a number up to 65,535. Strange. In any case, this is the constraint that we are attempting to work around with the web font versioning on jsDelivr.

you should know that the CDNs cache those for one week, so it can take up to a week for new changes to be available for everyone

This should not be a major issue for us.

we also have an API which allows you to purge the CDN cache at any time by making a simple http request

but this is helpful to know if it is! Can you point to the documentation on this so that we are aware should we need to purge the cache at any point?

Thanks for all of your help! It is greatly appreciated!

[MartinKolarik]

@chrissimpkins from http://semver.org/

Does semver have a size limit on the version string?

No, but use good judgment. A 255 character version string is probably overkill, for example. Also, specific systems may impose their own limits on the size of the string.

I don't think there are any strict limits you need to worry about.

but this is helpful to know if it is! Can you point to the documentation on this so that we are aware should we need to purge the cache at any point?

You just need to make a GET request to the same URI but at purge.jsdelivr.net instead of cdn.jsdelivr.net. For example, to purge the cache for https://cdn.jsdelivr.net/npm/hack-font@3 you would make a request to http://purge.jsdelivr.net/npm/hack-font@3.

[chrissimpkins]

@MartinKolarik

thank you Martin!

[chrissimpkins]

Here is where we are with directory size in dev branch.

 52K    CHANGELOG.md
 12K    CONTRIBUTING.md
 12K    FAQ.md
4.0K    LICENSE.md
4.0K    Makefile
8.0K    README.md
2.5M    build
 16K    build-subsets.sh
8.0K    build-ttf.sh
8.0K    build-woff.sh
8.0K    build-woff2.sh
 32K    docs
3.4M    img
4.0K    package.json
 44K    postbuild_processing
 27M    source
 48K    tools

We should be gtg with npm > jsDelivr web font CDN use.

[chrissimpkins]

@MartinKolarik @jimaek

Martin and Dmitry-

I reviewed your documentation and it appears that we can use Github releases rather than npm to get the files to the jsDelivr CDN. This would likely work better for us due to versioning differences between npm and the OpenType table definitions (that don't follow semver).

Your documentation on this page https://www.jsdelivr.com/?docs=gh states that NPM should be the preferred approach. I wanted to see if the use of GH releases is a reasonable approach for webfont redistribution via the jsDelivr CDN or if this applies to our project as well. We can definitely go through npm but npm is not necessary for our releases and is an additional step.

If we choose the Github release approach, can you point to documentation (or let us know) how the git tag for the release is converted to a version string in the URL string. We use git tags for releases with the format vX.XXX and I am wondering if this converts to the following URL string:

https://cdn.jsdelivr.net/gh/source-foundry/[email protected]/build/web/hack.css.min

Thanks much for your help!

[chrissimpkins]

For my reference:

jsDelivr documentation for Github releases:

 // load any GitHub release

// note: we recommend using npm for projects that support it

https://cdn.jsdelivr.net/gh/user/repo@version/file

// load jQuery v3.2.1

https://cdn.jsdelivr.net/gh/jquery/[email protected]/dist/jquery.min.js

// use a version range instead of a specific version

https://cdn.jsdelivr.net/gh/jquery/[email protected]/dist/jquery.min.js

https://cdn.jsdelivr.net/gh/jquery/jquery@3/dist/jquery.min.js

// omit the version completely to get the latest one

// you should NOT use this in production

https://cdn.jsdelivr.net/gh/jquery/jquery/dist/jquery.min.js

// add ".min" to any JS/CSS file to get a minified version

// if one doesn't exist, we'll generate it for you

https://cdn.jsdelivr.net/gh/jquery/[email protected]/src/core.min.js

// add / at the end to get a directory listing

https://cdn.jsdelivr.net/gh/jquery/jquery/

[chrissimpkins]

For my reference:

jsDelivr documentation for NPM releases:

// load any project hosted on npm

https://cdn.jsdelivr.net/npm/package@version/file

// load jQuery v3.2.1

https://cdn.jsdelivr.net/npm/[email protected]/dist/jquery.min.js

// use a version range instead of a specific version

https://cdn.jsdelivr.net/npm/[email protected]/dist/jquery.min.js

https://cdn.jsdelivr.net/npm/jquery@3/dist/jquery.min.js

// omit the version completely to get the latest one

// you should NOT use this in production

https://cdn.jsdelivr.net/npm/jquery/dist/jquery.min.js

// add ".min" to any JS/CSS file to get a minified version

// if one doesn't exist, we'll generate it for you

https://cdn.jsdelivr.net/npm/[email protected]/src/core.min.js

// omit the file path to get the default file

https://cdn.jsdelivr.net/npm/[email protected]

// add / at the end to get a directory listing

https://cdn.jsdelivr.net/npm/jquery/

[MartinKolarik]

On npm vs GitHub:

Both /npm/ and /gh/ work in a similar way, but npm packages provide better UX (for example, they'll be searchable on our new website.

Versioning (applies to both npm and GitHub):

If you use valid semver versions for your tags, then you can use "version aliasing" in the links to get automatic updates to your users E.g. https://cdn.jsdelivr.net/npm/hack-font for latest version, https://cdn.jsdelivr.net/npm/hack-font@1 for latest 1.x.x version and so on. If you don't want this, you can also specify exact versions: https://cdn.jsdelivr.net/npm/[email protected]/

In case you decide to use GitHub tags, the version in url can be either the tag you use without any changes, or the tag with leading "v" removed. E.g., v3.000 or 3.000

[MartinKolarik]

GitHub link example: https://cdn.jsdelivr.net/gh/source-foundry/[email protected]/

[chrissimpkins]

👍 thank you very much Martin!!

[chrissimpkins]

@MartinKolarik a new acknowledgment coming in our docs. https://github.com/source-foundry/Hack/blob/dev/README.md#acknowledgments

We really do appreciate all of your help and this service that you provide. Thanks again!

[chrissimpkins]

Web fonts are updated to version 3.000 via npm ---> jsDelivr CDN. Loaded to CDN without issues and gtg! Thanks all!

[chrissimpkins]

Workflow to update npm (for my future reference as I look this up everytime we push...):

$ npm version minor
$ npm publish