Tools: Add hash to file names after building

[idoshamun]

The best practice for managing cache is to change the file name when it is being updated.
This way, one can set the cache to never expire because the file is immutable.
It would be great to see it implemented in polymer-cli.

Take a look at gulp-rev as an example.

[jimbojetlag]

How is the current build system designed when it comes to caching? After building and deploying a new version, how does the user get the new version, while the old ones are cached, and the file names are the same?

The documentation about this is non existent.

[FredKSchott]

We are currently working on a project to make build more customizable. I believe this functionality will be added through that interface once it is built.

pinging @justinfagnani to make sure he's aware of this.

[eeid26]

+1
I am surprised why this feature is not critical. How can you use polymer in production without this feature?
Disable the cache?

[jimbojetlag]

Yes, how is this not critical? Not everyone here is ready to uses service workers.

Adding a hash at the end of the file name shouldn't be that complicated...

[FredKSchott]

@eeid26 @jimbojetlag We're currently juggling a lot of critical projects atm :) While there hasn't been any progress on this issue exactly, we have just completed a major refactoring of the build pipeline and pulled it into its own library that anyone can customize and use: https://github.com/Polymer/polymer-build

If you're interested in this feature, I'd recommend experimenting with adding https://github.com/sindresorhus/gulp-rev to the polymer-cli build pipeline or to a custom polymer-build pipeline if you already have one (see https://github.com/PolymerElements/generator-polymer-init-custom-build for an example project that has one).

And when you get it working, please share what you did here or in a PR! A working demo would be a great first step to getting this prioritized and merged into the CLI.

[eeid26]

I should have a working build/app and a deployed demo app next week

[eeid26]

I created a sample app to use the service worker or browser cache in
https://github.com/eeid26/polymer-app

The sample app uses the request params in the url to update the cache instead of changing the file names. I guess I just followed the way dojo did it before without checking that is not the best way now.

Maybe I will add support for busting the cache with request params or with changing the file name.

[eeid26]

The version that uses the file fingerprint in the url is here https://github.com/eeid26/polymer-app2
Creating two copies of each of the entry-point/shell/fragments was easier to support when the app runs than to rename the files during the build.
This way service worker works without any changes. When service worker is not available, webcomponents-lite handles the mapping from urls without fingerprints to urls with fingerprints to bust the cache.

[rbjarnason]

Thanks @eeid26 I will look to see if this works for me. But... :+1: on making this easier and maybe a standard feature of polymer-cli. I don't see how you can launch a production app without managing this. I use versionAppend in an older app that was based on PSK and I'm in the process of moving it to polymer-cli and missing this feature is a blocker...

[eeid26]

I am sure eventually this will be supported by polymer build process. We just need to experiment with what would work first.
This is not just a build issue. We need to make sure renaming the files works with service worker as a first option, otherwise the app should use the browser cache.

[FredKSchott]

@eeid26 thanks for taking a crack at this! I'll try to take a look at this asap.

One thing that makes this much more difficult in an HTML Imports world is that we're referencing the files by path directly (bleeding edge ftw!). Where other build processes build these references for you and can add hashes inside that mapping step, we would need to modify your actual import paths during build.

[nachoab]

Working with AWS S3 and Cloud Front makes it hard without this because CF ignore S3 until unvalidations are performed.

I came up with this setup, let me know what you think:

• Prepend the version number for the SW files: replacePrefix: v${version}
• Update the fetch fn of the generated SW so it updates the event request url to prepend the version. This is because requests from the client will come without a version: bower_components/polymer/polymer.html but we have requested and stored v${version}/bower_components/polymer/polymer.html within the SW
• Deploys to s3 are copied into the bucket root and into the folder version

/bower_components/* - updated
/index.html - updated
/service-worker.js - updated
....
/v0/... - new
/v1/.. - new

• index.html always loads the root service-worker. Once a deploy is performed, we just need to invalidate the /service-worker.js , which will load everything within the /v${latestVersion}/* and then warn the user about a new version. On next reload new SW will take control and get eveything updated from cache

It seems to be working. Otherwise I can only think of a build process in which I should change all files version names. HTML imports should be changed as well as maybe dynamic imports in js (which I have for lazy load for instance). It looks difficult

What do you think?

[BorntraegerMarc]

Is there an update on this?

[kylecordes]

Arriving from various other SPA framework, build tools, and CLIs, I found the following two things jointly surprising:

• Polymer CLI and the tools underneath it support, out of the box immediately, PWA, service worker, HTTP2, rather impressive bundling, super high scores on all the various tools that rate Web application loading performance, etc. So impressive.
• At the same time, these tools don't do boring old hash-based asset renaming, or some variation thereof, so that the build output web assets can be deployed with caching (ideally, "permacaching") enabled except for the index.html, while still reliably delivering new versions to users without any mismatched assets (JS or otherwise)... including for lesser browsers that don't yet support service workers.

Given how impressive the first bit is (wow!), I'm guessing there is some straightforward temporary workaround for the second bit, maybe someone in the know could write a few words here or elsewhere? Could it be as simple as post-processing the build output with gulp-rev and friends - is the build/ output "safe" to send through such tools?

[jab]

While support for this is not yet baked-in, has anyone found a clean way to modify the out-of-the-box build to add this? (New to polymer build and haven't yet found any places to hook into the build process to customize it (beyond what you can specify in polymer.json) – is that because no such hooks exist?)

[web-padawan]

We use custom build with gulp and polyner-build and the last step is assets fingerprinting via gulp-rev-all + gulp-rev-replace.

There used to be a bug with gulp-rev-all: it could corrupt occurences like dom-module id and is, our workaround was to only use this lib to generate manifest and another one to rewrite references.

[jab]

Thanks! Can you share a few lines of a gulpfile (or npm script etc.) that
can be run on an out-of-the-box Polymer starter kit app to demonstrate?

On Sat, Sep 2, 2017 at 12:38 PM Sergey Kulikov [e-mail]
wrote:

We use custom build with gulp and polyner-build and the last step is
assets fingerprinting via gulp-rev-all + gulp-rev-replace.

There used to be a bug with gulp-rev-all: it could corrupt occurences
like dom-module id and is, our workaround was to onky use this lib to
generate manifest and another one to rewrite references.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
https://github.com/Polymer/polymer-cli/issues/283#issuecomment-326755162,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD94KN7OSnjsYiVKtfBsZSwWYopBLMMks5seYSJgaJpZM4I5Gkj
.

[davidmaxwaterman]

I have to use app cache to support mobile safari (no service worker, yet), so this is crucial and it is delaying things that I have to find a solution, potentially dropping polymer-cli, which is something I don't want to do :/

[BorntraegerMarc]

@davidmaxwaterman if you are only using app cache for safari you might want to take a look at https://jakearchibald.com/2016/caching-best-practices/ :)

[davidmaxwaterman]

@BorntraegerMarc thanks! more reading :) That's a bit more recent than his 'douch bag' one, so that's an improvement.

So far, I think I'm using pattern 1 - the files are essentially static, and I only want to change them when I update the app, so the gulp-rev tool is the one I need using the md5sum.

I really wanted it to be as simple as polymer-cli makes other such tasks that aren't actually the app....then I found this.

[CaptainCodeman]

I think the hashed filename approach works great when the extent of the references tended to be small e.g. index.html -> script.js but with HTML Imports (soon to be ES Modules) and an unbundled build + http/2 server push, any change to a file is possibly going to have more of a ripple / cascade impact on the references that need to change, thus changing those files and anything referencing them and so on.

To make my app CDN cache friendly I went with using an app-level hash - i.e. I add a hash as a top-level folder so files can be set to cache for a year but loaded as a complete and consistent set by changing this hash when a new version is deployed.

This seems to give much better performance for first hit at the expense of possibly having slightly more files re-download when the app is updated (but some of that could be reduced with ETag headers based on the content hash)

[web-padawan]

@jab sorry for not responding you. There is a related polymer-build issue and I have already shared my code in this comment.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.