manifest 🚀 - Add a unique identifier for a PWA

@adewale, thanks! we will see how the gulliver project handles that and if a solution emerges.

marcoscaceres on 23 Jun 2017

The spec does specify a tuple that uniquely identifies the app, but unfortunately it's got a huge problem (I thought there was a bug on it but I can't find one, so I just filed #668).

The steps for processing a manifest are given by the following algorithm. The algorithm takes a string text as an argument, which represents a manifest, and a URL manifest URL, which represents the location of the manifest, and a URL document URL.

This means that the entire identity of the app is uniquely determined by the tuple (text, manifest URL, document URL). Though practically, since text can be derived from manifest URL, it means just the pair (manifest URL, document URL).

I think the fact that it is a function of document URL is a problem, as outlined in #668. If we fix that, then the manifest URL becomes the sole unique identifier of an app.

(Note: The Service Worker URL does not need to be included as part of the identifier. The SW is an implementation detail of the app --- a detail that we require, but we do not need to know where it lives, what its scope is, etc.)

mgiuca on 2 May 2018

https://pwa-directory.appspot.com/ has a collection of 1366 manifests; of these around 71 (5%) look "versioned":

$ curl -sSL 'https://pwa-directory.appspot.com/api/pwa/?limit=4000' | jq -r '.[] | .manifestUrl' | sort | perl -ne 'print if /[0-9a-fA-F]{7}/ || /v[0-9]+/ || /v=/'
https://ademola.adegbuyi.me/_nuxt/manifest.1c4bdc21.json
https://app.mangahigh.com/fea_201803191329/misc/mobile-manifest.json
https://assets.production.spokeo.com/assets/v9/manifest-25a702bcac88b536992cff4cc78d9e75d7d40dc36f746ed69604a2c40d0aba5d.json
https://beta.mic.com/manifest.json?b=1478894131181397
...

https://ademola.adegbuyi.me/_nuxt/manifest.1c4bdc21.json
https://app.mangahigh.com/fea_201803191329/misc/mobile-manifest.json
https://assets.production.spokeo.com/assets/v9/manifest-25a702bcac88b536992cff4cc78d9e75d7d40dc36f746ed69604a2c40d0aba5d.json
https://beta.mic.com/manifest.json?b=1478894131181397
https://betcruncher.com/manifest.3183cc2d8ff6fa85748fc8c6a4f796cd2a95d2e9.json
https://big-andy.co.uk/content/themes/v5/manifest.json
https://blackjack.io/manifest.9f463e8a23e16b31f7219dce967e1df6.json
https://blendle.com/manifest-5a96b3b4ec.json
https://boardom.io/manifest.json?v=3
https://bookourplane.com/manifest.json?v=LbbRAnjJQL
https://browsersync.io/manifest.json?v=qAqkxQaJm0
https://cdn.bloodhorse.com/current/favicons/manifest.json?v=KmbG9gpjz7
https://cdn.getyourguide.com/static/c6754d394589/customer/desktop/static/manifest.json
https://cdn.lyft.com/webclient/icons-463e5ce/manifest.json
https://cdn.shopify.com/s/files/1/0014/1962/t/21/assets/manifest.json?17982843544509738478
https://choualbox.com/manifest.json?v=1282
https://clay.io/manifest.json?data=eyJpY29ucyI6W3sic3JjIjoiaHR0cHM6Ly9jZG4ud3RmL2QvaW1hZ2VzL3N1cGVybm92YS9pY29uLnBuZyIsInNpemVzIjoiMjU2eDI1NiIsInR5cGUiOiJpbWFnZS9wbmcifV0sInNob3J0X25hbWUiOiJDbGF5IEdhbWVzIiwibmFtZSI6IkNsYXkgR2FtZXMiLCJzdGFydF91cmwiOiIuLz91dG1fc291cmNlPXdlYl9hcHBfbWFuaWZlc3QiLCJiYWNrZ3JvdW5kX2NvbG9yIjoiI2ZhZmFmYSIsInRoZW1lX2NvbG9yIjoiI2ZmOGEwMCIsImRpc3BsYXkiOiJzdGFuZGFsb25lIn0=
https://cs1.wettercomassets.com/wcomv5/images/icons/favicon/manifest.json?201708031719
https://d1c42d2bmccy49.cloudfront.net/manifest.json
https://dev-quests.appspot.com/static/manifest.b9d743cdb670650edbb180662a9443e56add2d7fcbc9e7c5d7f73c7bfd20ded5.json
https://developer.chrome.com/devsummit/static/manifest.32a1e88bd98d232c73fbf2f2c5ff552b4c9782f991d6114a3ffa17c5f9390528.json
https://devpractic.es/notifmanifest.php?v=635
https://direct.asda.com/on/demandware.static/-/Sites-ASDA-Library/default/dwb2a11ac9/Manifest/manifest.json
https://ephemeral.now.sh/manifest.91ccc2dacd83c8815c8286043c23a9ae.json
https://erwinandres.github.io/tudu/manifest.json?v=2
https://facerepo.com/app/images/favicons/manifest.json?v=a701bd98
https://feeddeck.glitch.me/manifest.json
https://flat.io/manifest.json?v1
https://grocery.walmart.com/js/icons-4b00caed44fcb95f57dd4efc82d1a2c2/manifest.json
https://hn.nuxtjs.org/_nuxt/manifest.d7491a08.json
https://hpbn.co/7a58c37113db4464699ec4f4646b5566.json
https://jimdo-dolphin-static-assets-prod.freetls.fastly.net/cms/static/manifest.c4bb9662.json
https://kuranz.com/manifest.0252de652255e03775ee2f57d96ec003.json
https://m-travel.jumia.com/manifest.9cd19691.json
https://m.apkpure.com/manifest_v10.json
https://m.avito.ru/s/mobile/web-app-manifest.json?5e1ff91
https://m.badoo.com/badoo/manifest-en.json?v101
https://m.gala.de/r1519124462501/manifest.json
https://magento-imagine-2018.firebaseapp.com/_nuxt/manifest.555d3617.json
https://magnetis.com.br/assets/magnetis_app/manifest-638829635f8669ddb668e944e37aee4241964bd32d7f2dd857d1d7c8e16e8bfd.json
https://memoui.com/static/20180412001537/manifest.json
https://motog3.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=995691934152
https://preact-pwa-yfxiijbzit.now.sh/manifest-a57e627c89.json
https://prpl-dot-captain-codeman.appspot.com/20170806/es6-unbundled/manifest.json
https://quillie.net/manifest.38100eca.webmanifest
https://reittiopas.foli.fi/icons-turku-6aa88e8a010a06d1d30d24205371f8d3//manifest.json
https://rofr.in/manifest.json?v6=bOO8oaa856
https://schsrch.xyz/resources/0350094f9232803bcc0fd86c3cbd31f1.json
https://sp-web.search.auone.jp/manifest_v2.json
https://ssl.tzoo-img.com/res/favicon/manifest.json?v=2kq2msw2
https://static1-ssl.dmcdn.net/images/neon/favicons/manifest.json.vb58fcfa7628c92052
https://static3.1tv.ru/assets/web/favicon/manifest-1d3e08042839f3a7499da28ea190f0d5.json
https://theomg.github.io/Lifelike/manifest.ee9a11377982a365a8aeae5b9095fe11.json
https://townwork.net/js/manifest?v=20160302001
https://travel.jumia.com/manifest.9fa818c2.json
https://unacademy.com/dist/manifest.json?1487235791853
https://unacademy.com/dist/manifest.json?1505821603929
https://weather.com/weather/assets/manifest.507fcb498f4e29acfeed7596fe002857.json
https://webamp.org/manifest.60fc98cc18ea0b3ab073cda74610efa1.json
https://www.amarujala.com/manifest.json?v=85b484467f
https://www.boldsky.com/browser.json?v=1.0.1
https://www.buzzfeed.com/static-assets/data/manifest.0edfa72a42a9e70e5bf211f64eae9384.json
https://www.colorblindsim.com/manifest.8b7a3d31.webmanifest
https://www.cookscountry.com/_search_assets/cco-manifest-707681872ff6b432492f3fe509aaae89.json
https://www.elo7.com.br/v3/manifest/webapp.json
https://www.freecharge.in/mobile/manifest.json?v=1
https://www.ft.com/assets/manifest/manifest-v6.json
https://www.gp.se/polopoly_fs/3.200.1523348202!/sites/se.gp/images/manifest.json
https://www.iheart.com/manifest.6a2f10c7f194b2a76747f18937e42951.json?rev=7.44.0
https://www.imperialcarsupermarkets.co.uk/manifest.json?v=gAEgYPxJpw
https://www.istitlaa.me/_nuxt/manifest.57352a3d.json
https://www.johnlewis.com/assets/fc539d9/favicons/manifest.json
https://www.koolsol.com/manifest-20170311-01.json
https://www.koolsol.com/manifest-20170526-01.json.php
https://www.liverpoolecho.co.uk/manifest.json?v=548e74556b39b6b25a2b7a4828f7783e
https://www.nouvelobs.com/manifest.json?1510150956
https://www.onthemarket.com/assets/52bbb4af/gzip/js/manifest.json
https://www.onthemarket.com/assets/80f6edfa/gzip/js/manifest.json
https://www.openrent.co.uk/manifest.json?v=9BaGKJ78xe
https://www.otto.de/static/all/img/global-resources/fc44d9d421d3577b/favicons/manifest.json
https://www.otto.nl/3ce8d08884c912ec9b98774bab49a8eff3604010/assets/ottonl/resources/manifest.json
https://www.padpiper.com/manifest.ab5a95547c7ae8833813533907eb0631.json
https://www.pigiame.co.ke/assets/pi-site/favicon/site-ad611bc177.webmanifest
https://www.pitchup.com/manifest.json?v=4
https://www.pricehipster.com/manifest.json?v=1
https://www.reittiopas.fi/icons-hsl-18da13427c6e362f148f4a5b783ee98c//manifest.json
https://www.selcobw.com/skin/frontend/selco/default/assets/manifest.json?6335544
https://www.sho-yamane.me/_nuxt/manifest.7e00d6b4.json
https://www.stylewe.com/manifest.json?v=9255619
https://www.thekitchn.com/assets/tk/favicons/manifest-8afd9804080ba4ee9351cb5adc20383f47f40fe276d62bd25467bdadf5d5c0d6.json
https://www.viz.com/favicon/manifest.json?v=oLLRlE8ljO
https://www.walmart.ca/assets/9d1a7c78e21cc1c3c71ae9f8a8918b0d-home-screen-manifest-en.json
https://www.yiv.com/manifest.json?2017022101

So if Chrome and others switch to using manifest URL to uniquely identify PWAs (and this data is representative of PWAs in general), then around 5% of sites will generate a new A2HS prompt when the manifest URL changes (perhaps only when the content of the manifest changes, but potentially on every deployment).

(Is Chrome using manifest URL right now? I tried changing the manifest URL on a test site and didn't get the A2HS prompt. So I suspect Chrome is currently applying a different heuristic to identify new/updated PWAs.)

ithinkihaveacat on 5 May 2018

Interesting that so many of them are versioned. I wonder where this advice comes from? Could it be that "best practice" with service workers is to version all assets, and the manifest is just being versioned along with that?

So if Chrome and others switch to using manifest URL to uniquely identify PWAs ... then around 5% of sites will generate a new A2HS prompt when the manifest URL changes

I think there's still some confusion here. Chrome already uses the manifest URL to uniquely identify an app. If the manifest URL changes, it's a different app. There are no changes to Chrome that need to be made along these lines (this bug is to document this in the spec, which I think is reasonable).

Is Chrome using manifest URL right now? I tried changing the manifest URL on a test site and didn't get the A2HS prompt. So I suspect Chrome is currently applying a different heuristic to identify new/updated PWAs.

I think it is. If you change the manifest URL you should get a new app. Theories for why you aren't:

When you changed the manifest URL, the new manifest didn't satisfy the PWA checks.
Someone mentioned that Chrome for Android has a limit of 3 apps per origin. It's possible you hit that limit?
I'm mistaken and Chrome for Android is in fact using the start_url or something as the unique key.

mgiuca on 7 May 2018

@mgiuca wrote:

Interesting that so many of them are versioned. I wonder where this advice comes from? Could it be that "best practice" with service workers is to version all assets, and the manifest is just being versioned along with that?

Yeah, @jakearchibald and friends were promoting this a while back as part of SW development (that's not to point fingers - caching is hard, and that approach works well). However, it's still a hack... and like all hacks, it has pros/cons. Additionally, the file hashing may be baked into some developer/command line tools, like webpack - but Jake probably knows more.

marcoscaceres on 7 May 2018

Pretty sure I never recommended versioning manifests specifically. But it's good practice to version assets and treat their URLs as immutable generally. This isn't anything to do with service worker, it's just good caching practice https://jakearchibald.com/2016/caching-best-practices/.

If manifest is an exception to the rule, we need to do some dev rel'ing so folks understand why. The service worker script url is one of these exceptions, and I documented it here https://developers.google.com/web/fundamentals/primers/service-workers/lifecycle#avoid_changing_the_url_of_your_service_worker_script.

jakearchibald on 7 May 2018

@jakearchibald Yeah the manifest URL should have the same policy applied as the SW URL --- it's probably more important since you can migrate to a new SW URL (just takes some fiddling) but it's not generally possible to move to a new manifest URL (without segmenting your installed base).

(Aside: I think we should support updating manifest URL using HTTP 301 Moved Permanently; not sure if this needs to be specced or if we can just implement this.)

mgiuca on 8 May 2018

👍1

This issue has come up again in the context of updating installed PWA manifest data.

When a site has changed its name/scope/theme_color/start_url/manifest URL how do we know we're looking at the same app and not a different one that shares the same scope?
When a PWA installation is synced across devices how do we know the sync has been satisfied when sites may have arbitrary device specific differences in their metadata?

I don't think making an app identified by its manifest is reasonable long term. Sites should be able to re-architect their directory structure/web framework necessitating a change in manifest URL during the lifetime of a user install.

I think we should add an optional "id" field to the manifest that defaults to the manifest URL but can be overridden with whatever the site likes. This ID will be scoped to the start_url's origin and cannot collide with IDs from other origins. This would enable sites to update any aspect of their manifest except their origin and the id.

alancutter on 12 Jul 2019

Please give some examples of these proposed IDs.

On Fri, 12 Jul 2019 at 05:04, alancutter notifications@github.com wrote:

This issue has come up again in the context of updating installed PWA
manifest data.

When a site has changed its
name/scope/theme_color/start_url/manifest URL how do we know we're looking
at the same app and not a different one that shares the same scope?

When a PWA installation is synced across devices how do we know the
sync has been satisfied when sites may have arbitrary device specific
differences in their metadata?

I don't think making an app identified by its manifest is reasonable long
term. Sites should be able to re-architect their directory structure/web
framework necessitating a change in manifest URL during the lifetime of a
user install.

I think we should add an optional "id" field to the manifest that defaults
to the manifest URL but can be overridden with whatever the site likes.
This ID will be scoped to the start_url's origin and cannot collide with
IDs from other origins. This would enable sites to update any aspect of
their manifest except their origin and the id.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/w3c/manifest/issues/586?email_source=notifications&email_token=AAAKU7CBTGKS2MRTV72MHOLP67YDFA5CNFSM4DQH6PJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZYRHFI#issuecomment-510727061,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAKU7CE7U7CS435RIKYIDTP67YDFANCNFSM4DQH6PJQ
.

adewale on 12 Jul 2019

The id field will default to the manifest URL e.g. "https://app.com/manifest.webmanifest" but can be any string e.g. "jdklklfpinionkgpmghaghehojplfjio".
The actual app ID will be a tuple of (start_url origin, manifest id) e.g. ("https://app.com/", ""jdklklfpinionkgpmghaghehojplfjio").

alancutter on 12 Jul 2019

Note that the ID itself will be a totally meaningless (to the web platform) string; it's just an opaque token that uniquely identifies the app within the origin's namespace (so there are no naming conflicts between origins, but you must be careful to uniquely identify your app within your own origin).

We would probably recommend that the ID be a URL relative to the origin, since that would guarantee uniqueness, but we wouldn't derive any meaning from it.

The default of it being the manifest URL would be to preserve the historical fact that the manifest has uniquely identified the app.

mgiuca on 12 Jul 2019

I like @mgiuca's idea (https://github.com/w3c/manifest/issues/586#issuecomment-510761731) of the id just being a meaningless URL resolved against the manifest URL.

marcoscaceres on 2 Apr 2020

the id just being a meaningless URL resolved against the manifest URL

That's not quite what I was suggesting. I was saying it's a meaningless string (doesn't have to be a URL at all). It's an arbitrary character string, that isn't resolved against the manifest URL; it forms part of a unique key, in a pair with the origin (so that two origins with the same id won't collide).

mgiuca on 2 Apr 2020

ah, sorry, I misread. I still like the idea :)

marcoscaceres on 2 Apr 2020

I remember this topic being debated at some length in the sysapps working group in about 2013. My personal opinion has always been that the manifest URI alone should be treated as the identifier of a web application and a different manifest URI should be assumed to be a different application.

Some of the reasons being:

It provides a simple URI as an identifier to use as an index in a database of apps, which also happens to resolve to the metadata describing the app
The manifest URI can be resolved periodically by the user agent to check for updates
No ambiguity over whether two applications within the same origin/sharing the same start_url/sharing the same scope/claiming the same internal ID are the same app or different apps

In implementing the manifest specification recently I found it a real pain trying to use some kind of combination of the origin/start URL/manifest URL/content hash as an identifier and in the end gave up and just used the manifest URL anyway.

I understand why people might want to version the manifest URL and caching is indeed hard, but I would argue there are other solutions to that problem. Cool URI's don't change.

benfrancis on 2 Apr 2020

I appreciate the sentiment that Cool URIs don't change (especially since that page seems to have existed for 22 years at the same URL). But the reality is, developers do want to change their URLs, including the manifest, not just for versioning but to keep their site organised.

The problem as I see it is that we've never _specified_ what makes a unique identifier for an app. So implementations can use the manifest URL, but that's essentially creating a de facto standard that developers have to divine based on the (conflicting) implementations. This isn't just some user-agent-specific logic, it actually affects how developers are allowed to run their sites (i.e., am I allowed to change my manifest URL? The spec doesn't say, I just have to try it and see if it breaks browsers.) So whatever the answer is, it should be specified and consistent across browsers.

I do like the idea of manifest URL being the key, for the reasons you said 1 and 2 (you can just point a store listing or admin install config at a manifest URL and it tells you everything you need to index and install the app).

But it has the significant drawback that developers can never change their manifest URL once the site is launched. We can possibly solve around that by adding an explicit ability to migrate users to a new manifest URL (which could be as simple as stating that a HTTP 301 redirect on the manifest URL says to update to the new location). But it would be simpler if we didn't tie the key to the manifest URL in the first place.

3: No ambiguity over whether two applications within the same origin/sharing the same start_url/sharing the same scope/claiming the same internal ID are the same app or different apps

That is true of any standardized solution. The ambiguity comes from the current reality of it not being specified.

mgiuca on 3 Apr 2020

👍1

I think we should expect to need to add an ID migration mechanism anyway to cover changing origins. Being able to ping the manifest directly is extremely attractive and perhaps having to perform a migration to change your manifest URL is worth it.

alancutter on 3 Apr 2020

Yeah, that's true. Did we have any other reasons (@alancutter) to propose the explicit id scheme, besides being able to migrate your manifest?

I suppose we should consider two separate use cases here:

Once-in-awhile developer wanting to migrate their manifest URL.
Manifest URL is versioned so it changes every time the manifest changes.

Doing an explicit migration is suitable for 1. But I don't think you'd want to do this for 2, otherwise you'd have to make your old manifests 301 to the new one every time. So this would probably preclude being able to version your manifests. Which as @jakearchibald said in 2018, is actually best practice (or would be, if it worked; at present it's best practice for everything _but_ the manifest because of this problem).

mgiuca on 3 Apr 2020

@mgiuca wrote:

whatever the answer is, it should be specified and consistent across browsers.

I agree.

The "Updating the manifest" section of the specification has been empty since 2016 when the same-origin constraint was dropped for manifest URLs and default scope was defined as "unbounded" (later changed) which made things more complicated.

Whatever solution is eventually specified for updates will obviously be influenced by what is used as the unique identifier for an app. Having a relatively stable manifest URI that can be fetched periodically seems like the obvious solution to me. When apps can have overlapping navigation scopes and start URLs can change, something needs to be stable.

Migrating manifest URIs via redirects could work for occasional changes to app structure, but as you suggest it could get unwieldy if the developer tries to change the manifest URL every time the manifest's content is updated for caching purposes. In practice it might be simpler for a developer to just treat a significantly restructured version of the app as a new app, and use other strategies for caching/versioning.

benfrancis on 3 Apr 2020

Note, I expect we will need an identifier mechanism for service workers as well for similar use cases; e.g. migrating from one scope to another. It would be difficult for sites to manage the teardown of one service worker and migration to another without something like this.

Do you plan to make your proposal work for service workers as well?

The strawman I had been thinking of was something lile:

navigator.serviceWorker.register('sw.js', {
  scope: '/some/scope',
  token: 'my-origin-unique-token',
});

So if you call register again with the same token, but a different scope we would migrate the current service worker registration to the new scope.

Edit: Sorry if this was already discussed in this thread. I've only be lightly following until recently.

wanderview on 3 Apr 2020

The strawman I had been thinking of was something lile:
navigator.serviceWorker.register('sw.js', {
  scope: '/some/scope',
  token: 'my-origin-unique-token',
});
A couple of thoughts:

I would find it odd if a _web_ application was identified by anything other than a Uniform Resource Identifier. Apart from being what makes the web the web, URIs make great origin-unique tokens! I would hope there's no need to invent another type of ID namespace like Play Store/App Store style app IDs.
What's the latest thinking on the mapping between a service worker and an app?

It would be really neat if there was a 1:1 mapping between the two and app scope == service worker scope, then you could use the manifest URI as the unique identifier for both and update both navigation scope and service worker scope together in a single update. (There used to be a similar kind of mapping in the manifest, but the other way around.)

But my understanding is that isn't the case and it's currently possible to have multiple service workers per app or multiple apps per service worker, or have one and not the other. If the two technologies are entirely de-coupled then maybe they have to have their own mechanisms for identifying an installed application vs. an installed service worker.

benfrancis on 3 Apr 2020

I'd rather not tie any of this to service workers; it increases the complexity by an order of magnitude. (That's why we ended up removing service worker from the manifest; they are unrelated, and that's by design, as with all the pieces of the web platform, they are separate and composable.)

The way I view it, the service worker is an implementation detail of the application (something the user can't see or interact with at all), while the manifest is the user-facing concept of an application. While generally websites will want to have them at the same scope, you can for instance have a top-level SW scope but with lots of smaller-scoped app manifests. I remember discussions early on in the desktop PWA project on Chrome to have links open in the app if they were within the _service worker_ scope, which I shot down because I don't think service worker scope should have any bearing on the way the user experiences the app. (In much the same way as the user shouldn't care about whether there's a proxy server in between the client and the real backend.)

Under that philosophy, I don't see there being a particular need for a SW-to-SW migration. If you just tear down the old SW and spin up a new one, you can just re-cache everything (or find some mechanism to transfer the cache so it doesn't have to be redownloaded). That's a very different problem to _manifest_ migration, which can't be done in user-space because it involves changing the URL that installed OS-level "apps" are pointing to, and potentially informing the user that the application is changing.

I would find it odd if a web application was identified by anything other than a Uniform Resource Identifier. Apart from being what makes the web the web, URIs make great origin-unique tokens! I would hope there's no need to invent another type of ID namespace like Play Store/App Store style app IDs.

True. I like identifying things with URLs*. I would be OK with saying the "id" is a URL. But when we thought about it, the URL never actually gets resolved, so it would effectively just be an opaque string. If we did want to use URL syntax to express the ID, we shouldn't use the "https" scheme (since that implies it's an actual resolvable resource). We'd have to come up with our own scheme, like "webapp://example.com/user-specified-id". But in the id field, the origin would be implicit (since we can't let you specify the origin of your app's ID, it has to be the origin of your start_url/scope). So we may as well just make the id an opaque string, which if you like, can be formed into a URL like the above, but in practice, you'd never see the URL, and it would be easier to just state that "the unique identity of an application is the pair of (app origin, user-specified-id)", rather than inventing a whole new URL scheme.

*By the way, I'm avoiding use of the term "URI" simply because the URL Standard says that the term "URI" is deprecated in favour of "URL". I personally think it's useful having a distinction, but I fought this years ago, and gave up.

mgiuca on 6 Apr 2020

We'd have to come up with our own scheme, like "webapp://example.com/user-specified-id".

This is similar to what we did with Firefox OS, where we created URLs like webapp://1dd47458-abac-4637-b7e6-12c6e0ef9846. With hindsight I think creating a protocol scheme and namespace separate from the web was the biggest single technical mistake we made on the project, because over time it allowed applications to evolve into something which was missing many of the key benefits of the web, especially linkability. This is one of the key principles of "Progressive Web Apps".

*By the way, I'm avoiding use of the term "URI" simply because the URL Standard says that the term "URI" is deprecated in favour of "URL". I personally think it's useful having a distinction, but I fought this years ago, and gave up.

Yeah I was just using the terms to distinguish between a URI which _identifies_ an app and a URL which can be resolved to _locate_ its metadata, but what I'm advocating for is a manifest URL which serves both functions.

My understanding of the proposed use cases for an ID in this thread are:

Uniquely identify a web app in a directory or app store
Uniquely identify a web app when the metadata in its manifest is updated
Sync installed web apps across devices, even if the server serves slightly different manifest metadata to different user agents

With the additional requirements:

An ID which is guaranteed to be unique within its origin
Ideally have backwards compatibility with user agents which have used manifest URL as an ID in the past
Allow developers to change the URL of a web app manifest when needed and migrate to that new URL
Enable caching a manifest and invalidating the cache of a manifest

Using the manifest URL to identify the app seems to me to fulfill all of these requirements. It can be used as a globally unique identifier (which is therefore also unique within its origin), can identify an app even when the contents of the manifest is updated or differs between user agents, allows cache control using cache headers and can be migrated to a new URL if necessary using HTTP redirects.

It also has the benefit that it doesn't require inventing a new URL scheme for a new non-web namespace for installed web applications and doesn't require an algorithm to derive the identity of an app from multiple inputs. And finally, it has the benefit of providing a potential simple update mechanism which the manifest specification still doesn't have a solution for.

benfrancis on 6 Apr 2020

So if you call register again with the same token, but a different scope we would migrate the current service worker registration to the new scope.

What state is persistant that needs migrating for service workers? I was under the impression caches were origin scoped. I'm not super familiar with service workers so genuinely asking.

alancutter on 7 Apr 2020

What state is persistant that needs migrating for service workers? I was under the impression caches were origin scoped. I'm not super familiar with service workers so genuinely asking.

The version of the service worker that is active is part of the state of the app. The service worker lifecycle is designed to support keeping other storages in sync with your script state. But if you have separately identified service workers you lose the ability to keep them in sync. You now have to deal with possibly two service workers being in flight in various states at once. Its possible to deal with, but complex.

wanderview on 7 Apr 2020

Anyway, I'll file a separate issue for the service worker issue.

wanderview on 7 Apr 2020

We are going to defer on the ID for now. We will pick this up again after CR.

marcoscaceres on 25 May 2020

I see the following issues with using manifest url:
1) This locks developers into a CDN / the manifest url host. I can see a world where someone scraping manifests would see a ton of duplicate webapps here because they find links to lots of different manifest urls for the same app (versioned, or on different CDNs). There would all be separate apps
2) This prevents versioning of the manifest in the name
3) This makes it difficult to serve different manifests based on client hints like language, etc.

I think not having a developer-facing / obvious ID system also has some problems:
1) Sites accidentally break webapp updating by changing start_url or manifest_url w/o knowing that none of their old users won't get updated. This segments their users and there is no way to fix it.
2) Without a standard here different user agents do different things (chrome uses start_url, Android uses manifest_url, Firefox uses manifest_url?)
3) Hard to have two different webapps on the same origin that works for multiple user agents

We have already seen major sites break their webapp by changing the start_url or manifest_url without knowing that this breaks things.

So in general, I think this is really important to fix. I think manifest_url as the key is "ok", but I'd much rather allow the manifest_url to change for a webapp to avoid the issues up top (do you see any other issues?)

I'm thinking maybe we can use a unique-per-origin ID that a manifest can set. It can be a string of anything, and the default value could match either start_url or manifest url to prevent breaking on one system or another (someone is going to break).

WDYT?

dmurph on 31 Jul 2020

Yes that's what we've been thinking of.

unique-per-origin ID that a manifest can set

I think the best way (i.e., most consistent with how other manifest keys like this work) to express this is that the ID is a URL that must be same-origin as scope*. This URL is resolved against the manifest URL, like every other URL in the spec. This URL is never requested, it's just used as an identifier.

Making it a URL like this and requiring same origin gives it a natural uniqueness per origin.

This means if your manifest is on a different origin to your start/scope, you must specify the ID as an absolute URL (same as all the other URLs). And it means if your id is path-relative, moving your manifest would change the ID, so you have to be careful to keep the ID stable when moving the manifest. We would recommend always specifying the ID as path-absolute (e.g., "/my-id").

Another approach is to make it relative to the origin of the scope, so you can just specify the ID as a string and it will do as expected. That would create less confusion for developers, at a cost of being different to how all the other URLs resolve. That's a trade-off we could make.

* Note: I said "same-origin as scope" here, not "within scope" which is more typical. That's to prevent sites from accidentally being locked in and unable to change their scope. For example, if ID was "/foo" and scope was "/", they would be unable to change scope to "/bar" without breaking the ID. Since there is no technical reason for the ID to be within scope (since we never actually navigate to that URL), it can just be same origin.

could match either start_url or manifest url to prevent breaking on one system or another (someone is going to break).

Ideally, the default would be scope. Scope is the best identifier of an app at the moment; it's the least likely to change over time and it's usually a containing URL for both start and manifest. That would break both desktop and mobile Chrome's current representation, but since we're going to cause breakage, maybe we should just do it globally?

If not scope, I don't have a strong opinion about whether start_url or manifest URL is the best default.

mgiuca on 3 Aug 2020

I suspect manifest URL is less likely to change over time than start_url, should we need to fall back to one of those two.

dominickng on 3 Aug 2020

If there is no manifest should that be some kind of "null" ID within the origin?

alancutter on 3 Aug 2020

If the ID for a web app is a URL inside a manifest and the manifest URL itself can change:
1 How would updates be handled? (Especially if in future web apps can be installed independently of a document.)

What happens if two web app manifests provide the same ID?

benfrancis on 3 Aug 2020

The user agent can fetch the start URL to retrieve the latest manifest data.
If two web apps provide the same ID (on the same origin) then they are the same web app.

alancutter on 4 Aug 2020

I do think that there is one big downside for using manifest_url as the ID - this means that a manifest wouldn't be inherently 'packaged' by itself. Like - you couldn't install an app just from a manifest without that manifest url (or the id being specified).

If everyone specifies an ID, this I guess isn't that big of a deal - but it is difficult to fake for the 'webapp'ing that current browsers do. Right now, you can create a fake manifest for a site and just set the start_url to the url that is being shown, and bam, webapp. But if manifest_url becomes the unique ID, and systems are designed around that, then that becomes more complicated.

for the questions above:

I'm thinking updates would happen when the browser encountered a manifest, where the ID of the manifest matches the id of an existing manifest of that origin. The traditional way browser encounter the manifest is to see it when a page is loaded - but I could imagine that it could be provided another way.
As Alan said, then they would be the same webapp. I think Chrome's behavior is to apply the last-seen manifest as the updated manifest.

dmurph on 5 Aug 2020

@dmurph wrote:

I do think that there is one big downside for using manifest_url as the ID - this means that a manifest wouldn't be inherently 'packaged' by itself. Like - you couldn't install an app just from a manifest without that manifest url (or the id being specified).

I personally think that's a good thing because trust in the origin a manifest was retrieved from is surely an important factor in the implict permissions a user grants by installing a web application? It also makes the app more linkable and discoverable if the ID actually dereferences to something.

it is difficult to fake for the 'webapp'ing that current browsers do. Right now, you can create a fake manifest for a site and just set the start_url to the url that is being shown, and bam, webapp. But if manifest_url becomes the unique ID, and systems are designed around that, then that becomes more complicated.

For a hack like a fake manifest, which won't be following the specification anyway, could browsers generate a special cased local URL like chrome://apps/myfakeapp.webmanifest ?

they [two manifests providing the same ID] would be the same webapp

That would presumably make https://foo.github.io/repo1/app1.webmanifest and https://foo.github.io/repo2/app2.webmanifest (or https://google.com/calendar/app.webmanifest and https://google.com/mail/app.webmanifest) the same app, if they provided the same ID.

That arguably isn't a huge issue as the origin is ultimately the trust boundary, but it could be a bit of a footgun.

benfrancis on 6 Aug 2020

@dmurph wrote:

I do think that there is one big downside for using manifest_url as the ID - this means that a manifest wouldn't be inherently 'packaged' by itself. Like - you couldn't install an app just from a manifest without that manifest url (or the id being specified).

I personally think that's a good thing because trust in the origin a manifest was retrieved from is surely an important factor in the implict permissions a user grants by installing a web application? It also makes the app more linkable and discoverable if the ID actually dereferences to something.

I guess I don't see the host of the manifest being the "trusted" origin, I see it being the origin of the start_url / the implied scope. Technically someone right now can host a manifest (B.com) that lists A.com as the start url, and that just works. Arguably maybe not a great thing to be valid, but that's a separate discussion I think.

it is difficult to fake for the 'webapp'ing that current browsers do. Right now, you can create a fake manifest for a site and just set the start_url to the url that is being shown, and bam, webapp. But if manifest_url becomes the unique ID, and systems are designed around that, then that becomes more complicated.

For a hack like a fake manifest, which won't be following the specification anyway, could browsers generate a special cased local URL like chrome://apps/myfakeapp.webmanifest ?

yeah that might work

they [two manifests providing the same ID] would be the same webapp

That would presumably make https://foo.github.io/repo1/app1.webmanifest and https://foo.github.io/repo2/app2.webmanifest (or https://google.com/calendar/app.webmanifest and https://google.com/mail/app.webmanifest) the same app, if they provided the same ID.

That arguably isn't a huge issue as the origin is ultimately the trust boundary, but it could be a bit of a footgun.

Yeah, but fixable by the developers at least. I think that is a much easier problem to avoid than the current situation, where they can unknowingly segment their users w/o obvious problems initially

dmurph on 6 Aug 2020

I'm pretty late to the party, but I have a few thoughts.

Overall, iiuc, the main motivation for having an explicit ID instead of using Manifest URL is to allow the Manifest URL to change (including for versioning). It's been mentioned that the migration path for the app to find a new Manifest URL will be to load the Start URL and find the Manifest URL from the \ So my question is, what if a developer changes both URLs? For example, if they migrate their whole app to a new domain or new subdirectory? Haven't we effectively just changed the problem to "yes, we can now move the Manifest URL, but _only_ if it's guaranteed that the original Start URL remains up-to-date and accessible forever"?
And to emphasize, this means the original Start URL. It's possible that existing apps that have already changed their Start URLs still have straggler users who haven't updated to the new Start URL yet. If they now move their manifest, will the straggler users be able to find the new manifest?
Speaking of moving an app to a new domain, is that handled by this proposal? There's a lot of talk of unique-per-origin IDs. Does that mean moving to a new origin still qualifies as "making a whole new separate app"?
I'm wondering about the migration strategy from the current world. If we say that an empty ID will default to the Manifest URL, then all current apps already have a default ID assigned. How do they migrate from this default ID to their new desired explicit ID without losing all their current users?
It seems to me that there's almost zero benefit of an app specifying a relative ID. Possibly even a negative benefit, since some devs might think specifying an ID is protecting them and allowing them to move their Manifest URL, and they will end up accidentally breaking their app when the ID changes after a manifest move. What's the point of allowing a relative ID? Why not make absolute IDs mandatory?

@dmurph wrote:

I see the following issues with using manifest url:

This locks developers into a CDN / the manifest url host. I can see a world where someone scraping manifests would see a ton of duplicate webapps here because they find links to lots of different manifest urls for the same app (versioned, or on different CDNs). There would all be separate apps

This prevents versioning of the manifest in the name

This makes it difficult to serve different manifests based on client hints like language, etc.

Could you elaborate on (3)? Do you mean that it would be better to have a set of manifests, one per language, for example, and they would all have the same ID (making them all parts of the same app), and then the start_url page would dynamically choose which to point to in the \

Also, it seems potentially weird to have multiple manifests all be part of the same app. While there are legitimate use-cases (like translation), it opens up a potentially very confusing situation where completely different things will all be considered the "same" app (they could customize anything in the manifest - start url, name, color schemes, even scope, and we'd still consider it "the same app"). Yes, technically this is already possible by hosting a dynamic manifest, but this makes it a more explicitly "allowed" strategy, and one that could even be done accidentally.

glennhartmann on 6 Aug 2020

👍1

@glennhartmann wrote:

I'm pretty late to the party, but I have a few thoughts.

Overall, iiuc, the main motivation for having an explicit ID instead of using Manifest URL is to allow the Manifest URL to change (including for versioning). It's been mentioned that the migration path for the app to find a new Manifest URL will be to load the Start URL and find the Manifest URL from the tag.
So my question is, what if a developer changes both URLs? For example, if they migrate their whole app to a new domain or new subdirectory? Haven't we effectively just changed the problem to "yes, we can now move the Manifest URL, but _only_ if it's guaranteed that the original Start URL remains up-to-date and accessible forever"?
And to emphasize, this means the original Start URL. It's possible that existing apps that have already changed their Start URLs still have straggler users who haven't updated to the new Start URL yet. If they now move their manifest, will the straggler users be able to find the new manifest?

I think the desire is to allow developers to change the start url & the manifest url. They would have to deal with the case that an old start url is still registered for various users forever, so they would have to handle that somehow. I'm not sure what the best route would be here - I'm guessing they have to somehow serve the new manifest on the old start url, OR they can redirect & serve the new manifest on the new start_url, and since the IDs will match up, then it can update.

Speaking of moving an app to a new domain, is that handled by this proposal? There's a lot of talk of unique-per-origin IDs. Does that mean moving to a new origin still qualifies as "making a whole new separate app"?

2) cross-origin migration is a non-goal here. But if we make the IDs unique (instead of semi-unique), then that might be easier? I'm not trying to tackle this problem right now, we dont' have requests for it, we only have people right now trying to create multiple PWAs in the same domain & struggling, or people updating PWAs & having new start_urls, which accidentally segments their userbase.

I'm wondering about the migration strategy from the current world. If we say that an empty ID will default to the Manifest URL, then all current apps already have a default ID assigned. How do they migrate from this default ID to their new desired explicit ID without losing all their current users?

They would have to use their old manifest url as their ID, forever. Or whatever we have as default. I guess we could have some custom spec language here around "if you didn't have an ID and you set one, then that is the ID, as long as the manifest url matches"?, but that might be complicated. Open to thinking about that though.

It seems to me that there's almost zero benefit of an app specifying a relative ID. Possibly even a negative benefit, since some devs might think specifying an ID is protecting them and allowing them to move their Manifest URL, and they will end up accidentally breaking their app when the ID changes after a manifest move. What's the point of allowing a relative ID? Why not make absolute IDs mandatory?

Sure, I don't mind them being absolute / globally unique. It seems weird though as basically prepending the origin to the id would basically make it unique, so we could just do that for them, and say it only has to be unique for the origin. Question - how would the ID change after the manifest move? The ID must stay the same to move the manifest & not break people. Not sure why absolute is necessary here.

@dmurph wrote:

I see the following issues with using manifest url:

This locks developers into a CDN / the manifest url host. I can see a world where someone scraping manifests would see a ton of duplicate webapps here because they find links to lots of different manifest urls for the same app (versioned, or on different CDNs). There would all be separate apps

This prevents versioning of the manifest in the name

This makes it difficult to serve different manifests based on client hints like language, etc.

Could you elaborate on (3)? Do you mean that it would be better to have a set of manifests, one per language, for example, and they would all have the same ID (making them all parts of the same app), and then the start_url page would dynamically choose which to point to in the tag based on client hints? Is that much more difficult than serving dynamic manifest contents from the same URL based on client hints?

Also, it seems potentially weird to have multiple manifests all be part of the same app. While there are legitimate use-cases (like translation), it opens up a potentially very confusing situation where completely different things will all be considered the "same" app (they could customize anything in the manifest - start url, name, color schemes, even scope, and we'd still consider it "the same app"). Yes, technically this is already possible by hosting a dynamic manifest, but this makes it a more explicitly "allowed" strategy, and one that could even be done accidentally.

Regarding my 3) - I think you're right, and I don't like that use case anymore. I'm a bigger fan of making the manifest multi-lingual, I think there are proposals here. I don't there there should be multiple manifests, just one per app.

dmurph on 6 Aug 2020

They would have to use their old manifest url as their ID, forever. Or whatever we have as default. I guess we could have some custom spec language here around "if you didn't have an ID and you set one, then that is the ID, as long as the manifest url matches"?, but that might be complicated. Open to thinking about that though.

Yes. The point of having a hard-specified default is that site authors know exactly what their site used to default to, so if they want to change the thing that the default is based off (e.g., the manifest URL), they can set the ID to exactly the string that used to be the default, to avoid their ID changing.

We can put a non-normative note about this, but we don't need any normative text around the temporal changes to a manifest file.

Also, it seems potentially weird to have multiple manifests all be part of the same app.

Well. "Multiple manifests" is a bit hard to define (if a manifest changes its content or its URL, is that "multiple manifests"?). Essentially the entire point of this ID _is_ to formally identify when a manifest change represents a new app, versus a mutation of an existing app.

Using a different manifest URL _is_ the currently recommended and only viable way to provide localized manifest metadata. So we have to support that, unless we want to block on #676 (properly supporting localization). I think this works fine: you would make all of your different-locale manifests have the same ID, so they all represent the same app. Whichever manifest was served at install time determines what language you see. That way, if the user changes their language, and the start URL starts pointing at another language's manifest URL, the browser's updater will go "aha, I'll update to a new version of the manifest" as opposed to "that app is not installed". (This is _exactly_ the point of having an ID, so we can distinguish those cases.)

Going back to what @benfrancis said, the same answer applies:

What happens if two web app manifests provide the same ID?

The whole point of the ID is so that we know when "two web app manifests" represent two different apps versus two different versions of the same app.

mgiuca on 7 Aug 2020

@mgiuca wrote:

Using a different manifest URL is the currently recommended and only viable way to provide localized manifest metadata. So we have to support that, unless we want to block on #676 (properly supporting localization).

That's not strictly correct. The specification also mentions that servers can use the "Accept-Language" header to provide the user with a manifest in their preferred language. It's perfectly valid for the same resource at the same URL to have different representations as a result of content negotiation like this, it doesn't make it a separate resource. An HTTP URL identifies the resource, not its representation.

Either way, I don't see this as a problem. If the user installs the French version of an app, they presumably want to continue using the French version of the app when the manifest is updated, either based on the default language preference set in the user agent or by manual selection via a query string. (I would argue this is "properly supporting localization" and as I understand it was an intentional design decision, but I will read and comment on the other issue about that.)

Edit: I see you wrote a whole document on this topic, so maybe you just forgot ;)

Well. "Multiple manifests" is a bit hard to define (if a manifest changes its content or its URL, is that "multiple manifests"?).

As above, if the URL changes then yes. If the content changes but the URL remains the same then no. This works in either localisation case because either you're explicitly requesting a resource which is a manifest for a French app, or you're requesting a multi-lingual resource and asking for the French representation of it.

The whole point of the ID is so that we know when "two web app manifests" represent two different apps versus two different versions of the same app.

I would argue the manifest URL can already provide this. The only compelling reason I've heard so far for adding an additional identifier URL is where people are using version URLs for manifests in a CDN. I have to be honest that this isn't a problem I've ever come across, but I know this is an approach that Google recommends so maybe it's more common and a harder problem than I realise.

Personally as a web developer I'm used to the idea that a URI identifies a resource and it has always made sense to me that a web app be identified by its manifest URL. If a resource is superceded by a new resource (as opposed to a new version of the old one) I would have thought the normal practice is just to redirect to it. As a user agent implementer I would far rather just fetch the manifest URL to check for updates than have to look for the start URL, fetch the start URL, parse the HTML, look for the manifest link and then fetch the manifest. Especially if the start URL can change too.

@dmurph wrote:

we only have people right now trying to create multiple PWAs in the same domain & struggling, or people updating PWAs & having new start_urls, which accidentally segments their userbase.

IIUC then using the manifest URL alone as an identifier rather than some combination of manifest URL, start URL, scope and content then the problem of accidentally segmenting users by changing the start_url would go away? Multiple PWAs per domain shouldn't be a problem either. Or am I missing something?

benfrancis on 7 Aug 2020

@dmurph wrote:

I think the desire is to allow developers to change the start url & the manifest url. They would have to deal with the case that an old start url is still registered for various users forever, so they would have to handle that somehow. I'm not sure what the best route would be here - I'm guessing they have to somehow serve the new manifest on the old start url, OR they can redirect & serve the new manifest on the new start_url, and since the IDs will match up, then it can update.

Right, I guess mainly what I'm wondering is whether this is that much of an improvement over using Manifest URL. If we say that manifest url is the ID, then start url is already trivially updatable, like any other manifest attribute. Moving the manifest is also doable via HTTP 301 redirect. The main drawback afaict is that it requires an explicit action by the developer, and continued control or maintenance over the original manifest url.

It seems to me that the new proposal (unless we come up with a better migration process) has similar explicit action and maintenance required. We're just changing the problem for developers from "we need to redirect the old manifest URL" to "we need to make sure all previous start URLs continue serving content and point to the current manifest URL". Either way moving the start URL and manifest is doable, but requires explicit thought and work to get it right.

To be clear, I'm not against the idea of an explicit ID, I just want to make sure it's buying us as much of a benefit as we think it is, and enough to justify the cost of implementation.

There are a few benefits I can think of, but I'm not sure how big they are:

changing static-hosted contents may be easier than issuing an HTTP 301 redirect in some cases
getting the migration wrong with an explicit ID is possibly more fixable than getting it wrong with a manifest URL

@dmurph wrote:

cross-origin migration is a non-goal here. But if we make the IDs unique (instead of semi-unique), then that might be easier? I'm not trying to tackle this problem right now, we dont' have requests for it, we only have people right now trying to create multiple PWAs in the same domain & struggling, or people updating PWAs & having new start_urls, which accidentally segments their userbase.

Ok, gotcha.

@dmurph wrote:

I'm wondering about the migration strategy from the current world. If we say that an empty ID will default to the Manifest URL, then all current apps already have a default ID assigned. How do they migrate from this default ID to their new desired explicit ID without losing all their current users?

They would have to use their old manifest url as their ID, forever. Or whatever we have as default. I guess we could have some custom spec language here around "if you didn't have an ID and you set one, then that is the ID, as long as the manifest url matches"?, but that might be complicated. Open to thinking about that though.

Makes sense, thanks.

@dmurph wrote:

It seems to me that there's almost zero benefit of an app specifying a relative ID. Possibly even a negative benefit, since some devs might think specifying an ID is protecting them and allowing them to move their Manifest URL, and they will end up accidentally breaking their app when the ID changes after a manifest move. What's the point of allowing a relative ID? Why not make absolute IDs mandatory?

Sure, I don't mind them being absolute / globally unique. It seems weird though as basically prepending the origin to the id would basically make it unique, so we could just do that for them, and say it only has to be unique for the origin. Question - how would the ID change after the manifest move? The ID must stay the same to move the manifest & not break people. Not sure why absolute is necessary here.

Sorry, I think I was responding specifically to this:

@mgiuca wrote:

I think the best way (i.e., most consistent with how other manifest keys like this work) to express this is that the ID is a URL that must be same-origin as scope*. This URL is resolved against the manifest URL, like every other URL in the spec. This URL is never requested, it's just used as an identifier.

Making it a URL like this and requiring same origin gives it a natural uniqueness per origin.

This means if your manifest is on a different origin to your start/scope, you must specify the ID as an absolute URL (same as all the other URLs). And it means if your id is path-relative, moving your manifest would change the ID, so you have to be careful to keep the ID stable when moving the manifest. We would recommend always specifying the ID as path-absolute (e.g., "/my-id").

Which I guess we haven't actually landed on.

@mgiuca wrote:

Also, it seems potentially weird to have multiple manifests all be part of the same app.

Well. "Multiple manifests" is a bit hard to define (if a manifest changes its content or its URL, is that "multiple manifests"?). Essentially the entire point of this ID _is_ to formally identify when a manifest change represents a new app, versus a mutation of an existing app.

Using a different manifest URL _is_ the currently recommended and only viable way to provide localized manifest metadata. So we have to support that, unless we want to block on #676 (properly supporting localization). I think this works fine: you would make all of your different-locale manifests have the same ID, so they all represent the same app. Whichever manifest was served at install time determines what language you see. That way, if the user changes their language, and the start URL starts pointing at another language's manifest URL, the browser's updater will go "aha, I'll update to a new version of the manifest" as opposed to "that app is not installed". (This is _exactly_ the point of having an ID, so we can distinguish those cases.)

Going back to what @benfrancis said, the same answer applies:

What happens if two web app manifests provide the same ID?

The whole point of the ID is so that we know when "two web app manifests" represent two different apps versus two different versions of the same app.

Right, this all seems reasonable. My main concern was developers accidentally (via copy/paste error, probably) specifying the same ID for multiple different apps, which would result in a pretty weird state. I'm probably overthinking this point, though, as it seems likely to be an infrequent case.

@benfrancis wrote:

Using a different manifest URL is the currently recommended and only viable way to provide localized manifest metadata. So we have to support that, unless we want to block on #676 (properly supporting localization).

That's not strictly correct. The specification also mentions that servers can use the "Accept-Language" header to provide the user with a manifest in their preferred language. It's perfectly valid for the same resource at the same URL to have different representations as a result of content negotiation like this, it doesn't make it a separate resource. An HTTP URL identifies the resource, not its representation.

Either way, I don't see this as a problem. If the user installs the French version of an app, they presumably want to continue using the French version of the app when the manifest is updated, either based on the default language preference set in the user agent or by manual selection via a query string. (I would argue this is "properly supporting localization" and as I understand it was an intentional design decision, but I will read and comment on the other issue about that.)

I tend to agree with "If the user installs the French version of an app, they presumably want to continue using the French version of the app when the manifest is updated". But it's also true that if the manifest being served depends on the user's IP address, for example, then while travelling, they could end up getting install prompts for a different language of an app they already have installed, because the user agent has no way of knowing that these are just two different localizations of "the same app".

glennhartmann on 7 Aug 2020

👍1

@glennhartmann wrote:

@dmurph wrote:

I think the desire is to allow developers to change the start url & the manifest url. They would have to deal with the case that an old start url is still registered for various users forever, so they would have to handle that somehow. I'm not sure what the best route would be here - I'm guessing they have to somehow serve the new manifest on the old start url, OR they can redirect & serve the new manifest on the new start_url, and since the IDs will match up, then it can update.

Right, I guess mainly what I'm wondering is whether this is that much of an improvement over using Manifest URL. If we say that manifest url is the ID, then start url is already trivially updatable, like any other manifest attribute. Moving the manifest is also doable via HTTP 301 redirect. The main drawback afaict is that it requires an explicit action by the developer, and continued control or maintenance over the original manifest url.

It seems to me that the new proposal (unless we come up with a better migration process) has similar explicit action and maintenance required. We're just changing the problem for developers from "we need to redirect the old manifest URL" to "we need to make sure all previous start URLs continue serving content and point to the current manifest URL". Either way moving the start URL and manifest is doable, but requires explicit thought and work to get it right.

To be clear, I'm not against the idea of an explicit ID, I just want to make sure it's buying us as much of a benefit as we think it is, and enough to justify the cost of implementation.

There are a few benefits I can think of, but I'm not sure how big they are:

changing static-hosted contents may be easier than issuing an HTTP 301 redirect in some cases

getting the migration wrong with an explicit ID is possibly more fixable than getting it wrong with a manifest URL

I guess, in the case where a website accidentally assumes they can just reference a different manifest link for an update, and that doesn't obviously break anything at first, is still an issue. We have already had partners make this mistake with start_url on desktop and they can't fix it (half of their population installed after the change, half before the change. So they have two separate apps now they have to maintain and people get confused because they can install both, etc).

Maybe we position the id as a way to fix this type of situation. original_manifest_url, or manifest_url_override, or maybe just id is fine. But it's basically a way for a website to fix this situation if it happens, instead of encouraging all people to use an id? Then your 'copy-paste' mistake case won't be as much of an issue?

My preference would be to:

Add an id field that uniquely identifies the web app in the origin
default this value to the relative manifest url (edge case here of the manifest being served in a different location than the start url, Matt has a scoping issue about this somewhere that probably solves it),
encourage webapp authors to set it, and clarify that this uniquely identifies the webapp as a single webapp, and any manifest in that origin that has that id will be considered for the same webapp

BUT I'm also fine with framing it as a "fixing" field, and not encouraging developers to use it unless they have broken their population by moving their manifest hosting location.

WDTY?

dmurph on 14 Aug 2020

@dmurph wrote:

I guess, in the case where a website accidentally assumes they can just reference a different manifest link for an update, and that doesn't obviously break anything at first, is still an issue. We have already had partners make this mistake with start_url on desktop and they can't fix it (half of their population installed after the change, half before the change. So they have two separate apps now they have to maintain and people get confused because they can install both, etc).

I can see why the segmentation problem exists when the identifier of an app is a tuple of manifest URL + start_url + something else (this is not something a developer would intuitively expect). But if the identifier was the manifest URL alone then is there any evidence to suggest that developers _do_ assume different manifest URLs will be treated as the same app by user agents?

The reason that I ask is that the cost of fixing this (potentially hypothetical) problem by adding an additional identifier is that updates could become a lot more complex. It's much simpler to fetch a fixed manifest URL to look for updates than to have to figure out what document belongs to the app (bearing in mind start_url and scope can change), parse the manifest link relation from that document's HTML and then fetch the manifest.

I suggest the underlying problem here is just that the specification currently doesn't specify what the identifier of an app is or how apps are updated so developers are left to guess what user agents use to identify their app and each user agent may do something different.

benfrancis on 14 Aug 2020

@benfrancis wrote

The reason that I ask is that the cost of fixing this (potentially hypothetical) problem by adding an additional identifier is that updates could become a lot more complex. It's much simpler to fetch a fixed manifest URL to look for updates than to have to figure out what document belongs to the app (bearing in mind start_url and scope can change), parse the manifest link relation from that document's HTML and then fetch the manifest

The algorithm that we use is to basically visit any manifest links we see and then update if the identifier matches (for chrome desktop this is start_url). This code path used for install detection as well (and for setting theme color, etc, manifests are always supposed to be visited) , so it's not too much of added complexity for us to check all manifests we see because we do that anyways. Not sure what android does.

@benfrancis wrote

I suggest the underlying problem here is just that the specification currently doesn't specify what the identifier of an app is or how apps are updated so developers are left to guess what user agents use to identify their app and each user agent may do something different.

I think this is exactly right. Declaring the manifest_url to be the authoritative 'ID' would also work, as long as all user agents behave as such (i.e. we can get that into the spec). Would you be OK with the conclusion here to use 'manifest_url' as the unique identifier for a web app?

dmurph on 14 Aug 2020

@dmurph wrote:

The algorithm that we use is to basically visit any manifest links we see and then update if the identifier matches (for chrome desktop this is start_url). This code path used for install detection as well (and for setting theme color, etc, manifests are always supposed to be visited) , so it's not too much of added complexity for us to check all manifests we see because we do that anyways. Not sure what android does.

Ah I see. Yes unfortunately I don't think start_url makes the best identifier because it's the most likely to change. I tried using scope (+ origin) in the past for Firefox OS, which is a bit better but still has problems.

I'm not sure what Fenix currently does? Do manifests on Android currently ever get updated once they are installed?

I currently have a slightly different use case where I'd ideally like to be able to install a single web app to a kiosk runtime remotely by its manifest URL, without it being installed from browser chrome on the device itself. I'm hoping that https://github.com/w3c/manifest/pull/834 might make that possible by removing the dependency on a document URL. In this case manifest link relations may never actually be followed, which makes the approach you describe above for updates a bit tricky. I recognise this is not the core intended use case for a manifest, but it might also be relevant to installing a web app from an app store.

Would you be OK with the conclusion here to use 'manifest_url' as the unique identifier for a web app?

Yes, that's what I'm suggesting.

benfrancis on 14 Aug 2020

We've been using the manifest-url as a unique identifier for Gulliver for a few years now. We had more issues with developers versioning the Manifest URL early on, but this has significantly improved since then. There are still issues with duplication in some cases though, but those mostly come from Gulliver allowing non-installable apps being added (eg: Manifests hosted via http).

andreban on 16 Aug 2020

I'm okay with standardising on origin + manifest_url being the identifier because we can always transition later to an explicit id field that defaults to manifest_url should we find that devs need to move their manifest around.

alancutter on 17 Aug 2020

👍4

I think Chrome desktop is going to be the main pain point here, since that's the one that doesn't use manifest URL. If @dmurph and @alancutter are happy with migrating Chrome desktop over to manifest, I think we'll all be in a much better place (and yes, we can live with that and add id field later if there are still issues).

FWIW, the issue with just having manifest as the ID and not an explicit id field is that it restricts what people are able to do with their site layout. Even if developers are no longer having *this* issue due to versioning their manifest URL, there is a reason developers like to version assets (for caching), so forcing them to not version their manifest URL could be causing other problems that we don't have much visibility into. That is why I originally proposed the id field.

mgiuca on 17 Aug 2020

👍1

Does anyone here have experience with customers who want to do experimenting with manifest attributes (like, if a site wants to have 5% of the population have 'minimal-ui' instead of 'standalone')? This is something where I could see the manifest_url constraint being a hinderance, as the website now has to dynamically change their manifest file server-side based on probably cookies in the request header. Not impossible, but very constraining - I could imagine this being a primary use-case for introducing the id-style field.

dmurph on 18 Aug 2020

I know of Construct 3 from an old bug. Their app URLs appear to be versioned.
https://editor.construct.net/ has manifest URL https://editor.construct.net/r210-2/appmanifest.json. Back when that bug was active their manifest URL was https://editor.construct.net/r131/appmanifest.json.

Edit: Google Keep is another example of a versioned manifest: https://ssl.gstatic.com/keep/manifest/v2.webmanifest

alancutter on 18 Aug 2020

I'm also ok with manifest_url. I'm fine with taking on the work to migrate to manifest url in chrome desktop. I don't think this is on the roadmap until at least q4.

dmurph on 17 Sep 2020

During the f2f/TPAC, @dmurph agreed to provide us a summary of where we are at an our options here.

marcoscaceres on 26 Oct 2020

👍1

I don’t think this was addressed above—forgive me if it was—but what is to stop a bad actor from spoofing the id of another app? Do you think there should be an additional check in place? Maybe something like _origin_ + id? It would complicate things like domain migrations, but I feel like we could probably come up with a best practice for that as well.

aarongustafson on 1 Dec 2020

The id is intended to be only unique when combined with the domain - so the unique ID is technically origin+id. So a bad actor wouldn't be able to steal an app unless they can host a manifest on the same origin, which I think the group has considered an appropriate security boundary (and I believe is used for other specs).

If we were to talk about origin migration (which is out of scope of this discussion), I'm assuming, as a bare minimum, we would need 2 way authentication (old manifest points to new one, new one points to old one).

dmurph on 1 Dec 2020

The id is intended to be only unique when combined with the domain - so the unique ID is technically origin+id. So a bad actor wouldn't be able to steal an app unless they can host a manifest on the same origin, which I think the group has considered an appropriate security boundary (and I believe is used for other specs).

Perfect. Thanks for the clarification. Like I mentioned, I didn’t see this particular abuse vector addressed above, but it’s a long thread.

If we were to talk about origin migration (which is out of scope of this discussion), I'm assuming, as a bare minimum, we would need 2 way authentication (old manifest points to new one, new one points to old one).

Agreed. I’ll start a separate issue to discuss best practices for domain migration.

aarongustafson on 1 Dec 2020

I just published a Ratings & Reviews Prompt Explainer which makes use of the unique ID (implicit or explicit). Feedback encouraged.

aarongustafson on 3 Dec 2020

During the f2f/TPAC, @dmurph agreed to provide us a summary of where we are at an our options here.

Is there a written record of this summary, and whether a decision was made? (Sorry I wasn't able to attend TPAC due to being on parental leave).

benfrancis on 3 Dec 2020

https://www.w3.org/2020/10/26-webapps-minutes.html

dmurph on 3 Dec 2020

👍1

+1 for allowing for the modification of the start_url, and the ability to set an identifier.

As developers of large apps, we have two use cases this would help solve:
1) We would like to be able to change our start_url, so we can add a tracking parameter. This also helps with a potential future migration of one of our app's start page.
2) We also have apps that don't have a stable manifest url, where an explicit identifier field would be needed. Some of our apps use a versioning model for the manifest url. We are also considering using XSRF tokens in the manifest url, to allow apps to generate per-user manifests securely (eg: dogfooding or A/B experimenting certain manifest changes).

ralphch0 on 9 Dec 2020

Something to think about when discussing "manifest_url" (see this comment above) are data URLs: <link rel="manifest" href="data:application/manifest+json;charset=utf-8,…">. I have seen PWAs repeatedly create manifests dynamically based on a JSON object.

tomayac on 17 Dec 2020

😕1

oh jeez wow not sure how we would deal with that.....
if we are OK with complicating things even more, we could say that data urls are uniquely identified by the start_url....

I'd prefer to ban that, and say people should serve a manifest from their serviceworker if they really want it to be locally generated.

dmurph on 29 Dec 2020

Data URLs are perfectly valid URLs and have been supported for web app manifests for >5 years. I would be very uneasy about banning them - needing to do so suggests that the solution which requires banning them may not be a sufficient one.

dominickng on 5 Jan 2021

👍3

Should we consider decoupling the two issues?
1) Introducing an id field that that takes precedence over the default id scheme
2) Consolidating the default id scheme across platforms.

ralphch0 on 7 Jan 2021

We should also spec a serialisation format for app IDs so sites can refer to each other.

alancutter on 2 Feb 2021

@alancutter wrote:

We should also spec a serialisation format for app IDs so sites can refer to each other.

Or alternatively, if the manifest URL is used as the stable identifier for a web application, there's no need to invent a new ID serialisation format which can be used to link to other web apps.

Having the identifier dereference to the manifest could also have added benefits in use cases like the pwa-url-handler one linked above, because its metadata could directly be used to generate UI elements representing the handling apps (e.g. using the app title and icon in a permission prompt).

benfrancis on 2 Feb 2021

@philloooo is doing an in-depth analysis here that we should be able to publish here by the end of the week.

So far manifest_url seems like it might be a security issue -

pretend Bing now has a music service, Bing Music, which is a PWA.

Along comes malware.com....
malware.com/manifest.json:

{
  ...
  id: "https://music.bing.com/manifest.json",
  name: "Bing Music!",
  start_url: "https://music.bing.malware.com",
  ...
}

And now malware.com has taken over bing music!

Anyways, explainer coming soon, and we're excited for feedback!

dmurph on 2 Feb 2021

@dmurph wrote:

So far manifest_url seems like it might be a security issue
...
{
  ...
  id: "https://music.bing.com/manifest.json",
  name: "Bing Music!",
  start_url: "https://music.bing.malware.com",
  ...
}

To be clear, this is not what I mean when I say use the manifest URL as the identifier for the app.

Don't parse the manifest URL from an id member inside the manifest, use the URL from which you fetched the manifest as its identifier. That way only music.bing.com can host a manifest with the identifier of https://music.bing.com/manifest.json

Also, IIRC start_url currently has to be same-origin as document URL, so if this manifest was linked from a document hosted at malware.com then this start_url would not be allowed and would fall back to the document URL the app was installed from.

benfrancis on 2 Feb 2021

It is a legitimate use case to host a manifest on a different origin to the start URL (e.g. a CDN). music.bing.com could have an identifier of cdn.bing.com in that case.

And while start_url currently has to be same-origin as document URL, https://github.com/w3c/manifest/pull/670 proposes to change that to the parent path of the manifest URL.

dominickng on 3 Feb 2021

Great - I think we can have a really good discussion soon once our explainer is out and we can talk about this stuff and consider all options :)

Sorry to jump ahead a bit!

dmurph on 3 Feb 2021

@dominickng wrote:

It is a legitimate use case to host a manifest on a different origin to the start URL (e.g. a CDN). music.bing.com could have an identifier of cdn.bing.com in that case.

Yes that's fine, by using the URL the manifest was served from as the ID, by definition an ID of https://cdn.bing.com/manifest.json can only be served from https://cdn.bing.com/manifest.json. There still can't be a manifest at https://malware.com/manifest.json with an ID of https://cdn.bing.com/manifest.json.

And while start_url currently has to be same-origin as document URL, #670 proposes to change that to the parent path of the manifest URL.

Yes, that would be awesome. But as I understand it, even if that change finally lands, it just means that if https://malware.com/manifest.json doesn't provide a start_url then its start_url defaults to https://malware.com/. It still can't provide a start_url of https://music.bing.malware.com because if both the document URL and start_url are provided they still have to be same-origin.

Even if #668 succeeds in detaching manifest processing from the document URL entirely (which would be even more awesome), there still couldn't be a manifest at https://malware.com/manifest.json with an ID of https://cdn.bing.com/manifest.json.

I'm looking forward to reading Google's latest analysis, but so far from this 3.5 year thread my personal conclusion is that it really boils down to whether enabling URL versioning of manifests without needing to set up HTTP redirects is worth all the added complexity of:

Inventing a new ID/serialisation/linking system for web apps, parallel to web URLs, making web apps less directly linkable and discoverable
Making manifest updates a more complex multi-step process for operating systems, which can only take place when a user happens to visit a page which links to a manifest with a matching ID, using the same user agent they used to install the application
Dealing with the case of different manifests from different URLs claiming the same ID, which could itself be a security issue if not handled carefully

I think you could argue this risks compromising four key attributes of Progressive Web Apps: linkable, discoverable, safe and fresh.

I realise using manifest URL as the identifier would mean changing the current behaviour of Chrome desktop, but that seems like a comparatively small price to pay (which @dmurph has already volunteered to take on 😉).

benfrancis on 3 Feb 2021

I'm quite compelled by preserving the linkability of manifest URLs as IDs.
We could support manifest versioning via imports instead.
E.g.

https://app.com/manifest.json:
{
  "import": "manifest-r31.8.json"
}

alancutter on 4 Feb 2021

Good point that using manifest_url is more linkable. Can you give an example use case that having the ID being a resolvable URL is needed/better?

Re: manifest_url versioning, If I understand it correctly, doing 301 redirect, or add an "import" syntax means that: the document will always still reference the original manifest, so user agents will keep the same previous ID for the app, and always do another round trip to fetch the latest content?

Note: we also still have the use case of specifying manifest as data url.

philloooo on 4 Feb 2021

@philloooo wrote:

Good point that using manifest_url is more linkable. Can you give an example use case that having the ID being a resolvable URL is needed/better?

I have a few key use cases in mind:

Updates - If user agents can index web apps by their manifest URL, all they need to do to get the latest version of the manifest is to fetch that URL. This would help fill a key hole in the manifest specification regarding how to update a manifest, where the alternative would be much more complex. This is particularly useful where the web runtime is separate from the operating system (i.e. not something like Chrome OS or Firefox OS), because the operating system can fetch updates itself (e.g. to update icons on an app launcher). If the manifest URL does need to change for some reason, an HTTP redirect can tell the user agent to replace the current app rather than install a new one.
Remote installation
1. App stores - We don't really talk about the fact that the way Microsoft uses manifests in the Microsoft Store is not compliant with the specification, because you can only really install an app from a document belonging to the app itself. Having the app ID resolve to its manifest is a step towards allowing installation of web apps from app stores/directories, if all you need to link to and install an app is its manifest URL. (See also: #668).
2. App provisioning - Similarly, it could be possible in future to remotely provision a web app to a collection of user agents (e.g. a classroom full of Chromebooks), just by sending the manifest URL.
3. Digital signage - Similar to the above, I'm currently working on a use case in digital signage where I'd like to be able to remotely install a web app to a kiosk runtime, which doesn't have its own user agent UI to install a web app. The kiosk runtime runs a single web app at a time full screen. It's quite common in digital signage to remotely provision APKs to an Android runtime, but the same is not currently possible with web apps.
Link relations - There was an example linked above where one web app manifest might want to refer to another web app manifest, to denote some kind of relationship between to the two apps. By using a simple web URL as an identifier it's possible to use link relations which describe all kinds of relationships between apps. Having the identifier resolve to metadata about the app is even more convenient (e.g. in the linked use case, for building the UI for a permissions prompt using the app's icon and title).

Re: manifest_url versioning, If I understand it correctly, doing 301 redirect, or add an "import" syntax means that: the document will always still reference the original manifest, so user agents will keep the same previous ID for the app, and always do another round trip to fetch the latest content?

As described above, if a user agent receives a redirect response when fetching a manifest URL, it can choose to replace the app (including its new ID), rather than install the new manifest as a separate application. That way the redirect only has to happen once on the user agent (but the server would need to maintain redirects for old manifest URLs for user agents which haven't updated yet).

The import approach could also work and might be more CDN friendly, but has the downside of always requiring an additional round trip, as you say.

There are obviously also other ways of telling a user agent that a manifest has changed which don't involve changing its URL, or re-fetching the whole manifest, (e.g. ETag or Last-Modified headers with less aggressive caching).

Note: we also still have the use case of specifying manifest as data url.

That would still work, a data URL can be used as an ID (I've done this in the past for storing icons). Obviously the limitation of choosing to use a data URL for a manifest is that if the manifest content changes, the manifest URL also changes and it becomes a new web app. But to me that feels like a reasonable compromise for such an edge case.

benfrancis on 4 Feb 2021

@philloooo wrote:

Good point that using manifest_url is more linkable. Can you give an example use case that having the ID being a resolvable URL is needed/better?

I have a few key use cases in mind:

Updates - If user agents can index web apps by their manifest URL, all they need to do to get the latest version of the manifest is to fetch that URL. This would help fill a key hole in the manifest specification regarding how to update a manifest, where the alternative would be much more complex. This is particularly useful where the web runtime is separate from the operating system (i.e. not something like Chrome OS or Firefox OS), because the operating system can fetch updates itself (e.g. to update icons on an app launcher). If the manifest URL does need to change for some reason, an HTTP redirect can tell the user agent to replace the current app rather than install a new one.

Updates are definitely easiest in the manifest_url=global_id case. They are still possible using other schemes, but less nice.

Remote installation

App stores - We don't really talk about the fact that the way Microsoft uses manifests in the Microsoft Store is not compliant with the specification, because you can only really install an app from a document belonging to the app itself. Having the app ID resolve to its manifest is a step towards allowing installation of web apps from app stores/directories, if all you need to link to and install an app is its manifest URL. (See also: #668).

I would expect that there would have to be a little more of a verification step here, either by the user agent or(and) by the store, to make sure that the document at start_url has a manifest link that matches the manifest_url. This step would probably be similar with other id schemes as well.

App provisioning - Similarly, it could be possible in future to remotely provision a web app to a collection of user agents (e.g. a classroom full of Chromebooks), just by sending the manifest URL.

This functionality, in theory, already exists in Chromium, as you can policy-install apps. It definitely has more hoops to jump through (loads the given URL, looks for manifest link, then uses that), but is possible.

Digital signage - Similar to the above, I'm currently working on a use case in digital signage where I'd like to be able to remotely install a web app to a kiosk runtime, which doesn't have its own user agent UI to install a web app. The kiosk runtime runs a single web app at a time full screen. It's quite common in digital signage to remotely provision APKs to an Android runtime, but the same is not currently possible with web apps.

Do you mind expanding on this? Would web packaging solve this use case? I could see us sending a web package to the kiosk as an option here too. I can also see the above policy-install functionality work too (more round-about, yes, but it works)

Link relations - There was an example linked above where one web app manifest might want to refer to another web app manifest, to denote some kind of relationship between to the two apps. By using a simple web URL as an identifier it's possible to use link relations which describe all kinds of relationships between apps. Having the identifier resolve to metadata about the app is even more convenient (e.g. in the linked use case, for building the UI for a permissions prompt using the app's icon and title).

I find this interesting! This is a nice feature. I think this is the first case here where it more on the impossible-side (or too-klunky-to-be-workable) for other schemes vs this scheme. But maybe I haven't thought of a way - any ideas how this could work in a different id scheme?

Re: manifest_url versioning, If I understand it correctly, doing 301 redirect, or add an "import" syntax means that: the document will always still reference the original manifest, so user agents will keep the same previous ID for the app, and always do another round trip to fetch the latest content?

As described above, if a user agent receives a redirect response when fetching a manifest URL, it can choose to replace the app (including its new ID), rather than install the new manifest as a separate application. That way the redirect only has to happen once on the user agent (but the server would need to maintain redirects for old manifest URLs for user agents which haven't updated yet).

The import approach could also work and might be more CDN friendly, but has the downside of always requiring an additional round trip, as you say.

There are obviously also other ways of telling a user agent that a manifest has changed which don't involve changing its URL, or re-fetching the whole manifest, (e.g. ETag or Last-Modified headers with less aggressive caching).

Note: we also still have the use case of specifying manifest as data url.

That would still work, a data URL can be used as an ID (I've done this in the past for storing icons). Obviously the limitation of choosing to use a data URL for a manifest is that if the manifest content changes, the manifest URL also changes and it becomes a new web app. But to me that feels like a reasonable compromise for such an edge case.

We could also do something weird like - say that the id = scope + "/manifest.webmanifest". Probably a bad idea. Any ideas about how to allow these to update & maybe migrate to a manifest file?

I think the main other issue with manifest_url = global_id is that you are STUCK with the url - if that was on a CDN, and that CDN goes out of business... or goes down, etc. You are stuck forever. Probably would mean, if we choose this option, we encourage devs to make sure they control the URL & 301 (is there a dns redirect you can do?) or something to a CDN if they need to. But I could see this biting people in the butt.

dmurph on 4 Feb 2021

One other use case would be some means of informing users they already have an app installed if they go to install the PWA from a different browser. Not so much to block it, but to inform the user that they will have 2 copies of the app (which could be confusing).

aarongustafson on 4 Feb 2021

@dmurph wrote:

Digital signage - Similar to the above, I'm currently working on a use case in digital signage where I'd like to be able to remotely install a web app to a kiosk runtime, which doesn't have its own user agent UI to install a web app. The kiosk runtime runs a single web app at a time full screen. It's quite common in digital signage to remotely provision APKs to an Android runtime, but the same is not currently possible with web apps.

Do you mind expanding on this?

Basically I have a kiosk runtime which acts as a kind of remote control web browser. It hosts a web interface (and WoT API) into which you can enter a URL to tell the kiosk to load that web page on its screen and act as a digital sign. In addition to just loading a web page I'd like to be able to remotely install a web app to the kiosk, including installing a service worker so that content can be cached for offline use in the case of network outages. The kiosk can also be interactive, so the runtime may want to restrict users to the navigation scope of the installed app. The kiosk runtime does not have a user interface for navigating to and installing web apps itself, that functionality is only provided remotely via a separate UI which doesn't have access to documents inside the browsing context. Ideally I would like to be able to remotely install a web app onto the kiosk from that UI using its manifest URL.

Would web packaging solve this use case? I could see us sending a web package to the kiosk as an option here too.

I'm not sure, I'm afraid I've lost track of the latest attempts to bring packaging to the web since packaged apps were such a disaster for Firefox OS. But ideally I'd like to be able to just give the kiosk any manifest URL and have it pull down a PWA itself.

I can also see the above policy-install functionality work too (more round-about, yes, but it works)

I agree it could be possible to implement a workaround for this use case using another approach, but it would be much more complicated and unreliable to have to download and parse HTML files in order to extract manifest link relations and find manifest URLs (I'd rather leave the tricky task of parsing HTML to the browser engine, rather than my application).

I think the main other issue with manifest_url = global_id is that you are STUCK with the url

I understand why this might be tricky in some production environments, but I personally happen to think it's a good thing.

benfrancis on 4 Feb 2021

@benfrancis wrote:

I can also see the above policy-install functionality work too (more round-about, yes, but it works)

I agree it could be possible to implement a workaround for this use case using another approach, but it would be much more complicated and unreliable to have to download and parse HTML files in order to extract manifest link relations and find manifest URLs (I'd rather leave the tricky task of parsing HTML to the browser engine, rather than my application).

As a side note, if you're doing this with a build of chromium, you can use the kWebAppForceInstall pref today - that will force-install apps as described above today, with some override options as well.

dmurph on 4 Feb 2021

I thought of a use case for Chromium browsers that makes manifest_url difficult - "Create Shortcut..." apps. These currently create a fake manifest if there is none, and then will 'upgrade' to a real manifest if one is ever linked. Unless I'm missing something easy here, these would have to be handled specially if we end up going with the manifest_url approach.

dmurph on 5 Feb 2021

Manifest-less Create Shortcut apps should use the start_url as their ID. Once a manifest is detected we can upgrade the app to use the manifest URL. This may require a user prompt to avoid cross-device sync conflicts.

alancutter on 5 Feb 2021

@benfrancis

As described above, if a user agent receives a redirect response when fetching a manifest URL, it can choose to replace the app (including its new ID), rather than install the new manifest as a separate application. That way the redirect only has to happen once on the user agent (but the server would need to maintain redirects for old manifest URLs for user agents which haven't updated yet).

This means though that the site can never switch to linking to the new manifest url. Since it would not know if the client still has the old version of the app keyed to the old manifest url. The user agent would probably need to refetch all the existing manifests on the domain first to make sure the IDs are up to date, a process that's likely to be brittle (defunct urls, temporary server errors, etc..).

ralphch0 on 5 Feb 2021

@ralphch0 wrote:

This means though that the site can never switch to linking to the new manifest url. Since it would not know if the client still has the old version of the app keyed to the old manifest url. The user agent would probably need to refetch all the existing manifests on the domain first to make sure the IDs are up to date, a process that's likely to be brittle (defunct urls, temporary server errors, etc..).

What are you worried will happen if the user agent visits a page linking to the new manifest before it has updated to that new manifest? And whatever it is, wouldn't that always happen in the alternative case of an id which can only be updated after following a link from such a page? The manifest could even theoretically never get updated if the user only ever visits pages which were within the navigation scope of the previous manifest but outside the navigation scope of the new manifest and/or no longer link to the manifest?

At least with a fixed manifest URL the user agent always has the opportunity to update.

benfrancis on 5 Feb 2021

In the alternative case, If a manifest is updated , the document that's within the previous navigation scope will update their manifest link to the latest one or the document url will redirect to the new document url, but still keep the stable ID if the apps' origin is not changed.

I think either options, the user agents always have ways to update. The difference is with your proposal of how to do the redirect,
the ID will be changed and we could hit a race condition for user agents, that loading the latest page finds the manfiest_v2,
before existing installed app v1 is updated to v2, and result in duplicated apps being installed. Or we need to follow what @ralphch0 said to go through all existing apps to update their manifests whenever you browse to a link.

This also gets trickier with user agents that support syncing apps across devices.

philloooo on 5 Feb 2021

@philloooo wrote:

In the alternative case, If a manifest is updated , the document that's within the previous navigation scope will update their manifest link to the latest one or the document url will redirect to the new document url, but still keep the stable ID

That's an interesting solution, I hadn't considered the possibility of pages which are no longer within the navigation scope of the application continuing to link to the manifest. Could that cause problems for new users who come along and install the web app from those old pages though? That might be OK, as long as the page doesn't become part of a new navigation scope (e.g. if a scope is split off into two separate apps or two apps are combined into one scope).

if the apps' origin is not changed.

This is of course another limitation of "an opaque token that uniquely identifies the app within the origin's namespace", it may not be safe to move the app/manifest to a different origin, including switching CDN.

I think either options, the user agents always have ways to update. The difference is with your proposal of how to do the redirect,
the ID will be changed and we could hit a race condition for user agents, that loading the latest page finds the manfiest_v2,
before existing installed app v1 is updated to v2, and result in duplicated apps being installed. Or we need to follow what @ralphch0 said to go through all existing apps to update their manifests whenever you browse to a link. This also gets trickier with user agents that support syncing apps across devices.

I agree that could be a problem. It seems recoverable though, by merging app installations once a redirect to the new ID is detected.

Ultimately I think if you want the start URL, scope _and_ manifest URL to all be changeable, then there are going to be edge cases where things can get into a broken state. Introducing a new non-web ID namespace into the equation can solve some of those problems, but also create others (e.g. getting stuck in an update loop where two different pages point to two different manifests claiming the same ID).

I posit that there is no perfect solution, which is why after about a decade of discussing this topic (I'm not joking), we still haven't landed on a solution upon which everyone agrees.

I still personally think manifest URL is the best solution. The second best solution I've come across is origin + scope, but that has problems too. Bad experiences from packaged apps in Firefox OS make me very wary of introducing a new non-web ID namespace for web apps. If there absolutely has to be an ID inside the manifest, I would suggest it should at least be a URL which defaults to manifest URL, but I'm concerned that will paint us into a corner where #668 (and therefore some of the use cases described above) become difficult or impossible.

benfrancis on 8 Feb 2021

Hi All, I created an explainer doc in my personal repo. Please take a look and create issues if you think there are missing use cases/incorrect information or any suggestion for edits!
We can still keep the ongoing conversations about which way is better here.

philloooo on 9 Feb 2021

And to report current conclusions on the doc - the current best solution, given all the use cases that we found, is the "global_id = start_url_origin + manifest_specified_id", where the default value of "global_id" is the start_url.

We always could have missed something though, so eyeballs & feedback appreciated on that explainer :)

dmurph on 10 Feb 2021

Thanks @philloooo & @dmurph for the explainer.
Personally in agreement with the conclusions of the doc.

One comment I had was regarding the default ID when the "id" field is missing:

Feels to me like if we were building this from scratch, we would likely go with scope? It's the same as service workers, and less likely to change. Sounds like the main reason start_url is preferred is because we would only be changing expectations for one set of devs (Android), but I wonder if long term scope will make more sense for future developers. Is start_url even that intuitive/expected by Desktop developers today?

I assume here that regardless of what we choose, already installed apps will be migrated to the new ID scheme? (i.e. on the first chrome version that has the new code, the ID'ing will be migrated to the new scheme, and updates to the app will continue working by matching using the new ID scheme). IOW, there should be no technical diff (since I assume almost all apps out there don't use a dynamically changing scope), just a change in developers mental model that they may need to be aware of.

ralphch0 on 12 Feb 2021

CC @fabricedesre @tantek @mounirlamouri

@philloooo wrote:

Hi All, I created an explainer doc in my personal repo. Please take a look and create issues if you think there are missing use cases/incorrect information or any suggestion for edits!
We can still keep the ongoing conversations about which way is better here.

Thank you for sharing that analysis, @philloooo.

When discussing behaviors on Android, this doc is exclusively only covering Chromium-based browsers on Android

...

When discussing behaviors on desktop, this doc covers Chromium-based browsers and Firefox (desktop).

As I understand it Firefox desktop does not support installing PWAs, so this analysis actually only covers Chrome desktop and Chrome for Android.

Taken for what it is (an analysis of what works best for the Chrome team, given their particular set of requirements and technical debt), I think the conclusions are completely reasonable.

Overall, if we use manifest_url as the global_id, we need to not support the use case of making manifest_url updatable or deal with non stable primary keys for managing apps across devices... If there weren't existing apps & infrastructure that relied on changing manifest_url, and this was being designed from the beginning, this option is a lot more attractive.

My reading of this is "If we'd have thought about this earlier we may have concluded that manifest URL was the best identifier for a web app, but since Chrome used start URL as the identifier and we never told developers that manifest URLs shouldn't change, we're now going to have to specify what Chrome desktop already does, but with a hack to work around the fact that start URL was always a bad choice as it's the URL which is most likely to change."

The explainer hasn't done anything to change my own conclusions about the best solution (manifest URL as ID, using HTTP redirects for discouraged URL changes), but it would be great if we could hear from other implementors.

To the best of my knowledge, the following user agents all support install/add to home using a web app manifest:

Android
- Firefox
- Chrome
- Samsung Internet
- Edge
- Opera
- Brave
- UC Browser
- QQ Browser
- Huawei Browser
- Baidu Browser (caniuse.com says manifest is supported, but I can't find any information)
iOS & iPadOS
- Safari
Windows
- Chrome
- Edge
MacOS
- Chrome
- Edge
Linux
- Chrome/Chromium
Chrome OS
KaiOS
Webian (for completeness, but probably only of interest to me, Webian Shell uses manifest URL as the ID)

As the explainer notes, user agents are not the only consumers of manifests. What about app stores? E.g.

Microsoft Store
Play Store
AppStore (App-Bound Domains)
Kai Store
PWA Rocks
PWA Store
ProgressiveApp.Store

benfrancis on 15 Feb 2021

Feels to me like if we were building this from scratch, we would likely go with scope? It's the same as service workers, and less likely to change. Sounds like the main reason start_url is preferred is because we would only be changing expectations for one set of devs (Android), but I wonder if long term scope will make more sense for future developers. Is start_url even that intuitive/expected by Desktop developers today?

I agree. The scope is a better choice if not considering change of expecations from one set of devs. A default value is supported mostly to allow smooth transition from existing web apps that don't have IDs. That's why we are favoring start_url as that's expected by existing desktop users.

I assume here that regardless of what we choose, already installed apps will be migrated to the new ID scheme? (i.e. on the first chrome version that has the new code, the ID'ing will be migrated to the new scheme, and updates to the app will continue working by matching using the new ID scheme).

They will be migrated to the new ID scheme and if we are choosing the ID scheme that has a new id field, the default value will be used if the apps don't have it specified.

As I understand it Firefox desktop does not support installing PWAs, so this analysis actually only covers Chrome desktop and Chrome for Android.

Oh thanks for pointing out, I just found out the announcement of them dropping the support recently. I will remove firefox desktop from the doc.

My reading of this is "If we'd have thought about this earlier we may have concluded that manifest URL was the best identifier for a web app, but since Chrome used start URL as the identifier and we never told developers that manifest URLs shouldn't change, we're now going to have to specify what Chrome desktop already does, but with a hack to work around the fact that start URL was always a bad choice as it's the URL which is most likely to change."

I think this is misrepresenting what I was saying in the doc. The doc was comparing manifest_url vs a id field. With id not specified, start_url was preferred as the default value. But that's totally different from using start_url as the id. And apparently id is the most stable option. I also tried to explain in detail that doing 301 redirect for use case of changing manifest_url either doesn't really support the use case, or ends up with unreliable behaviors for user agents.

Thanks for listing out the current user agents and PWA stores. Most of the user agents you are listed are chromium based.
As I stated in the doc, safari doesn't support updating/looking up installed PWAs, so it doesn't add any use cases relevant to the unique ID problem.
I'd be interested to know any use cases that I am missing in this doc.

PWA stores are being considered when comparing the pros and cons of the options. I can elaborate more on specific examples of PWA stores in the beginning of the doc if that's helpful.

philloooo on 16 Feb 2021

It seems like the main sticking point here is the 4th scenario - changing the manifest url. We know many sites do this, and crawling the data from https://pwa-directory.appspot.com shows us about 8.4% of manifests seem to have versions. Even with false positives, and that directory not being authoritative, that's a really high percentage, and a higher percentage than 2018:
https://github.com/w3c/manifest/issues/586#issuecomment-386820382.

If chromium never supported a changing manifest url, then this scenario wouldn't be as important, but unfortunately it was supported and thus people now rely on it. Seems to be the story of the web platform, for better or worse :/

Command:

curl -sSL 'https://pwa-directory.appspot.com/api/pwa/?limit=4000' | jq -r '.[] | .manifestUrl' | sort | perl -ne 'print if /[0-9a-fA-F]{7}/ || /v[0-9]+/ || /v=/'

Total PWAs & percentage:

last valid page: https://pwa-directory.appspot.com/?page=130
130 * 32 = 4160 total

353 PWAs in the below list
353 / 4160 = ~8.4%

Details

http://res.cloudinary.com/dyyjph6kx/raw/upload/v1502969304/webui/eng/icons/manifest.json
https://13tv.co.il/wp-content/themes/reshet_tv/assets/images/favicon/manifest.json?v=1.3.12.0
https://15f7fy16numklnwbn3m52e15-wpengine.netdna-ssl.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=303798283697
https://3235253.ucoz.ru/appmanifest/manifest.json
https://3235253.ucoz.ru/manifest.json
https://3235253.ucoz.ru/manifest_u.json
https://3235253.ucoz.ru/manifest_v.json
https://3235253.ucoz.ru/rss/manifest.json
https://7352323.ucoz.ru/7manifest.json
https://7352323.ucoz.ru/manif9est.json
https://7352323.ucoz.ru/manifest.json
https://ae.almosafer.com/mweb/manifest_almosafer.json?1539868957
https://agen.bukalapak.com/_nuxt/manifest.387a3d55.json
https://alleideen.com/manifest.0d8eb347138956bc50bca91b31fb6204.json
https://app.milanote.com/manifest-20201221.json
https://app.truewealth.ch/assets/favicons-d5c20aa/manifest.json
https://app.uplancer.io/_nuxt/manifest.dacc6f01.json
https://aqi.breezo.in/manifest.json?v=0.2
https://arquivosapk.com/wp-json/pwa-for-wp/v2/pwa-manifest-json
https://assets.slideslive.com/assets/favicon/manifest-1145378c0ec8e607c095f2fe43a6a16e7c22f86161165c502d836eebe2452832.json?v=vMr5gnj6OR
https://assets.vita4you.gr/static/version1575339893/frontend/Fixit-mobile/vita4you/el_GR/manifest.json
https://assets.vodspy.de/img/favicons/manifest.json?v=rM3LLEeJYB
https://atmocast.com/static/weather_main/www/static/weather_main/dist/manifest/manifest.eea759a01fcb.json
https://auskunft.avv.de/manifest.json?v=2.3.4
https://auth.magicleap.com/manifest.json?v=20171020
https://b2v8w6eq1p1erh.cdn.jp.idcfcloud.com/sp/top/json/manifest.json
https://badoo.com/badoo/manifest-en.json?v101
https://bael-theme.jake101.com/_nuxt/manifest.a2e69f0a.json
https://banca.oimparcial.com.br/app/themes/banca/assets/imgs/icons/manifest.json?v=518
https://bber.unm.cool/_nuxt/manifest.6ac3b74e.json
https://bitsofco.de/assets/manifest.json?v=25a0d663d4
https://bitsofco.de/assets/manifest.json?v=aa20601b17
https://brex.com/icons-e860f2dd2739fb15edb772012ada22b9/manifest.json
https://buddy.works/manifest.json?v=800
https://burdie.co/wp-json/app/v1/pwp-manifest
https://catalogue.accasoftware.com/mn-catalogo-en.json?v=110
https://cdn.framebridge.com/assets/favicons/manifest-33209ed395f245282c3a56d5cad485e1.json?v=BGBGy60NMJ
https://cdn.launchforth.io/7bf59d7/images/forth/favicons/manifest.json?v=OmyQRN88Nf
https://cdn.quiqup.com/webapp/qa/2.27.2/develop/f315cdaa3883ac2a24a2fb7ca2f8cec33c7d1d74/manifest.5c43e6283efbdbaa8fd7d567201c273b.json
https://cdn.shopify.com/s/files/1/0086/4865/4895/t/2/assets/site.webmanifest?v=10225873222616858641
https://cdn.shopify.com/s/files/1/0196/9616/t/72/assets/manifest.json?8865377658911702158
https://cdn.shopify.com/s/files/1/0199/9492/t/112/assets/manifest.json?15459288187714632786
https://cdn.shopify.com/s/files/1/1346/5473/t/60/assets/manifest.json?9199080155338096461
https://cdn.shopify.com/s/files/1/1366/8315/t/7/assets/manifest.json?8987383188635618396
https://cdn.shopify.com/s/files/1/1425/6082/t/97/assets/manifest.json?6342006907692054657
https://cdn.shopify.com/s/files/1/1751/8993/t/60/assets/manifest.json?3304484036080071639
https://cdn.shopify.com/s/files/1/2546/6304/t/3/assets/manifest.json?13114392473843896000
https://cdn.shopify.com/s/files/1/2964/9158/t/27/assets/manifest.json?14159899360358630437
https://celebrities.pictures/wp-json/wp/v2/web-app-manifest
https://chastnik-m.ru/manifest.json?v=0.2
https://club.tarifhaus.de/manifest.json?v=v1.17.0
https://competitiondigest.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=467456738029
https://connery.dk/graphics/manifest.json?version=1532614009
https://d2zcl7c6767qct.cloudfront.net/web-static-08/web_weathertab/favicon/manifest.2dee4bb9ca57.json
https://d30hza86imyv6n.cloudfront.net/statics/js/fcm/manifest.json?v=20180717122826
https://dbushell.com/manifest.webmanifest?v=10.2.0
https://de.tonybet.com/assets/theme15/images/favicons/manifest-449749a474a645bd5d1b8ada7264e2c5.json
https://decider.com/wp-content/themes/nypost-2016/static/decider-manifest.json?ver=1ea14db262b1746d91d4
https://deliverydudes.com/manifest.json?v=3.4.2-fbbd0343066d0de6d6899ebfdc26a8a2e6bcc035-desktop
https://deliverydudes.com/manifest.json?v=3.4.3-22b7c67ac3ede49412caab9ba33a59f88fa6929b-desktop
https://demo.scandipwa.com/static/version1573566737020/frontend/Scandiweb/pwa/en_US/Magento_Theme/manifest.a848d42b8c744af528977d4408ceb768.json
https://designwebkit.com/manifest.json?v1
https://dev.bber.unm.edu/_nuxt/manifest.6ac3b74e.json
https://dj-extensions.com/templates/dj-exetensions-v3/icons/manifest.json
https://dl.myqiwei.com/jdb-gamehallv3/pool/manifest.29b173a7.json
https://do.getcompliant.com/manifest.json?v=20180927
https://dresslily.com/manifest.json?20180117
https://dwbxi9io9o7ce.cloudfront.net/firebase-cloud-messaging/manifest.83370ce1333a.json
https://dwtricks.net/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=862555086377
https://earningkart.in/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=511885697223
https://ecourse.el-css.edu.om/manifest.8d9136a424a4e9fe28c4540e560b2eb4.json
https://edugorilla.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=482472720033
https://ehazi.hu/manifest.json?v=2
https://emojibombs.com/manifest.a73d7e65.webmanifest
https://emojibombs.com/manifest.d0fb4354.webmanifest
https://ethiopian-calendar.netlify.com/assets/icons-b23883c8/manifest.json
https://eurekamag.com/favicons/manifest.json?v=rM3ed7neBK
https://face2faceafrica.com/wp-content/themes/f2fa-v3/manifest.json
https://facerepo.com/app/images/favicons/manifest.json?v=8190
https://farlofacile.com/s/tmpl/on/manifest/manifest.json?v=dLBXnGO3e6
https://findpizza.space/_nuxt/manifest.99f3e3ec.json
https://fotobiz.pro/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=151586796557
https://gazeta-licey.ru/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=338097788326
https://geekpublicitario.com.br/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=45077066116
https://generative.fm/c58bca5e9803118f86e9c9afcf9bdb1f.webmanifest
https://gruzovichkof.ru/site.webmanifest?v=69BqJv4KQN
https://hackr.io/manifest.json?v=1.1
https://hardwaresfera.com/wp-json/pwa-for-wp/v2/pwa-manifest-json
https://health.cloudtcm.com/_nuxt/manifest.cb7a46f8.json
https://hindi.indiansutras.com/browser.json?v=1.0.2
https://horacekeung.github.io/scrum-poker/_nuxt/manifest.3fb4b0a0.json
https://huren.flixbus.nl/manifest.json?dcea861a57
https://hydrobuilder.com/skin/frontend/rwd/hydrobuilder/manifest.json?v=NmP8E8oJLa
https://infohookah.ru/wp-json/app/v1/pwp-manifest
https://intriper.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=10954234324
https://invidio.us/site.webmanifest?v=a3045a3
https://is-nb.a8e.net.br/images/mobile/favicon/manifest.json?v=92aaaa
https://jcl.cz/favicons/manifest.json?v=8j6XJMAvAp
https://job-draft.jp/assets/favicon/manifest-5e26f226e3746a244b9991698dd9f76d0d2a8498f2f0e221ad4bffb2aeeb067e.json
https://journalisticapp.com/_nuxt/manifest.452c72d4.json
https://jovemaprendizbr.com.br/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=878054065872
https://kannada.indiansutras.com/browser.json?v=1.0.2
https://lavito.pl/images/favicon/manifest.json?v=20160928100049
https://leifheit.de/manifest.5cdb6c2deddae2a7fde58d6f1d9ed298.json
https://lifeacademy.ru/Media/favicons/manifest.json
https://liveindex.org/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=247458917002
https://livetoday.online/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=46304322746
https://liwli.ru/s/i/manifest.json?v=5A5ydyp6Jy
https://lowerspendings.com/_nuxt/manifest.070d2d8c.json
https://lunarworks.co.uk/site.webmanifest?v=XBzAaWkGGw
https://m-loi-pinel.trouver-un-logement-neuf.com/manifest.json?v=8jeqgYk97l
https://m.ithome.com/json/manifest.json?v=20190214
https://m.ithome.com/json/manifest.json?v=2019042801
https://m.livingathome.de/r1531314069712/manifest.json
https://m.met.hu/site.webmanifest?v=2bBRvozOrO
https://m.visitjeju.net/_nuxt/manifest.b93ff49c.json
https://malayalam.indiansutras.com/browser.json?v=1.0.2
https://market24hclock.com/manifest.json?v=2016.01
https://matexperience.com/site.webmanifest?v=694olv6vGq
https://meapaixonei.com.br/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=766161200110
https://melaku.ml/assets/icons-b23883c8/manifest.json
https://mmm.dk/graphics/manifest.json?version=1532529712
https://mobile.48365365.com/WebAppManifestV2.json
https://msk.gruzovichkof.ru/site.webmanifest?v=69BqJv4KQN
https://mundomulheres.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=703126565085
https://net4game.com/manifest.json?v=3
https://new.reddit.guide/_nuxt/manifest.3fd2d7ad.json
https://nico.news/_nuxt/manifest.e3e59dbe.json
https://nocorruption.in/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=548458404239
https://number-place-puzzle.net/manifest.json?20180620
https://numerics.info/_nuxt/manifest.5c7e420c.json
https://oimparcial.com.br/app/themes/impar/assets/imgs/icons/manifest.json?v=3
https://optinium-apps.co.uk/Assets/Icons/site.webmanifest?v=PYePN5lxOa
https://parismatch.be/app/themes/parismatch-v1/assets/images/favicons/manifest.json
https://prod-static.disney-plus.net/builds/a823eb64c7e453be430322b0d8a71a8e4ae624e3_1579906393987/images/favicons/manifest.json
https://psd.download.land/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=1040297877182
https://push.nextincareer.com/app/v3/manifest.json.php?gcm_sender_id=
https://pymex.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=34581308698
https://r34.app/_nuxt/manifest.02b4b57d.json
https://r34.app/_nuxt/manifest.6dc733da.json
https://r34.app/_nuxt/manifest.b0237dbf.json
https://r34.app/_nuxt/manifest.cd2b7647.json
https://r34.app/_nuxt/manifest.d10eabcf.json
https://radio.codemonkey-spb.ru/res/js/manifest.json?1ea25136
https://rainu.github.io/dev-notes/_nuxt/manifest.1a0b3aa1.json
https://reittiopas.hameenlinna.fi/icons-hameenlinna-dac763a78f4a5fda6d72934070bb5c33//manifest.json
https://resource.wakelet.com/v2/4.4.3/manifest.json
https://reteteculinare.ro/assets/push/manifest.json?t=1532538296
https://s1.hdslb.com/bfs/static/jinkela/mstation-h5/manifest.01d8c0c950f8b24714dd3e8afc8d88f8de18494b.json
https://sanieattivi.it/s/tmpl/on/manifest/manifest.json?v=dLBXnGO3e6
https://sayhello.ch/wp-json/app/v1/pwp-manifest
https://seekingalpha.com/samw/manifest.b3b95f524139044321637a3833ada808.json
https://sexshop69.pl/favicons/manifest.json?v=wAAwRrg2LW
https://siamoption.com/manifest.json?v=5
https://sizeer.cz/common/images/favicon/manifest.json?v=
https://sizeer.de/common/images/favicon/manifest.json?v=
https://sizeer.lt/common/images/favicon/manifest.json?v=
https://sizeer.sk/common/images/favicon/manifest.json?v=
https://smartelement.netsons.org/index.php?rest_route=/wp/v2/web-app-manifest
https://socialcdn.i-say.com/assets/data/manifest.json?t=1532528798881
https://startsat60.com/app/themes/startsat60v7/favicons/manifest.webmanifest
https://static-cache.md.uaprom.net/image/portal/icons/manifest_prom.json?r=31b0755eb4f7b6cae4042261a24a38ce
https://static.glampinghub.com/img/pets-manifest.95617630.json
https://static5.mnlcdn.com/assets/favicons/manifest.json?v=kPgJaJEYB0
https://studitemps.de/wp-content/themes/limitless/manifest.json?v=2
https://suzuki-club.ru/manifest.json?v=m2dX696Pkj
https://szybkagotowka.pl/manifest.json?v=pgrXrjp8JB
https://tamethebots.com/_nuxt/manifest.a71cb8c6.json
https://tcg.loke.dev/_nuxt/manifest.2ca00a5a.json
https://techmaniacs.gr/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=811528213463
https://tehnot.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=662102224587
https://teknobolge.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=372304022646
https://terahertz.fr/wp-json/pwa-for-wp/v2/pwa-manifest-json
https://thebroodle.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=304973626749
https://thriva.co/_nuxt/manifest.7b37346d.json
https://tonybet.com/assets/theme15/images/favicons/manifest-449749a474a645bd5d1b8ada7264e2c5.json
https://topwords.me/site!7e0a6c72546e9794ac2477c41d73a550.webmanifest
https://tribunaonline.com.br/theme/tol/assets/favicon/manifest.json?5bf6fec5b9
https://tsuriho.com/wp-content/themes/tsuriho_f0ca56f/icons/manifest.json
https://unsplash.com/site-v2.webmanifest
https://v8.dev/.webmanifest
https://web.sylinghim.com/wp-content/plugins/OneSignal-WordPress-Plugin-a176bc7f4bfe19d4deca899c0aa0085dad14c7b5/sdk_files/manifest.json.php?gcm_sender_id=22356042089
https://webimpng.web.app/manifest.json?h=c7a5441a53a565aa8f9f7497c1126f52
https://wiiu.hacks.guide/images/manifest.json?v=PYEmwKvQAx
https://wmarket.com.ua/manifest.json?v=2
https://worldissmall.fr/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=2397340514
https://www.4588888.ru/manifest.json
https://www.aquasana.com/manifest.json?v=xQwx0zQO2d
https://www.arteblog.net/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=349712862781
https://www.autabuy.com/manifest.json?v=4
https://www.b-mall.ro/assets/v-20191205160838/vendor/onesignal/manifest.json
https://www.babygames.com/manifest.json?2017022102
https://www.belk.com/on/demandware.static/Sites-Belk-Site/-/default/dwa7de5e17/manifest.json
https://www.birminghampost.co.uk/manifest.json?v=00277067608714089d14c9532ff9fa55
https://www.bonmarche.co.uk/on/demandware.static/Sites-BONMARCHE-GB-Site/-/default/dwd11febf9/images/webclipicons/manifest.json
https://www.bons-plans-astuces.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=495554646901
https://www.bridgemedia.ru/manifest.json?v=3
https://www.browsersync.io/manifest.json?v=qAqkxQaJm0
https://www.buddy.works/manifest.json?v=800
https://www.burtonmail.co.uk/manifest.json?v=d6678a8591dc5e45e3a5d07875099abe
https://www.bustimes.org.uk/static/manifest.7111e27983b6.webmanifest
https://www.carethy.net/imgs/4/favicons/site.webmanifest?v=2
https://www.chesterchronicle.co.uk/manifest.json?v=246d5c7d7c46fe9606e71016395ab784
https://www.chroniclelive.co.uk/news/manifest.json?v=2d9b353308db652f99447f552b69d088
https://www.cityplug.be/favicon/manifest.json?v200
https://www.cizgiroman.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=7072932560
https://www.club-vulkan-club.com/uploads/pwa/manifest.json?v=1491403533
https://www.club-vulkan-slots.org/uploads/pwa/manifest.json?v=1491403533
https://www.codecourse.com/_nuxt/manifest.bf25d7b2.json
https://www.coininvest.com/uploads/site_settings/manifest/manifest.json?v=22
https://www.concealednation.org/manifest.json?v=2bb3QegpEf
https://www.connery.dk/graphics/manifest.json?version=1532614013
https://www.connexion-emploi.com/manifest.json?1532617102
https://www.croydonadvertiser.co.uk/manifest.json?v=9bb6f5c11e7bb1905bc12840740df6f2
https://www.culinar.ro/assets/push/manifest.json?t=1532532470
https://www.culinar.ro/assets/push/manifest.json?t=1532533689
https://www.cybrhome.com/favicons/manifest.json?20180702184436
https://www.dahaboo.com/manifest.json?v=20180709
https://www.dbushell.com/manifest.webmanifest?v=10.2.0
https://www.debaser.it/manifest.json?v=3
https://www.designwebkit.com/manifest.json?v1
https://www.diana.bg/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=392133639149
https://www.dubaijobs77.com/manifest.json?sndid=103953800507
https://www.dublinlive.ie/manifest.json?v=5651aec8749f353696bdd3bc2190b30d
https://www.e-lens.com.br/_nuxt/manifest.d05cec94.json
https://www.elk.co/favicon/manifest.json?f194cabf7e92c09271e5cc21f5f08876
https://www.elkedagietsleuks.nl/manifest.json?_=1532556244
https://www.essexlive.news/manifest.json?v=8f5dbf5344afe4d4ebd21eb7a51ac322
https://www.facerepo.com/app/images/favicons/manifest.json?v=8190
https://www.fastfoodmenuprices.com/wp-content/uploads/fbrfg/site.webmanifest?v=lk2j03rYvj
https://www.feedough.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=507835320803
https://www.findafishingboat.com/manifest.json?v=vMMo8wbo3G
https://www.findspecials.co.za/Images/Icons/manifest.json?v=JynK8GNLGe
https://www.finevent.top/_nuxt/manifest.3023ac05.json
https://www.finevent.top/_nuxt/manifest.b9d56fcd.json
https://www.finevent.top/_nuxt/manifest.e29fcaba.json
https://www.fontsquirrel.com/manifest.json?v=2
https://www.foodservicedirect.com/static/version1540372735/frontend/BrewInteractive/FsdTheme/en_US/assets/images/favicon.ico/manifest.json
https://www.football.london/chelsea-fc/manifest.json?v=4177f5a241e402b6a81f7603bc1b2701
https://www.freedym.com/main/wp-content/uploads/fbrfg/manifest.json?v=gAa4em0oyd
https://www.getsurrey.co.uk/manifest.json?v=6e38aa1ad34fbdb9bab04d45692696cc
https://www.gloucestershirelive.co.uk/manifest.json?v=8cfb8836786918f731c4c5838b3f9d13
https://www.goandroid.co.in/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=199824241014
https://www.goodsrepublic.com/media/manifest/manifest.json?20170919-213446
https://www.graphiccompetitions.com/site.webmanifest?v=Gvkqe429dM
https://www.grimsbytelegraph.co.uk/manifest.json?v=c8c7793858b56b2ea6a0db7b65941fe7
https://www.gruzovichkof.ru/site.webmanifest?v=69BqJv4KQN
https://www.gutezitate.com/manifest.json?v=2016
https://www.hertfordshiremercury.co.uk/manifest.json?v=2c9b01885310c626d32d56ac44cb795f
https://www.hpbn.co/7a58c37113db4464699ec4f4646b5566.json
https://www.hqroom.ru/manifest.json?v=777
https://www.icover.ru/manifest.json?v=A0RbE3GQBj
https://www.iheart.com/v8.24.1/2720e41/bundles/manifest.json
https://www.indianbodybuilding.co.in/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=813727397953
https://www.indiansutras.com/browser.json?v=1.0.2
https://www.inmyarea.com/manifest.json?v=1.0
https://www.inshared.nl/manifest.json?v3
https://www.jcl.cz/favicons/manifest.json?v=8j6XJMAvAp
https://www.jibo.com/wp-content/themes/jibo-v2/jibo.webmanifest
https://www.justwebworld.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=585128775620
https://www.koala.com.au/pwa/manifest.3b4de2b4.json
https://www.kosmetik4less.de/manifest.json?1532366103
https://www.leifheitsklep.pl/manifest.5cdb6c2deddae2a7fde58d6f1d9ed298.json
https://www.lifeacademy.ru/Media/favicons/manifest.json
https://www.lift.co/manifest.0279069221784098277312156ea5edc4.json
https://www.lincolnshirelive.co.uk/manifest.json?v=f5a577e6287fd6c05db173ddc72b10de
https://www.liverpoolecho.co.uk/all-about/liverpool-fc/manifest.json?v=6f25c85eff7278914c6d2b2e2ef98830
https://www.liwli.ru/s/i/manifest.json?v=5A5ydyp6Jy
https://www.magicleap.com/static/manifest.json?v=20171114
https://www.mahiroffice.com/wp-json/wp/v2/web-app-manifest
https://www.marktguru.at/assets/manifest.at.json?v3
https://www.minhatatuagem.com/wp-content/uploads/fbrfg/manifest.json?v=qAqvK0vEev
https://www.mirror.co.uk/manifest.json?v=e3a19310c81adecc1db07d28d9f6877a
https://www.mivoto.app/static/manifest.835ebd007499.json
https://www.mmm.dk/graphics/manifest.json?version=1532529717
https://www.monito.com/_sponse/manifest.142e82b8.json
https://www.motocms.com/manifests/manifest-en.json?v3
https://www.movieclock.com/manifestmovie.json?v=Misia
https://www.mrjmpl3-official.es/_nuxt/manifest.2ac1f5d0.json
https://www.mygadget.su/manifest.json?v=lkgnRnWjAx
https://www.nature.com/static/manifest.1a481c42b1.json
https://www.net4game.com/manifest.json?v=3
https://www.newsheadlines.com.ng/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=376386989667
https://www.numerics.info/_nuxt/manifest.5c7e420c.json
https://www.nur.kz/9497ff436/manifest.json
https://www.nydj.com/on/demandware.static/Sites-NYDJ-Site/-/default/dw13e5aa58/images/site.webmanifest
https://www.optimize.me/wp-content/themes/philosopher/images/favicons/manifest.json?v=X5rbgYn6zg
https://www.pescanova.pt/manifest.json?v=2.3.0
https://www.pinnacleperformanceandnutrition.com.au/a/sc/manifest.json?v=2
https://www.pkjobs77.com/manifest.json?sndid=103953800507
https://www.pooltracker.com/manifest.json?v=YAK4a4y4Rgc
https://www.presto-pizza.ro/bundles/mountfrontend/favico/manifest.json?version=$Id:%20c1488
https://www.qefira.com/assets/qe-site/favicon/site.a90bed74.webmanifest
https://www.qqcitations.com/manifest.json?v=2016
https://www.resto.be/across/resources/static/87c90c49965f12e0060453bbffe74b086c2f8b39/site/en.resto.be.json
https://www.retailnews.asia/wp-content/uploads/fbrfg/site.webmanifest?v=NmPGez5wvx
https://www.reteteculinare.ro/assets/push/manifest.json?t=1532538298
https://www.revealedrecordings.com/uploads/assets/1530694500.1528097602/static/manifest.json
https://www.rockantenne.de/assets/icons/rockantenne-de/manifest.json?v=2
https://www.scunthorpetelegraph.co.uk/manifest.json?v=c8c7793858b56b2ea6a0db7b65941fe7
https://www.sexshop69.pl/favicons/manifest.json?v=wAAwRrg2LW
https://www.share4you.com/en/wp-content/themes/share4youv2/manifest/en/manifest.json
https://www.siamoption.com/manifest.json?v=5
https://www.sizeer.cz/common/images/favicon/manifest.json?v=
https://www.sizeer.de/common/images/favicon/manifest.json?v=
https://www.sizeer.sk/common/images/favicon/manifest.json?v=
https://www.southportvisiter.co.uk/manifest.json?v=548e74556b39b6b25a2b7a4828f7783e
https://www.sozler.im/_nuxt/manifest.ffa97656.json
https://www.sparkk.tv/site.webmanifest?v=yyQK3y337b
https://www.speedtest-ptcl.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=238445687875
https://www.standoutbooks.com/manifest.json?v=PYY4W494ra
https://www.stavros.io/static/images/favicons/manifest.json?h=facfed0f
https://www.sumissura.com/en-us/manifest_json?v=1531917562
https://www.suzuki-club.ru/manifest.json?v=m2dX696Pkj
https://www.tajawal.com/mweb/manifest_tajawal.json?1539868353
https://www.tappytoon.com/v2/manifest.json?id=72de668d3ea21fc516b0
https://www.technodom.kz/static/version1573576417415/frontend/Technodom/pwa/en_US/Magento_Theme/manifest.472a41db5f3a18af69a553705dfa3fc6.json
https://www.tefenua.gov.pf/manifest.db4f1fcd181b13fb4f3a9158c895007c.json
https://www.theatlantic.com/assets/static/a/frontend/dist/theatlantic/manifest/lacroix/manifest.523d8b9c9b01.json
https://www.thehappening.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=237543628831
https://www.thesmackdownhotel.com/manifest.json?v=2017
https://www.thriva.co/_nuxt/manifest.7b37346d.json
https://www.tipsport.org/images/chance/favicon/manifest.json?v=2
https://www.todamujeresbella.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=628438311256
https://www.tonybet.com/assets/theme15/images/favicons/manifest-449749a474a645bd5d1b8ada7264e2c5.json
https://www.tribunaonline.com.br/theme/tol/assets/favicon/manifest.json?5bf6fec5b9
https://www.trulygeeky.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=749330624120
https://www.uninterrupted.com/d8182a65c526ea82b18d9b220253532e.json
https://www.vcclub-vulkan.com/uploads/pwa/manifest.json?v=1491403533
https://www.verkehrsmittelvergleich.de/columbus-assets/favicons/manifest-8ccb0648b58c03d33b070004605eacdf015ff1e132f4a6968ee7560afa57fa4d.json
https://www.visitjeju.net/_nuxt/manifest.1b1bbe02.json
https://www.vivus.com.ar/manifest.json?v=1.0.0
https://www.vivus.com.mx/manifest.json?v=1.0.0
https://www.vivus.dk/assets/favicon/manifest.json?v=1
https://www.vivus.lt/assets/favicon/manifest.json?v=1
https://www.vivus.lv/assets/favicon/manifest.json?v=1
https://www.vivus.pl/assets/favicon/manifest.json?v=1
https://www.viz.com/favicon/manifest.json?v=47MPqANpyj
https://www.volcano1.net/uploads/pwa/manifest.json?v=1491403533
https://www.vooks.net/img/fbrfg/manifest.json?v=GvJxPypopN
https://www.vulcan-klub.online/uploads/pwa/manifest.json?v=1491403533
https://www.vulclub.org/uploads/pwa/manifest.json?v=1491403533
https://www.we-are.travel/manifest.json?v=wAA9BoEKk8
https://www.webtures.com.tr/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=530367467346
https://www.westland.ru/favicons/manifest.json?v=Ryok6lQ6d3
https://www.whatsnewonnetflix.com/assets/favicon/manifest-ae3f7e8a4cb2298a9194b985101464e2348753f0f7a5015e174040de72de5e7d.json
https://www.winni.in/icons/manifest.json?v=m2leeBq7Gw
https://www.xn--kndigen-n2a.de/assets/manifest-77b841cae31bf680c8decc4641f376c14861b0537025485bb42d4897c780fb89.json
https://www.yaallah.in/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=499381340670
https://www.zoomcatalog.com/manifest.json?v6.2
https://www2.elkedagietsleuks.nl/manifest.json?_=1532556244
https://wynajem.flixbus.pl/manifest.json?dcea861a57
https://zellbury.com/static/version1595174166/frontend/Scandiweb/pwa/en_US/Magento_Theme/assets/manifest_v2.json

dmurph on 16 Feb 2021

👍1

@philloooo wrote:

I agree. The scope is a better choice if not considering change of expecations from one set of devs.

I also agree scope makes more sense conceptually than start URL. One other concern I have about scope though is that web apps may eventually need to support an array of scope URLs, so it could end up being a compound key which is less neat.

Oh thanks for pointing out, I just found out the announcement of them dropping the support recently. I will remove firefox desktop from the doc.

Firefox desktop has never supported installing web apps from a web app manifest. The recent decision to end the single site browser experiment doesn't change that, but it does remove one path to getting there.

I think this is misrepresenting what I was saying in the doc.

Sorry if my summary came across as a little cynical, I'm just a bit frustrated with how this has played out over the years.

I also tried to explain in detail that doing 301 redirect for use case of changing manifest_url either doesn't really support the use case, or ends up with unreliable behaviors for user agents.

I still think this could be workable. The worst case scenario is that users end up installing duplicate apps on the same device (if they don't remember they already installed it) or with different manifest URLs across different devices, which need de-duping as user agents follow redirects or synchronise with a server. I don't think it would be necessary for documents to always link to the original manifest URL, or for user agents to check all their manifests every time they encounter a new one.

Regarding the disadvantages you list for using processed manifest URL as a global ID...

S2: installed apps on desktop, S3: Apps that are installed on both Android and desktop needs migration.

Why would Chrome for Android need migration if it already uses manifest URL?

Change of implicit expectations for desktop WebApp developers

Whatever solution is chosen, the behaviour of some user agents (including at least one of Chrome desktop and Chrome for Android) will need to change

S6: Apps that specify manifest as data URL will not be updatable

Agreed, but this already isn't possible on most user agents today (Chromium desktop being the exception). This seems like a reasonable tradeoff given it is inherent in the nature of data URLs. Developers can switch to an HTTP manifest URL if they want to start supporting updates, requiring a one-off re-installation.

PWA stores will recognize different versions of manifest URLs as different web apps.

Depending on how PWA stores collect manifests (developer submission/user submission vs. web crawling) they may temporarily end up with duplicate records which need merging the next time a listing is updated. I agree this is the biggest disadvantage (and impacts sync services too), but I am of the view that developers should be discouraged from versioning their manifest URLs (like 92% of existing web apps), which would help reduce the impact.

S4: Apps that need to update manifest URLs will need to keep supporting the previous manfiest_url, handle 301 to the latest manifest forever. The document will need to always use the original manifest_url. An extra roundtrip will always be needed for loading the app.

I agree with the first point (in theory), but disagree with the second two points. Documents should always be able to link to the latest manifest URL.

If the manifest_url is hosted under a CDN, they will need to stick to that CDN provider.

I don't agree, they can redirect to a new one.

Difficult to reconcile with S8: apps installed from A2HS, which does NOT require a manifest. ID creating here is weird, and instead would have to have custom upgrade handling for when a manifest is found on the page.

I think fake apps should be considered out of scope given they don't follow the specification anyway. Fake apps could use a generated chrome:// URL as a manifest URL. If a manifest is later found on the same page I would argue it is desirable for that to be treated as a separate application to a fake app.

Most of the user agents you are listed are chromium based.

Yes, only Firefox, Kai OS and Safari are still using something other than Blink 🙁.

Do you think it's safe to assume that all Chromium-based user agents behave the same way as Chrome on desktop and on Android? (Genuine question as outside of the implementations I've worked on myself I don't know). Due to the interplay between the browser engine, browser chrome, underlying operating system and sometimes app store and sync service, evaluating support for the Web App Manifest specification is more complicated than for other web specifications which are implemented entirely inside a browser engine. Chromium-based user agents could have different browser chrome which implements add to home functionality differently, or use their own sync service or app store (Webian Shell for example is built on a desktop build of Chromium but uses manifest URL to identify web apps). Operating systems may also try to update apps independently of the browser they were installed from (I'm not sure whether this is currently the case).

As I stated in the doc, safari doesn't support updating/looking up installed PWAs, so it doesn't add any use cases relevant to the unique ID problem.

I don't think it's fair to exclude Safari from the analysis on this basis, given the specification doesn't currently define how to update a manifest it can't be expected for user agents to already support this. As I understand it Safari on iOS does store metadata from web app manifests when adding web apps to the home screen, so what it uses as an identifier to store those data still seems relevant. It should be assumed that Safari on iOS may support updating web apps in the future, if and when that feature is defined in the specification.

As I understand it Firefox for Android doesn't currently support updating manifests either.

I've chatted with people who work on Firefox for Android and KaiOS and tried to encourage them to contribute to this discussion. From what I can tell, Firefox for Android currently uses start URL to identify web apps. The upcoming release of KaiOS uses manifest URL.

I think if you count the number of user agents (including all the Chromium based browsers on Android), there are probably more user agents which currently use manifest URL than start URL.

I'd be interested to know any use cases that I am missing in this doc.

The remote installation and link relation use cases I mentioned above aren't explicitly called out as use cases or requirements, but they are mentioned within a couple of the possible solutions.

Out of scope

For a given id scheme, migrations between unique ids won’t be handled.

Migrating between origins.

FWIW if these weren't out of scope, it might be possible to handle both use cases with manifest URL as ID, using redirects.

benfrancis on 17 Feb 2021

@dmurph wrote:

It seems like the main sticking point here is the 4th scenario - changing the manifest url. We know many sites do this, and crawling the data from https://pwa-directory.appspot.com shows us about 8.4% of manifests seem to have versions.

Thank you for sharing this list, it helps illustrate the problem well. Although this also means 92% of manifests _don't_ version their manifest URLs, I acknowledge that it has created a real problem.

If chromium never supported a changing manifest url, then this scenario wouldn't be as important, but unfortunately it was supported and thus people now rely on it. Seems to be the story of the web platform, for better or worse :/

This is very frustrating to me, and a worrying sign of things to come for the web platform given Chromium's increasing dominance, but we are where we are.

The positives I see regarding Google's proposed solution are:

The id field in the manifest is optional
In practice all IDs are HTTP URLs, which is better than inventing a new non-web ID namespace (albeit URLs which don't resolve to a useful resource)

I'm wondering whether a better compromise between the best conceptual solution and the practical reality might be to add an id field as proposed, but make that id member an absolute URL and use processed manifest URL as the default. That would mean picking what Chrome for Android currently does as the default behaviour rather than what Chrome desktop does. It would allow start URL and scope to change without issues and allow developers who want to change the manifest URL to specify an ID inside the manifest in order to do so. Developers who don't want to change the manifest URL (but may want to change the start URL) can continue to use it as an identifier and don't need to use the optional id field at all.

A manifest with URL https://www.example.com/manifest.webmanifest but no id field would have the ID https://www.example.com/manifest.webmanifest

If the developer wants to change the manifest URL, they can add an id field to the manifest which matches the original manifest URL.

https://www.example.com/manifest.webmanifest?v=2

{
  "id": "https://www.example.com/manifest.webmanifest"
}

Apps would only be allowed to update using this approach if the origin of the manifest URL stays the same (switching origins could optionally be supported using HTTP redirects, which would happen far less often than the versioned URL use case).

Regarding the potential disadvantages:

S6: Manifests with data URL won’t be supported/updatable.

At first glance the approach above could work for data URLs too, by using the original data URL as an id inside the new data URL. Unfortunately because data URLs are treated as unique opaque origins that wouldn't be allowed, because otherwise any new manifest could claim to replace any existing manifest which uses a data URL.

This is the biggest downside, but seems like a reasonable tradeoff for that edge case given it is inherent to the nature of data URLs. Developers can always switch to an HTTP URL if they want to start supporting updates, requiring a one-off re-installation. Updating from a data URL is already not supported on any user agent except for Chromium desktop.

Change of implicit expectations for desktop WebApp developers

This is balanced out by the default behavior remaining the same for all those Chromium-based browsers on Android (and KaiOS).

S7: manifest is hosted under a different origin: could be a security concern to use a URL from different origin as the global key.

I would like to understand this better. What are the potential risks?

When S7: manifest is hosted under a different origin needs to change manifest_url, they can’t create an id field that matches previous manifest_url to keep the identity.

If the ID is an absolute URL rather than being resolved against the document URL, could this be possible (providing the new manifest URL has the same origin as the old one)?

S2: installed apps on desktop need to handle migration.

But all the Chromium-based browsers on Android don't.

S8: web apps that don’t have manifest URLs won’t be supported, they will have to use a different ID system and need to be migrated when they “upgrade”to have a manifest_url.

Is this about web pages which are added to the homescreen but don't have a manifest? It seems desirable to me that if a page starts linking to a manifest, the resultant app should be treated as separate to what is effectively a bookmark to a particular page within the app. Migrating from one to the other probably shouldn't be attempted.

benfrancis on 17 Feb 2021

...

@benfrancis wrote:
@philloooo wrote:

I also tried to explain in detail that doing 301 redirect for use case of changing manifest_url either doesn't really support the use case, or ends up with unreliable behaviors for user agents.

I still think this could be workable. The worst case scenario is that users end up installing duplicate apps on the same device (if they don't remember they already installed it) or with different manifest URLs across different devices, which need de-duping as user agents follow redirects or synchronise with a server. I don't think it would be necessary for documents to always link to the original manifest URL, or for user agents to check all their manifests every time they encounter a new one.

This is an edge case that is bad for users - we strongly want to avoid the case where duplicate apps would exist for a user, or the user would see an 'install' icon when that shouldn't be applicable (for this case & other cases). These types of issues are also really big for enterprise users who more often than not difficult-to-change infrastructure and need their user flows to go EXACTLY as expected. Weird things like install icons showing up or duplicate apps appearing.

@benfrancis wrote:

@philloooo wrote:

Change of implicit expectations for desktop WebApp developers

Whatever solution is chosen, the behaviour of some user agents (including at least one of Chrome desktop and Chrome for Android) will need to change

Kind of - the no default global_id has the option of no migration, but then kind of specs an opt-in migration with the legacy ids.

@benfrancis

@philloooo write:

S6: Apps that specify manifest as data URL will not be updatable

Agreed, but this already isn't possible on most user agents today (Chromium desktop being the exception). This seems like a reasonable tradeoff given it is inherent in the nature of data URLs. Developers can switch to an HTTP manifest URL if they want to start supporting updates, requiring a one-off re-installation.

It does break existing behavior that developers are relying upon. I would be interested in getting stats here too.

PWA stores will recognize different versions of manifest URLs as different web apps.

Depending on how PWA stores collect manifests (developer submission/user submission vs. web crawling) they may temporarily end up with duplicate records which need merging the next time a listing is updated. I agree this is the biggest disadvantage (and impacts sync services too), but I am of the view that developers should be discouraged from versioning their manifest URLs (like 92% of existing web apps), which would help reduce the impact.

Same comment above - bad for users to have this happening, and it would happen, possibly with big properties.

@benfrancis wrote:

If the manifest_url is hosted under a CDN, they will need to stick to that CDN provider.

I don't agree, they can redirect to a new one.

CDNs go out of business - it's dependency that people already rely on. Maybe it would never be an issue, but worth mentioning as a Con I think.

@benfrancis

As I stated in the doc, safari doesn't support updating/looking up installed PWAs, so it doesn't add any use cases relevant to the unique ID problem.

I don't think it's fair to exclude Safari from the analysis on this basis, given the specification doesn't currently define how to update a manifest it can't be expected for user agents to already support this. As I understand it Safari on iOS does store metadata from web app manifests when adding web apps to the home screen, so what it uses as an identifier to store those data still seems relevant. It should be assumed that Safari on iOS may support updating web apps in the future, if and when that feature is defined in the specification.

I'm guess I'm confused why you mentioned them then. Excited to help get that support prioritized though by figuring out this spec :)

@benfrancis wrote:
I'm wondering whether a better compromise between the best conceptual solution and the practical reality might be to add an id field as proposed, but make that id member an absolute URL and use processed manifest URL as the default. That would mean picking what Chrome for Android currently does as the default behaviour rather than what Chrome desktop does. It would allow start URL and scope to change without issues and allow developers who want to change the manifest URL to specify an ID inside the manifest in order to do so. Developers who don't want to change the manifest URL (but may want to change the start URL) can continue to use it as an identifier and don't need to use the optional id field at all.
....skipping some text...
If the ID is an absolute URL rather than being resolved against the document URL, could this be possible (providing the new manifest URL has the same origin as the old one)?

This is a good question - and I think this is actually the 2. global id = id (default processed manifest url). This would let someone make this manifest:

{
  ...
  id: 'https://www.google.com/manifest.json',
  start_url: "https://malware.com/step_3_profit",
  ...
}

I'm not sure how we would have this sort of malware takeover not happen. Let me know if you're thinking of something different here though.

I guess my perspective here is that, while not perfect, our proposed solution allows us to support all of the use cases, keeps all of the relevant information declaratively in the manifest & document (not needing to support redirects for edge cases, possibly forever), but sacrifices:

explicit discoverability
android webapp developer expectations (only option which doesn't change someone's expectations is the 'no default global id' one)
Migration cost (which exists for someone for all options, but at least is a one-time migration).

which, since the expectations & migration cost would happen here no matter what somewhere, the sacrifice would be discoverability. I think this is solvable by indexes & crawling, so it seems like the best solution to me. I don't think it's acceptable for us to break data urls and manifest versioning, and with some of the 301 workarounds I don't see how we could support the use cases without nasty user agent edge cases (e.g. "should I show install icon" check would need to re-fetch all existing manifests for an origin to see if any of them now redirect to the manifest listed in the html page). But maybe I'm not seeing something here.

(Ironically, this still would mean that Chromium desktop would still need migration work due to our sync implementation not ignoring 'bad' ids - but that's not really relevant, for this doc, as we're mostly focused on the use cases here.)

dmurph on 17 Feb 2021

@ralphch0 wrote:
One comment I had was regarding the default ID when the "id" field is missing:

Feels to me like if we were building this from scratch, we would likely go with scope? It's the same as service workers, and less likely to change. Sounds like the main reason start_url is preferred is because we would only be changing expectations for one set of devs (Android), but I wonder if long term scope will make more sense for future developers. Is start_url even that intuitive/expected by Desktop developers today?

start_url is how we currently allow updating, so I'm sure there is a lot of infrastructure that devs have in place that assume this case. But I don't have any concrete data here yet. @benfrancis noted that there is a request for having multiple scopes, which would make this probably a bad option if people need that.

@ralphch0 wrote:
I assume here that regardless of what we choose, already installed apps will be migrated to the new ID scheme? (i.e. on the first chrome version that has the new code, the ID'ing will be migrated to the new scheme, and updates to the app will continue working by matching using the new ID scheme). IOW, there should be no technical diff (since I assume almost all apps out there don't use a dynamically changing scope), just a change in developers mental model that they may need to be aware of.

As much as possible, yes. Some solutions would not support existing installed apps, and IDK what we would do in those cases. I think you're right - developers on any platform where the behavior is changing would have to have a change in mental model, yes.

dmurph on 17 Feb 2021

@benfrancis noted that there is a request for having multiple scopes, which would make this probably a bad option if people need that.

The plan of record in service-worker-land is to default to scope when using the legacy single scope attribute and there is no id. If you are using the new-fangled-scope attribute that supports multiple scope values, etc, then you are required to set an explicit id.

wanderview on 17 Feb 2021

This is a good question - and I think this is actually the 2. global id = id (default processed manifest url).

Yes sorry, that's the section I intended to link to. The difference being that id would be an absolute URL, rather than just a string which is concatenated with the start URL's origin.

This would let someone make this manifest:

{
...
id: 'https://www.google.com/manifest.json',
start_url: "https://malware.com/step_3_profit",
...
}

I'm not sure how we would have this sort of malware takeover not happen. Let me know if you're thinking of something different here though.

Ah, yes. That does look bad.

Just out of curiosity, and I'm sure I'm missing something here, but although this looks scary, what problems does it actually create? If the id URL is just an arbitrary URL used for identification purposes (which could be a manifest URL, or any other URL) and is never actually fetched or used for anything else, does it actually pose a security risk if a developer does something unexpected like use someone else's origin for their ID?

If a URL as an ID worked as I suggested and is only allowed to update an existing id-less app if the origin matches the original manifest URL, is it actually possible for malware.com to "take over" an app? I guess it is possible for the user to install malware.com's app first, which could then prevent them from later installing an app from google.com with a conflicting id? That does seem messy.

This does bring me back to an earlier point though - if id is a field inside the manifest what happens if two different web app manifests claim the same id? Is it not possible for user agents to get stuck in an update loop where they keep switching between two or more manifests linked from different web pages as the user happens to navigate to them? This is arguably another benefit of making the manifest URL the global identifier because that can't easily happen.

I guess my perspective here is that, while not perfect, our proposed solution allows us to support all of the use cases, keeps all of the relevant information declaratively in the manifest & document (not needing to support redirects for edge cases, possibly forever), but sacrifices:

explicit discoverability

android webapp developer expectations (only option which doesn't change someone's expectations is the 'no default global id' one)

Migration cost (which exists for someone for all options, but at least is a one-time migration).

...

I don't think it's acceptable for us to break data urls and manifest versioning, and with some of the 301 workarounds I don't see how we could support the use cases without nasty user agent edge cases (e.g. "should I show install icon" check would need to re-fetch all existing manifests for an origin to see if any of them now redirect to the manifest listed in the html page).

Losing the linkability/explicit discoverability of web app manifests does feel like a missed opportunity, as does further tying manifest processing to a document URL which limits use cases for manifests outside of a web browser (app stores, app provisioning, kiosks etc. as discussed above).

But if we really have created a situation where it's no longer possible to use manifest URL as an identifier because identity was left undefined in the specification for so long, and it's not even possible to use manifest URL as a default, then I agree with @ralphch0 that scope URL might be the next least worst option.

I used the processed scope URL as an ID for storing web apps in a Firefox OS tablet prototype and it seemed to work OK, but it was just a prototype. The thing that concerned me about it was the potential for overlapping scopes between web apps, but that's probably fine as long as two web apps don't use the exact same scope (see the comment about multiple manifests claiming the same ID above).

@wanderview wrote:

The plan of record in service-worker-land is to default to scope when using the legacy single scope attribute and there is no id. If you are using the new-fangled-scope attribute that supports multiple scope values, etc, then you are required to set an explicit id.

That's interesting, and eases my concern about multiple scopes inside a manifest. Is there a draft that we can read anywhere? Aligning manifest with Service Worker in this respect seems attractive.

The advantages of using scope as a default listed in the explainer are also quite attractive and the disadvantages boil down to it being equally inconvenient for everyone!

benfrancis on 18 Feb 2021

@benfrancis wrote:

@dmurph wrote:
This would let someone make this manifest:
{
...
id: 'https://www.google.com/manifest.json',
start_url: "https://malware.com/step_3_profit",
...
}
I'm not sure how we would have this sort of malware takeover not happen. Let me know if you're thinking of something different here though.

Ah, yes. That does look bad.

Just out of curiosity, and I'm sure I'm missing something here, but although this looks scary, what problems does it actually create? If the id URL is just an arbitrary URL used for identification purposes (which could be a manifest URL, or any other URL) and is never actually fetched or used for anything else, does it actually pose a security risk if a developer does something unexpected like use someone else's origin for their ID?

If a URL as an ID worked as I suggested and is only allowed to update an existing id-less app if the origin matches the original manifest URL, is it actually possible for malware.com to "take over" an app? I guess it is possible for the user to install malware.com's app first, which could then prevent them from later installing an app from google.com with a conflicting id? That does seem messy.

This would let malware 'take over' an app, as they would just set their id field to the manifest url. Even if we decided to 'verify' the start_url has a link to this new manifest, they could change the start_url in their bad manifest & it would look valid.

This does bring me back to an earlier point though - if id is a field inside the manifest what happens if two different web app manifests claim the same id? Is it not possible for user agents to get stuck in an update loop where they keep switching between two or more manifests linked from different web pages as the user happens to navigate to them? This is arguably another benefit of making the manifest URL the global identifier because that can't easily happen.

I think the real 'trusted' aspect here is the <link rel="manifest" href="...relative or absolute url here..."> part. This is, in a sense, the source of truth here, and I think we would treat the manifest linked from the start_url as the authoritative one. I think would have to be the same for all solutions here, even manifest_url - stores and browsers would verify that the start_url's manifest link actually links to the given manifest. If it doesn't match, then they would use the one in the rel-link.

@wanderview wrote:

The plan of record in service-worker-land is to default to scope when using the legacy single scope attribute and there is no id. If you are using the new-fangled-scope attribute that supports multiple scope values, etc, then you are required to set an explicit id.

That's interesting, and eases my concern about multiple scopes inside a manifest. Is there a draft that we can read anywhere? Aligning manifest with Service Worker in this respect seems attractive.

The advantages of using scope as a default listed in the explainer are also quite attractive and the disadvantages boil down to it being equally inconvenient for everyone!

Yeah, and most importantly here we would be breaking the expectations / infrastructure of all WebApp devs, as they are relying on either start_url or manifest_urlimplementations, instead of just one set of WebApp devs. Hard to say if it will continue to "just work" for most of them.

dmurph on 18 Feb 2021

Also, I see the id as becoming recommended, even to the point of us eventually requiring it for installability. Once everyone has an id set, then the world becomes a whole lot simpler. These defaults basically just help existing apps update to the new id world.

dmurph on 18 Feb 2021

This does bring me back to an earlier point though - if id is a field inside the manifest what happens if two different web app manifests claim the same id? Is it not possible for user agents to get stuck in an update loop where they keep switching between two or more manifests linked from different web pages as the user happens to navigate to them? This is arguably another benefit of making the manifest URL the global identifier because that can't easily happen.

That's the reason we force the id to prefix with the start_url's origin. So if within the same origin, they serve multiple manifests with the same id , yeah they will be recognized as the same app. But I think that should be expected by the app developers for that site as this is spelled out in the spec.

This does bring more work for PWA stores that do web scrawls without curation - they need to verify that the document of the start_url links to the manifest_url, if the manifest is at a different origin. I think this is the right thing to do though. Also I don't know which PWA store scrawl the web without any validation process right now.

Also, I see the id as becoming recommended, even to the point of us eventually requiring it for installability. Once everyone has an id set, then the world becomes a whole lot simpler. These defaults basically just help existing apps update to the new id world.

+1. I think with id added to manifest, this should be the recommended way. And the default is just a fallback mechanism that's optimized for backward compatibility. With scope it doesn't really give us backward compatibility benefit. With manifest_url, for cross-origin and data url cases, also doesn't allow backward compatibility.

philloooo on 18 Feb 2021

Do you think it's safe to assume that all Chromium-based user agents behave the same way as Chrome on desktop and on Android? (Genuine question as outside of the implementations I've worked on myself I don't know). Due to the interplay between the browser engine, browser chrome, underlying operating system and sometimes app store and sync service, evaluating support for the Web App Manifest specification is more complicated than for other web specifications which are implemented entirely inside a browser engine. Chromium-based user agents could have different browser chrome which implements add to home functionality differently, or use their own sync service or app store (Webian Shell for example is built on a desktop build of Chromium but uses manifest URL to identify web apps). Operating systems may also try to update apps independently of the browser they were installed from (I'm not sure whether this is currently the case).

The code for web app id is deeply embedded in Chromium in various places. Technically their forked version can change all the places to use a different id mechanism, but I will be surprised if they do that. Although I only know for sure Edge desktop doesn't deviate from Chromium for this feature.

I updated the explainer with Firefox Android behavior based on information from @NotWoods . They support updating manifest in newer firefox and android versions and use start_url as the id.

philloooo on 18 Feb 2021

@dmurph wrote:

This would let malware 'take over' an app, as they would just set their id field to the manifest url.

Ah yes I see that's what I was missing, so an absolute URL wouldn't work.

@philloooo wrote:

That's the reason we force the id to prefix with the start_url's origin. So if within the same origin, they serve multiple manifests with the same id , yeah they will be recognized as the same app. But I think that should be expected by the app developers for that site as this is spelled out in the spec.

Yes I agree that at least contains the problem to a single origin.

The code for web app id is deeply embedded in Chromium in various places.

Which I imagine also makes changing the current behaviour to a different solution quite tricky.

I updated the explainer with Firefox Android behavior

Thank you! Don't forget KaiOS browser.

OK, well I came here to offer my feedback, based on experience of implementing the specification a few times, that there's no need to add an id member to the manifest because manifests already have a natural universal identifier on the web which makes them directly linkable and discoverable and which de-references to a useful resource. It's unfortunate that we've gone so many years with no identifier being defined in the specification that it appears the obvious solution is no longer practical.

If there's now no option but to define an id member inside the manifest, then my recommendation is that if manifest URL can not be a default then scope is the next least worst option in the long term. If that makes manifest consistent with service workers then that's even better.

I wouldn't expect using scope as a default identifier to cause much breakage for existing applications because it is probably the most stable of all the available options inside the manifest, but I don't have any data available to me to validate that hypothesis. Start URL has always been a dubious choice as an identifier for a web application as it's the most likely to change, so it's unfortunate in my opinion that the Chrome team chose that option in the absence of a recommendation in the specification.

I also wouldn't expect breaking "implicit expectations" of developers to be a big problem because in practice developers don't just target a single browser, they are already targeting a range of desktop and mobile browsers which behave inconsistently. It may even fix problems they were previously having with changing start URL or manifest URL, without any action on their part.

As an independent Invited Expert I can only offer my feedback, I don't have a large user base behind me to give that feedback much leverage. So as long as Mozilla, Apple and KaiOS Technologies are not participating in the discussion, I expect the Chrome team will do what they believe is best for their users, and developers.

Thank you @philloooo and @dmurph for taking the time to explain your rationale.

benfrancis on 18 Feb 2021

thanks benfrancis, do you know if KaiOS support updating installed web apps?

philloooo on 18 Feb 2021

@wanderview wrote:

The plan of record in service-worker-land is to default to scope when using the legacy single scope attribute and there is no id. If you are using the new-fangled-scope attribute that supports multiple scope values, etc, then you are required to set an explicit id.

That's interesting, and eases my concern about multiple scopes inside a manifest. Is there a draft that we can read anywhere? Aligning manifest with Service Worker in this respect seems attractive.

I just want to highlight that in service-worker-land there is nothing like start_url, etc, to use as an id. So falling back to scope as the default id is mainly because scope is the identifier today (since there is nothing else unique to use). Since the fallback is for compatibility, it probably makes sense to fallback to whatever is currently being used as the id. In this case it sounds like at least two browsers use start_url for that (if I have followed correctly).

wanderview on 18 Feb 2021

thanks benfrancis, do you know if KaiOS support updating installed web apps?

Yes we do. I need a bit more time to digest this thread, but I plan to give feedback. Sorry for the delay!

fabricedesre on 18 Feb 2021

👍1

@fabricedesre 🙏 thank you! Excited to hear your thoughts

dmurph on 19 Feb 2021

@wanderview wrote:

I just want to highlight that in service-worker-land there is nothing like start_url, etc, to use as an id. So falling back to scope as the default id is mainly because scope is the identifier today (since there is nothing else unique to use). Since the fallback is for compatibility, it probably makes sense to fallback to whatever is currently being used as the id. In this case it sounds like at least two browsers use start_url for that (if I have followed correctly).

Yes, and by my count there are currently about 10 user agents which use manifest URL, which isn't really reflected in the explainer.

benfrancis on 19 Feb 2021

@benfrancis wrote:
Yes, and by my count there are currently about 10 user agents which use manifest URL, which isn't really reflected in the explainer.

Just to close the loop, I think we end up with this list:

Android
- Chromium (please correct me if any of these are forks of something else)
- Chrome
- Samsung
- Edge
- Opera
- Brave
- UC Browser
- QQ Browser
- Huawei Browser
- Baidu
- (etc)

And then I believe the following use start_url:

Android
- Firefox
Desktop
- Chromium
  - Chrome
  - Edge
  - Opera
  - Brave
  - UC Browser
  - QQ Browser
  - (etc)
- ChromeOS
(previously Firefox, WebApps not supported anymore)

And the following don't really have a unique id for their manifest:

Safari iOS

WebApps not supported:

Safari Desktop
Firefox Desktop

Unknown:

KaiOS

You're correct - we didn't expand on the Chromium forks for Android or Desktop, instead trying to focus on different user agent implementations - Chromium, Firefox, Safari.

dmurph on 22 Feb 2021

@fabricedesre Gentle ping on feedback here, do you think you could take a look by the end of the week?

dmurph on 25 Feb 2021

@dmurph yes I'll answer tomorrow.

fabricedesre on 25 Feb 2021

For KaiOS, the currently released devices uses the Open Web Apps (OWA) manifest from Firefox OS, while the new upcoming major release will use PWA manifests.

Let me talk a bit about the history here - keep in mind this work started in 2011 :)
In OWA, we always used the manifest url as the application id. This is explicit in the API itself, where many operations take the manifest url as the only parameter. I think overall it worked well because this was a very simple model. For instance this allows to manage app updates in a straightforward way: fetch the new manifest, check with etag and/or hash if it changed, and apply the update after verification that the app name hasn't changed to prevent spoofing.
We enforce that start_url is relative to the manifest url, and we don't care about the document url or origin. There is no scope either (we added something similar later for KaiOS though). We partitioned cookies and storage per manifest url, providing something close to the current Firefox containers.

When work started on PWA manifests, I distinctly remember that a disagreement between the Mozilla and Google teams was about how to identify apps and the whole document origin/manifest origin/start url processing. To be honest I still don't understand why the document origin should have a role to play here. We felt that a lot of complexity with little justification was added to the spec. It's good to see that now there is a lot more data from the field to help make decisions.

The new PWA based implementation we do for KaiOS keeps the manifest url as the app id, and in this regard is similar to Chrome for Android. Because of the form factor of our devices (non touch, small screens), we only provide store initiated installs, which means that we have a bit more control over the manifests that are installable. However I find the list of scenari quite interesting for when we'll offer browser initiated installs. I'm not sure I agree that all of them should be supported, but the intent is valid.
We don't support data: urls for manifests, and have no plans to change that. It looks like instead a service worker could generate them with a "real" url?

Out of all the proposals in the explainer, I could live with the first one "start_url_origin + specified id". I feel this is reasonable since that doesn't introduce a new global identifier namespace, and provides a fallback for backward compatibility. Ideally, I would use "manifest origin + specified id" instead, but that prevents support of data: urls for manifests.

Kind of a side question, but do you have data on how often start url has a different origin than the manifest url?

fabricedesre on 26 Feb 2021

👍2

@fabricedesre wrote:
When work started on PWA manifests, I distinctly remember that a disagreement between the Mozilla and Google teams was about how to identify apps and the whole document origin/manifest origin/start url processing. To be honest I still don't understand why the document origin should have a role to play here. We felt that a lot of complexity with little justification was added to the spec. It's good to see that now there is a lot more data from the field to help make decisions.

This was before my time :)

I kind of see the 'rel' link as the source of truth here - if the start_url document has a <link rel="manifest" href="manifest.json"> and that manifest is the one we are using, then we are 💯 . For example, if I was running a web app directory and someone submitted a manifest, then I would want to check that the document at start_url had a rel link back to that manifest to verify it. Maybe this doesn't really matter though.

The new PWA based implementation we do for KaiOS keeps the manifest url as the app id, and in this regard is similar to Chrome for Android. Because of the form factor of our devices (non touch, small screens), we only provide store initiated installs, which means that we have a bit more control over the manifests that are installable. However I find the list of scenari quite interesting for when we'll offer browser initiated installs. I'm not sure I agree that all of them should be supported, but the intent is valid.

I think this is also a similar model to how trusted web activities work in the Google Play store - since it is a curated model where the developer uploads their app.

We don't support data: urls for manifests, and have no plans to change that. It looks like instead a service worker could generate them with a "real" url?

Yes, but not without developers changing their code / infrastructure :(

Out of all the proposals in the explainer, I could live with the first one "start_url_origin + specified id". I feel this is reasonable since that doesn't introduce a new global identifier namespace, and provides a fallback for backward compatibility. Ideally, I would use "manifest origin + specified id" instead, but that prevents support of data: urls for manifests.

Kind of a side question, but do you have data on how often start url has a different origin than the manifest url?

Surprisingly I see none in the pwa-directory data, which is weird, as this used to be in there (see the comment above), and it still has a different origin manifest:
https://www.spokeo.com
There's a chance that these maybe got filtered out somehow?

Code I tried to find manifests that are not same origin as start_url (warning: I'm bad at jq):

curl -sSL 'https://pwa-directory.appspot.com/api/pwa/?limit=4200' | jq -r '.[] | (.manifestUrl | split("/")[2]) + " " + (.absoluteStartUrl | split("/")[2])' | awk '$1 != $2'

Maybe someone else can find another directory we can query, or other examples

dmurph on 26 Feb 2021

I just want to highlight that in service-worker-land there is nothing like start_url, etc, to use as an id. So falling back to scope as the default id is mainly because scope is the identifier today (since there is nothing else unique to use). Since the fallback is for compatibility, it probably makes sense to fallback to whatever is currently being used as the id. In this case it sounds like at least two browsers use start_url for that (if I have followed correctly).

Just a quick note here: I think it's definitely an advantage to try to reduce pain by making the fallback work for compatibility. But if feels scope is likely to achieve this also to a large extent, since it's likely the most stable field (maybe we can get numbers here?). The only thing that's meaningfully changing is the mental model for desktop devs (it's unclear if expectations are even well formed for most devs).

If we can make the id required at some point, as @dmurph suggested, then the non-intuitive fallback is limited to the short term. Is this likely though? Would we need to build some sort of allowlist of existing sites to exclude from the requirement, or have a migration period after which we will start breaking new installations?

If this fallback is likely stay forever, and we think that there are a lot more PWAs to build in the future than we have today, then it might make sense to pick a fallback that makes sense in the future.

That said, this is a slight preference from a design point of view and consistency with service workers. It won't meaningfully affect our own apps, if it's decided to go ahead with the current proposal for fallback.

ralphch0 on 4 Mar 2021

Hi ralphch0, I think making the id required at some point can be achieved by

have a warning in the lighthouse about the new field and a date when this becomes required.
make it required for the install icon to show up. The app can still be installed via shortcut if it's not installable.

I think it's quite reasonable to approach it this way, since we already have a model of constantly evolving the installabillity criteria to higher standard.

philloooo on 16 Mar 2021

Hi, thanks for all the feedback!
After going through & incorporating the feedback here to the explainer, I believe the current proposal (global_id = start_url_origin + manifest_id_member, default global_id = start_url) is the one that supports the use cases the best and allows the smoothest transitions from the existing ecosystem. I am moving forward with implementation on Chromium side.
You can track the status of the implementation on https://www.chromestatus.com/feature/6064014410907648

philloooo on 17 Mar 2021

Thanks for the Chrome update @philloooo. The above turned into a bit of a monster thread... can someone give me a tl;dr of what the actual id is? Is it a URL or is it a UUID or something else? (i.e., what a some of the processing rules)

marcoscaceres on 17 Mar 2021

Good question! I think it's easiest to think of these two categories of web apps:

If you are creating an app from scratch (no existing installs to update)

id is just a string, like a UUID, that uniquely identifies the app on the start_url_origin

(at https://www.new-app.com/):

{
  ...
  id: "NewAppId",
  start_url: "/index.html", // <- updatable now :)
  ...
}

The global id evaluates to https://www.example.com/ + NewAppId = https://www.example.com/NewAppId. This is not intended to be evaluate-able, but looks like a URL.

If you have an existing app (installs exist where the manifest did NOT have an `id` set)

id should be the relative path of the start_url so that the new manifest (with id specified) will apply, and thus update, the old manifest. EX:

old (at https://www.example.com/):

{
  ...
  start_url: "/index.html"
  ...
}

The default global id is: https://www.example.com/index.html

new:

{
  ...
  id: "index.html",
  start_url: "/index.html", // <- now this is updatable!
  ...
}

The global id evaluates to https://www.example.com/ + index.html = https://www.example.com/index.html, thus matching the old manifest.

Hopefully that makes sense?

(example of updating the start_url of the app)

{
  ...
  id: "index.html",
  start_url: "/nested/index.html",
  scope: "/nested/",
  ...
}

dmurph on 17 Mar 2021

👍1

Ok cool. Thanks for the clear explanation, @dmurph! It seems pretty straight forward to implement/spec. Should we give it a few months before adding this based on Chrome's rollout? Then at least we can be assured it works ok and we've not overlooked, or at least we hear about, any edge cases.

marcoscaceres on 18 Mar 2021

I think that sounds good. I imagine we'll make a pull request once we have a working / testable implementation.

dmurph on 18 Mar 2021

I'm happy to draft up the pull request based on the above. No worries.

marcoscaceres on 19 Mar 2021

FYI @philloooo is very interested in writing the pull request here when we get to that stage to get some good spec experience 😄

dmurph on 24 Mar 2021

@philloooo, that's great to hear. If you need any assistance or have any questions, feel to reach out. Happy to help!

marcoscaceres on 30 Mar 2021

I thought i would through my 2 cents in:
I use localforage to create a db and place a unique id on the user's system. Im using incremental values and once Ive added a unique id to the users system/browser i update the "last_unique_id table" in my mysql database.
I'm sure you could make some use of this method..

localforage will start with indexedDB and if its not available will move though a list of other database formats until it can land the unique id..

Hope this can help you..

PS: this a persistent storage method

antonyf on 9 Apr 2021

Manifest: Add a unique identifier for a PWA

Most helpful comment

All 121 comments

If you are creating an app from scratch (no existing installs to update)

If you have an existing app (installs exist where the manifest did NOT have an `id` set)

Related issues

Manifest: Add a unique identifier for a PWA

Most helpful comment

All 121 comments

If you are creating an app from scratch (no existing installs to update)

If you have an existing app (installs exist where the manifest did NOT have an id set)

Related issues

If you have an existing app (installs exist where the manifest did NOT have an `id` set)