Manifest: Allow for multiple scopes

[RobDolinMS]

There are some cases where a web app could have multiple scopes.

• In one case, a web app (scope:Example.com) might have a CDN (scope:ExampleCDN.com), personal data (scope:MyExample.com), etc. that is served from a second domain.
• In another case, a web app (scope:Chat.Example.com) might have peer subdomains that are in-scope (scope:Music.Example.com and scope:Video.Example.com) but might also have peer domains that are out-of-scope (scope:Events.Example.com)

PROPOSAL: Add an “additional_scopes” array element for web app developers to optionally specify additional scopes
FAQ:
Q: Why not replace the current scope object with a scopes array?
A: We do not want to break backwards compatibility with implemented manifests that use scope
Q: What might this look like?
A:
"scope": ... ,
"additional_scopes": [ ... ],

[kenchris]

If we call them secondary_scopes it makes it clearer to devs that there might be limitations

[benfrancis]

Scope is complex. I did a bit of research into this before the current scope property was defined and explored some potential solutions https://docs.google.com/document/d/1fOsQWOOVuKyqO7cXZoKmxZGQ9FLgLMwmCRw3OEqIKrQ/edit?usp=sharing

The current scope property is the simplest possible solution but isn't backwards compatible with the URL structures of all existing websites. You could argue that the current solution is too simplistic and inflexible, or you could argue that we should encourage simpler URL structures for modern web apps and restrict the scope of a manifest to a single origin.

[kenchris]

Could these things be made more safe by having the Service Worker handle these urls via foreign fetch? (sorry my understanding of foreign fetch isn't really there yet)

[marcoscaceres]

The problem this tries to solve is having the manifest continue to be applied as you go from a.com to b.com - so foreign fetch could help there, but it's still a huge security issue. There needs to be some kind of overarching authority that proves the continuity from a.com to b.com is safe.

[marcoscaceres]

So, it seems we might be able to use the Storage spec's "site" concept to solve some of this. Where site:

A site is eTLD+1.

which maps to: *.whatever.com

[marcoscaceres]

Gave this more thought on the weekend, and site feels right, but it conflicts with scope. They are mutually exclusive.

That is, if you declare a scope, you can't declare a site. But if you declare a site, it invalidates your scope (because scope can only apply to one origin).

The might be ok, because it covers the most basic cases:

1. "I own example.com", and I don't care where you go within my domain ("unbounded"). If you leave my domain, the browser "punishes" you.
2. "I own example.com", but I only want my app to be in "/foo": use scope.
3. "I own foo.example.com, bar.example.com, *.example.com", and I want to jump from one sub domain to the other. Use: site.

The highest priority case to solve is the multiple "site" problem, but in a way that is not reckless with cross-origin (i.e., I think we should forget about foo.com and bar.com, and instead only allow *.foo.com via "site").

@boyofgreen, @RobDolinMS, @kenchris, @benfrancis, wdyt?

[marcoscaceres]

As a quick refresher: site === eTLD+1.

[kenchris]

I agree that we can concentrate on the eTLD+1 use-case instead of foo.com+bar.com like scenarios.

That said, I think it is confusing to have something replace something else and I am not sure people will get that. We don't want to end up with something like density again here. Also, people might want to scope each of these sites

Why not just allow scope to be an array, but that any entry must be eTLD+1+scope?

[marcoscaceres]

That said, I think it is confusing to have something replace something else and I am not sure people will get that. We don't want to end up with something like density again here. Also, people might want to scope each of these sites.

Scoping per site would be messy - we should try to avoid that (below).

Why not just allow scope to be an array, but that any entry must be eTLD+1+scope?

Yeah, something like that might work.... but it's still super messy. Like:

{
  "scope": [
    "foo.com/bar"
   ]
}

Means all of these are in scope:

• sub.foo.com/bar,
• dub.foo.com/barista
• and so on...

So then you get a mess of:

{
  "scope": [
    "foo.com/bar",
    "foo.com/app"
   ]
}

Where now everything matching anything *.foo.com/bar* and *.foo.com/app* is scope and it's hard to reason about.

[benfrancis]

Some thoughts:

• eTLD is a pain for implementers because it requires a suffix list like https://publicsuffix.org/ (although this is already needed for other browser features anyway)
• Site _would_ allow for the use case of a mail inbox at mail.google.com with a login page at accounts.google.com or an inbox at snt152.mail.live.com with a login page at login.live.com
• A single site _wouldn't_ accommodate the use case of an inbox at mail.google.com with a login page at accounts.google.com but an unrelated app at calendar.google.com
• Site still _wouldn't_ accommodate the use case of youtube.com with a login page at accounts.google.com or a search at s.taobao.com with a results page at detail.tmall.com/item.htm
• Site + scope still _wouldn't_ accommodate the use case of a web app at www.linkedin.com with an unrelated page at www.linkedin.com/company which shouldn't be within the scope of the manifest
• Site + scope still _wouldn't_ accommodate the use case of an informational web page at www.evernote.com and an app at www.evernote.com/Home.action

It's true that using "site" could cater for a few additional use cases, but its interaction with scope seems a little unpredictable and clumsy.

I suggested some alternative solutions here https://docs.google.com/document/d/1fOsQWOOVuKyqO7cXZoKmxZGQ9FLgLMwmCRw3OEqIKrQ/edit#

Basically what we have now:

{
  "start_url": "/foo",
  "scope": "/foo"
}

A more comprehensive solution (still doesn't cover multiple deep-linkable origins):

{
  "start_url”: “http://foo.com/",
  "scope": {
    "include": ["/foo", "/bar"],
    "exclude": ["/baz", "/qux"]
  },
  "stay_in_app": ["http://norf.com"]
}

A middle ground:

{
  "start_url": "http://foo.com/bar",
  "scope": ["/bar", "/baz"],
  "stay_in_app": ["http://qux.com"]
}

Another approach might be an array of scopes which can span multiple origins but only within the same eTLD.

{
  "start_url": "http://foo.com/bar",
  "scope": [ "http://foo.com/bar", "http://baz.foo.com"]
}

Of course there's no guarantee that because someone has control over one origin in an eTLD that they own them all. Do we want evil.github.io capturing scope for w3c.github.io?

[boyofgreen]

This is similar to what we have implemented in Manifoldjs, we have an "extended_scope" value, which we feel makes more sense for scope:

{
   "start_url":"https://www.myapp.com",
   "scope":"https://www.myapp.com"
   "mjs_extended_scope":["http://www.myapp.com","http://www.myappauth.com"]
}

https://github.com/manifoldjs/ManifoldJS/wiki/Using%20Extended%20Scope

[RobDolinMS]

What if we used @benfrancis's idea of stay_in_app as a way to solve for @boyofgreen's use case?

{
   "start_url":"https://www.myapp.com",
   "scope":"https://www.myapp.com"
   "stay_in_app":["https://www.myapp.com","https://www.myappauth.com"]
}

/cc @marcoscaceres @kenchris

[kenchris]

PWAs currently are only allowed for HTTPS (because of service worker and because they may hide UI like the URL). For this reason I don't think we should allow any HTTP-only to be considered part of the app.

[boyofgreen]

I think that would work for most of the use cases we see. it also solves for us not rocking the scope boat :)

[kenchris]

I am OK with extended scope, as long as we are talking about "https". We could do something like Chrome Custom tabs for "http" though, instead of opening in the browser.

[benfrancis]

PWAs currently are only allowed for HTTPS

It's true that Chrome won't prompt you to add your web app to the homescreen if it's not using HTTPS, but that doesn't mean you can't use a web app manifest with HTTP. Even in Chrome I believe the user can still manually add to homescreen. There's nothing to stop user agents supporting web app manifests with HTTP (and scope with HTTP) even though Service Workers require HTTPS.

If stay_in_app is an array then should scope be allowed to be an array too? To allow multiple scope paths, at least within the same origin? I don't know, the problem with these more complex cases is that it gets messy pretty quick.

There's definitely a clear use case for stay_in_app though, with web app authentication which uses a separate domain (e.g. Google Accounts).

[RobDolinMS]

Thanks @kenchris and @benfrancis. I've updated my example above to use HTTPS as my intent was to focus on additional scopes via a stay_in_app object and will open a separate issue to track possible support for HTTP.

[benfrancis]

I don't think you need to do anything extra to "support" HTTP. It kinda just works ;)

[RobDolinMS]

@marcoscaceres Would you want to draft a PR? Would you like me to?

[marcoscaceres]

We need to write an email to the Web App Sec WG outlining the problem and maybe a solution. This impacts too many core web security things that I would be reluctant to include anything in our spec that is not first blessed by them.

On 22 Sep. 2016, at 4:55 am, Rob Dolin (MSFT) [email protected] wrote:

@marcoscaceres Would you want to draft a PR? Would you like me to?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[RobDolinMS]

Below are my quick thoughts on an email to the Web App Sec WG. I'll clean this up based on feedback. Thanks--
--Rob

TO: Web App Sec WG
FR: Rob Dolin

Dear Web App Security WG members,

I'm writing on behalf of a number of W3C members working on the W3C Web App Manifest spec. (http://www.w3.org/TR/appmanifest/)

We have two use cases where web app developers want to indicate a web app includes multiple domains.

• In one case, a web app (scope:Example.com) might have a CDN (scope:ExampleCDN.com), personal data (scope:MyExample.com), etc. that is served from a second domain.
• In another case, a web app (scope:Chat.Example.com) might have peer subdomains that are in-scope (scope:Music.Example.com and scope:Video.Example.com) but might also have peer domains that are out-of-scope (scope:Events.Example.com)

We are considering a proposal (https://github.com/w3c/manifest/issues/449) where the W3C Web App Manifest file (a JSON file residing on the primary domain) be able to indicate other (HTTPS-only) domains that could be rendered within the context of the web app rather than being rendered in the context of web browser chrome.

We would appreciate any questions or suggestions you have as we consider this proposal.

Thanks very much--
--Rob

[RobDolinMS]

@marcoscaceres @benfrancis @kenchris @boyofgreen - Any feedback on the draft email to Web App Sec WG or looks OK to send?

[marcoscaceres]

@RobDolinMS, I think it's ok to send to kick off the discussion. If possible, you might want to include some real websites that exhibit this need. @boyofgreen, do you recall some? I've found some in the past.

[delapuente]

I like the stay_in_app idea too but I would rename it to stay_in_context.

[Huxpro]

Loving the stay_in_app idea whatever the name is.
Loving the principle that do not break backwards compatibility.
Loving the idea of Chrome Custom tabs for "http" (or any URL from unbounded PWA IMO).

FWIW I really want to introduce the inheritance power to manifest cuz it solve many problems mentioned by @benfrancis well. Please checkout #539

[kenchris]

This is my suggestion. The additional scopes would somehow need to whitelist the manifest who is referring to them.

1. We want some kind of whitelisting
2. We want to be able to do "add to home screen" from the additional scopes and use the info of the main manifest.json

One way could be

In https://mysite.com/manifest.json, have:

"includes": ["https://otherofmysites.com/manifest.json"]

Then in https://otherofmysites.com/manifest.json, have:

"owner": "https://mysite.com/manifest.json",
"scope": "/"

Adding "owner" to a manifest means that it will refer to another manifest file as the data source for adding to homescreen, and only a few of the properties will be used (ie "scope" for now).

[hisuwh]

I would like to add a scenario to this which hasn't been mentioned. I am using an external provider for authentication (specifically Azure AD) so would like to stay in app when redirecting users to this site.

Currently the UX is horrible as Android opens up the pseudo browser when directing you to Azure, then you stay within this pseudo browser when redirected back, rather than going back into the App view.

[dominickng]

@hisuwh that sounds like a bug. Can you please file a repro case and screenshots on crbug.com/new? I was under the impression that when you are redirected back to your app, it should return to the app view.

More generally, adding a domain which you don't own but are using for authentication is exactly the sort of thing where we should be showing the user that they are entering their credentials on a site outside of the control of the app which they installed.

[marcoscaceres]

Could the "web authentication API" (or is it credential management API?... so confused, sorry!) help here at all? Isn't that supposed to cross origin handle authentication.

[MadSpindel]

I have the same problem as @hisuwh. Using Safari PWA with Azure AD breaks the PWA. The PWA opens a new Safari Browser (for authentication) and stays in it, making the PWA useless.

I need to find a solution to this problem.

[bpair]

I have the same problem as @hisuwh and @MadSpindel. I am using Auth0 for authentication, which then may forward to our SSO server for entering in username and password.

[rnnyrk]

Exact same issue as mentioned before. Using Keycloack for authentication which results in opening a new Safari window on iOS. Any workarounds?

[JudahGabriel]

Chiming in with a +1: the lack of multiple scopes is blocking our corporate PWAs from being installed.

Our PWA uses single-sign-on (SSO) for authentication in our corporate network. In a normal web page, our web app redirects to the SSO domain, does the login, and sends us back a token.

But as an _installed_ PWA, our app launches in its own window, but browsers like Chrome and mobile Safari spawn a new _separate_ browser window for our redirect to the SSO domain. And that separate browser doesn't send back the SSO token to our original PWA window. This busts the whole thing and prevents our app from working as an installed PWA.

I see I'm not alone: this study shows many of the top 20 websites require a different domain for login.

Any of the proposed solutions here would work: scopes being an array, or having a "stay_in_app" setting that lets us specify which URLs to load inside the app.

[g-ortuno]

I don't think you multiple scopes for your use case anymore.

Until recently the Manifest spec forced browsers to open out-of-scope navigations in new top-level browsing contexts. The spec recently changed to allow these navigations to happen in the same context since it broke cases like yours. This was implemented on Chrome for Android a while ago and should be included in Chrome 71 for desktop.

[mgiuca]

Thanks @g-ortuno ... specifically the change was #701. That should address all of the "off-origin authentication" issues. We shouldn't add multiple scopes for this (we specifically don't want to show an off-origin page without the user agent presenting an origin display box). If we do add multiple scopes, they will likely need to all be on the same origin (otherwise that introduces a bunch more complexity and security issues).

[JudahGabriel]

Thanks! Looking forward to seeing this in Chrome 71! I'll be re-enabling PWA installation for our business apps once it's rolled out.

[waelkdouh]

Thanks Everyone on the great insight! Does anyone know if Safari is planning on implementing this on IOS anytime soon? I noticed that everyone is pointing to the fact that chrome is fixing it but what about Safari?

[gnihi]

@waelkdouh: It seems like IOS 12.2 finally brings external link management among other changes for PWAs -> link.

[RichardNeill]

Can we please also have the ability for multiple apps to coexist within the same scope.
Consider a "swiss-army-knife" or "multitool" type app, where there can be multiple different functionalities, but sharing the same underlying data-structures, login pages, etc.

There is definitely some value in being able to install multiple different versions of the app, which should each have different icons and go to different start points (but within the same directory).
As an example, a shop might want to offer different versions of the add-to-homescreen icon, depending on whether the user prefers Groceries or Clothing.

Currently, it's possible to "misuse" the spec under Desktop Chrome, but not in the Mobile version.

[mgiuca]

I think this falls under the category of expanding the scope matching syntax, which Ben Kelly is exploring on wanderview/service-worker-scope-pattern-matching. Let's close this out now and assume it'll be addressed by the expanded scope pattern syntax.