Fetch: Allow * for Access-Control-Allow-Headers and Access-Control-Allow-Methods

[roryhewitt]

This has almost certainly been discussed before, but would it be possible to allow * (allow-all) as a separate value for the Access-Control-Allow-Headers CORS response header?

This would allow _all_ non-simple headers passed in the request to be added to the browser's preflight cache. This si currently possible by simply mirroring back the value of the Access-Control-Request-Headers request header, but this would be much simpler. The browser would need to track the request headers passed and add them all to their preflight cache (rather than simply parse them out from the Access-Control-Allow-Headers response header, assuming that's what they currently do), but that's not too hard to do.

So the spec would become the following:

Access-Control-Allow-Headers = "Access-Control-Allow-Headers" ":" #field-name | "*"

[annevk]

Implementers have traditionally been wary about doing this, since it could be a footgun. Servers might have forgotten about side effects of certain request headers, etc.

@sicking @tyoshino?

[sicking]

I'd have to defer to the mozilla security team.

[annevk]

@dveditz @bifurcation @freddyb?

[craigfrancis]

Just as a note, it is possible to use:

Access-Control-Allow-Origin: *

But .withCredentials = true on Chrome 49 will respond with:

XMLHttpRequest cannot load http://... A wildcard '*' cannot be 
used in the 'Access-Control-Allow-Origin' header when the 
credentials flag is true. Origin 'http://...' is therefore not
allowed access. The credentials mode of an XMLHttpRequest 
is controlled by the withCredentials attribute.

And Firefox 45:

Cross-Origin Request Blocked: The Same Origin Policy disallows 
reading the remote resource at http://... (Reason: CORS header
'Access-Control-Allow-Origin' does not match '*').

---

This is because we already have developers suggesting adding this header via the Apache config:

<VirtualHost ...>
    Header set Access-Control-Allow-Origin: "*"
</VirtualHost>

https://www.google.co.uk/search?q=%22Header+set+Access-Control-Allow-Origin%22

Which would mean that a Simple CORS request (e.g. from a malicious website), could get content from the victim website (e.g. get the members profile page, and if the user is logged in, we can now retrieve their details, and maybe a CSRF token as well).

---

So maybe we do allow wildcards in:

Access-Control-Allow-Headers: *
Access-Control-Allow-Methods: *

But like Origin, don't allow the wildcard when .withCredentials = true.

[mozfreddyb]

Thanks, @craigfrancis, I concur.
If we allow this (of which I'm not completely convinced yet), we should disallow credentials in the wildcard case. Of course, we should also disallow _forbidden headers_.

[roryhewitt]

Hey Anne,

Feetgun (footguns?) are bad, but if a security mechanism is too complex or
difficult to implement, people will continue to use JSONP and its ilk.

We can't protect everyone, but we can protect many, by making CORS simpler
to implement.

FWIW, there are many examples out on the internet (some good, some bad) of
CORS implementations, but there is no 'reference implementation'. Perhaps
that's something that WHATWG could consider providing?

[annevk]

@craigfrancis @mozfreddyb if you use credentials you need to include another header too, so that example wouldn't magically pass the CORS check unless you also included that other header.

@roryhewitt do use credentials with CORS? FWIW, I don't really disagree with your position, but as I mentioned elsewhere I'm not really the gatekeeper here. As for reference implementations, I'm not sure we have the bandwidth to maintain that, or are you volunteering? I'm personally also a little wary of writing both the specification and implementation as that leads to tunnel vision issues.

[craigfrancis]

@annevk, yep, the example was just to show the similarities for those 3 headers, and how we already have a wildcard on Access-Control-Allow-Origin :-)

As to specification and implementation, I'm not in a position to do this either.

[roryhewitt]

@craigfrancis, One thing I suspect is that users like to be able to specify

Access-Control-Allow-Origin: *

since it means that they don't need to worry about supplying the Vary: Origin header (for correct browser/proxy caching). So maybe in their code, they include a check for Origin, and if it's on their 'safe' list, they respond with:

Access-Control-Allow-Origin: *

rather than

Access-Control-Allow-Origin: http://www.goodguy.com

[roryhewitt]

@mozfreddyb:

we should disallow credentials in the wildcard case

Are you saying that if Access-Control-Allow-Headers: * is specified for the request, then the server cannot include Access-Control-Allow-Credentials: true in the response? Or that if Access-Control-Allow-Credentials: true is specified in the response, then the browser must throw an error?

I assumed that forbidden headers would be disallowed anyway :) Sorry if that wasn't clear.

To clarify, I would update https://fetch.spec.whatwg.org/#cors-preflight-fetch to have something like the following:

5.7.7.2: Let _headerNames_ be the result of parsing Access-Control-Allow-Headers in response's header list. If Access-Control-Allow-Headers was passed with a value of *, set the value of _headerNames_ to *.

5.7.7.6: If _headerNames_ is not set to * and if one of request's _header list' names_ is not in _headerNames_ and its corresponding header is not a simple header or if , return a network error.

Does that make sense?

[roryhewitt]

@annevk, I'm more than willing to try my hand at creating a reference implementation, although as you say, tunnel-vision may occur.

I was thinking more of providing some examples of good/bad implementations, including possible server responses.

[craigfrancis]

@roryhewitt, in summary, don't worry about the security side when adding wildcard support to:

Access-Control-Allow-Headers: *
Access-Control-Allow-Methods: *

In regards to your second comment, if Access-Control-Allow-Headers: * is specified for the request, the browser should ignore/reject a wildcard if the Credentials (e.g. cookies) were sent, in the same way it currently does for Access-Control-Allow-Origin.

---

Or in more detail; today, if a server responds with:

Access-Control-Allow-Origin: *

The browsers will normally handle this as you would expect (as a wildcard), but they will reject it when requesting a resource with .withCredentials = true.

For any website that wants to allow this behaviour (which is where the security risk comes in), they will _need_ to replace the wildcard with a proper Origin, and provide the Access-Control-Allow-Credentials header as well, e.g.

Access-Control-Allow-Origin: http://www.example.com
Access-Control-Allow-Credentials: true

So if we used the same logic with all 3 of these headers, then a response that contains the following should be fine:

Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: *
Access-Control-Allow-Methods: *

You can try this yourself with the following JS, which will only work if the full Origin/Credentials headers are sent from the remote website:

httpRequest = new XMLHttpRequest();
httpRequest.onreadystatechange = function() {
    if (httpRequest.readyState === XMLHttpRequest.DONE) {
        if (httpRequest.status === 200) {
            console.log(httpRequest.responseText);
        } else {
            alert('There was a problem with the request.');
        }
    }
}
httpRequest.withCredentials = true
httpRequest.open('GET', 'http://www.example.com/secure-page', true);
httpRequest.send(null);

[sicking]

I really like the idea that we allow Access-Control-Allow-Origin: *, Access-Control-Allow-Headers: * and Access-Control-Allow-Methods: *, but only when Access-Control-Allow-Credentials: true is not set.

When Access-Control-Allow-Credentials: true is set, then * is a forbidden value for all of Access-Control-Allow-Origin, Access-Control-Allow-Headers and Access-Control-Allow-Methods. If at that point a * is received for either of those headers, the header is ignored.

That's consistent with how Access-Control-Allow-Origin currently works, and should be very safe and cover the common use cases.

[roryhewitt]

@sicking, I'm fine with adding * support for all three of these headers (if
no credentials are in play).

One thought: A webserver admin might look at this and say "I don't want to
specify Access-Control-Allow-Methods: *, since that is effectively saying
that I will allow a DELETE request". Of course, we're only talking about
the preflight OPTIONS response here, and there's no requirement that the
server ACTUALLY responds to a DELETE request, but it might raise some red
flags. From a conceptual POV, AC-Allow-Headers and AC-Allow-Methods are
different.

On Tue, Mar 22, 2016 at 11:26 AM, Jonas Sicking [email protected]
wrote:

I really like the idea that we allow Access-Control-Allow-Origin: *, Access-Control-Allow-Headers:

• and Access-Control-Allow-Methods: *, but only when Access-Control-Allow-Credentials:
  true is not set.

When Access-Control-Allow-Credentials: true is set, then * is a forbidden
value for all of Access-Control-Allow-Origin, Access-Control-Allow-Headers
and Access-Control-Allow-Methods. If at that point a * is received for
either of those headers, the header is ignored.

That's consistent with how Access-Control-Allow-Origin currently works,
and should be very safe and cover the common use cases.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/whatwg/fetch/issues/251#issuecomment-199951266

Rory Hewitt

http://www.linkedin.com/in/roryhewitt

[annevk]

@roryhewitt I'm happy to add more examples to the specification. New issue or PR for those would be appreciated.

I also like the idea of extending wildcard support for the no-credentials scenario. @tyoshino, any concerns with that?

[roryhewitt]

@annevk & @craigfrancis, what about allowing Access-Control-Allow-Headers: * even on credentialed requests?

I understand that we can't have credentialed requests with Access-Control-Allow-Origin: * at least in part because cookies need to be specific to a domain, so if the response includes the _Set-Cookie_ header, we need to know which domain it's going to be created in. But why should this limitation apply to specifying which request headers are allowed? Headers aren't specific to a requesting domain like cookies.

Unless I'm missing something security-wise, allowing Access-Control-Allow-Headers: * on a credentialed request wouldn't open up any new security holes, would it? All this would do is tell the browser to allow all headers to be sent in the CORS request to the server without being specified individually - it's not a guarantee that the server will actually 'process' them.

As far as Access-Control-Allow-Methods: * goes, given that the possible set of methods is far more limited, I don't feel strongly about it. It's probably nice to add, simply to have some standardization across all the Access-Control-Allow request headers, but it's not a deal-breaker.

@annevk, OK, I will create an issue/PR for each new example. Also, is there any way to get W3C to update the current CORS spec they have to note that they are no longer updating it, and that it's been taken over by WHATWG?

[annevk]

FWIW, no need to have an issue/PR per example, one for all is fine too. Getting W3C to acknowledge reality is going rather slow, see https://github.com/whatwg/fetch/issues/204#issuecomment-184257430 for progress on that.

As for *, I don't think the problem is that allowing it opens security holes, it's that it makes the handshake simpler and it's unclear that we want the credentialed handshake to be simple.

[roryhewitt]

@annevk, fair point.

My concern is that if we don't allow Access-Control-Allow-Headers: * on a credentialed request, then this will significantly restrict some really useful new functionality. And as I noted in #253 (comment) in a different (but related) thread, I think that a far greater percentage of CORS requests are credentialed than we might assume.

[sicking]

I think Access-Control-Allow-Headers: * would be quite easy to get wrong. Websites quite often today simply set a x-xhr-request: true header as a CSRF prevention mechanism. If such a website is able to opt in to Access-Control-Allow-Headers: * requests with credentials, they are immediately vulnerable to CSRF attacks.

[sicking]

To clarify. I think Access-Control-Allow-Headers: * for requests without credentials is quite fine. That's generally no different than what can be done from non-browser clients anyway.

I think Access-Control-Allow-Headers: * in requests with credentials is quite risky and will likely lead to security problems.

My concern is that if we don't allow Access-Control-Allow-Headers: * on a credentialed request, then this will significantly restrict some really useful new functionality.

I don't think this is an accurate characterization. All functionality is already there. What we're debating here is making certain things easier.

[roryhewitt]

Hmmm.

If allowing Access-Control-Allow-Headers: * in credentialed requests does indeed open up any additional security holes, what are they (to be clear, I am assuming that forbidden headers are not allowed in any case)?

What I mean is that it's great that we're discussing introducing new functionality to make CORS more usable, but it's a shame that we're also limiting it to a subset of applications. If there is a valid reason to do this (_actual_ quantifiable security risk), that makes sense. I just don't see the specific risk vector here.

Basically, what specific security issue is introduced by allowing this?

[craigfrancis]

I can't think of any problems with Access-Control-Allow-Headers: * being allowed on credentialed requests (on the basis that Access-Control-Allow-Origin: * is still blocked, and provides the protection).

I still prefer that we start with it only being available on non-credentialed requests, just so all 3 headers work in the same way.

This would be much easier for browsers to accept and implement (because you're matching how it works already). Whereas if you allowed it on credentialed requests as well, I think we will need a lot more people to check this over (just to make sure we haven't missed anything).

[sicking]

As with any security feature, if it's used correctly there's no problems. The judgement call here is how easy it is to use incorrectly and what problems occur if used incorrectly.

I already mentioned one way that it can be used incorrectly here.

[roryhewitt]

@craigfrancis I guess I'm fine with allowing Access-Control-Allow-Headers: * only on non-credentialed requests initially. I just want to know what criteria would be used to subsequently determine whether we should also allow it on credentialed requests in the future.

My concern is that if we don't allow it for credentialed request _now_, then it will _never_ be allowed for credentialed requests - in the absence of lots of user requests to add it (see my final paragraph below), no-one will ever get round to doing further research (or whatever) to decide whether to allow it. So it will sit there in the Dev queue as "another feature that we should consider", until it withers and dies... So I think it makes sense to make a final decision now to either allow it on all requests or only on non-credentialed requests, even if that might delay the implementation whilst more people weigh in on the security risks.

As everyone looking at this thread can figure out I feel strongly that we should allow it on all requests :). I think the benefit it provides, in terms of simplicity of use outweighs the possibility of security risk. Others disagree. Disagreement is good.

That being said, I see that @sicking has raised a specific concern about allowing it on credentialed requests:

I think Access-Control-Allow-Headers: * would be quite easy to get wrong. Websites quite often today simply set a x-xhr-request: true header as a CSRF prevention mechanism. If such a website is able to opt in to Access-Control-Allow-Headers: * requests with credentials, they are immediately vulnerable to CSRF attacks.

I'm not sure I understand how allowing XHR to send any headers opens up to CSRF attacks (@sicking can you elaborate for me?), but in any case, as he explicitly points out, many of the concerns that he has (indeed, that we all have) are 'just' about the ways in which incorrect implementation can open security holes - basically, we're concerned about footguns (feetgun?).

Personally, I think that allowing Access-Control-Allow-Headers: * for all requests should be enabled, as long as it is clearly discussed that there may be a possibility of security risk. Ideally with examples of things that should not be done. One of the benefits of WHATWG controlling the CORS spec rather than W3C is that the WHATWG documentation includes far more information - notes and discussion points - which are not included in the sparse W3C specs. I think we should make use of this ability to include _in the spec itself_ more information about implementations, good and bad. But that's just me.

Finally, do we have some feedback method from _users_ (as opposed to browser manufacturers) aside from raising issues here? For instance, I have made the contention that a relatively large percentage of requests are credentialed (even if it's still a minority). That's based on my experience, which is a) limited and b) different from other peoples. But do we have any idea of whether I'm correct, or completely mistaken? I guess it's not possible to poll users, but it would be interesting to know.

[annevk]

Well, we could ask Chromium folks such as @foolip and @mikewest to set up measurements and find out to figure out how much CORS credentialed requests there are vs non-credentialed.

The attack @sicking mentioned is that if you have * but also use a header as CSRF token, that token can now be spoofed by any other website, leading to CSRF vulnerabilities (which are almost exclusively a concern for credentialed requests).

As for including more information in the specification, there's at least one open issue to that effect, #206, and I am certainly open to that as I already mentioned elsewhere. It'll happen quicker with specific PRs or issues.

[roryhewitt]

@annevk, so the attack @sicking mentioned is only an issue if Access-Control-Allow-Origin: * is used, correct? So if the site is set up to return Access-Control-Allow-Origin: <value-of-Origin-header>, then this isn't a concern, with or without credentials? Or am I misunderstanding?

@sicking, I hope it doesn't sound like I'm attacking you - when I get the 'bit between my teeth', I tend to start trying to poke holes and cause trouble. If any of this comes off that way, my apologies.

[sicking]

The criteria for adding new features to the web should be to either

• Enable things that were simply impossible before.
• Make it significantly easier to do something which is commonly done but that is complex to do today.

But in both cases a requirement is: Don't make it too easy to have security issues. Generally speaking, it should be easier to do the safe thing, than to do the unsafe thing.

I don't think any of the features discussed lately matches the first bullet.

I think supporting Access-Control-Allow-Headers: * and Access-Control-Allow-Methods: * matches the second criteria. But if we allow them for requests with credentials it fails the "too easy to have security issues" test. This is especially true if we add something like #210.

Adding features to credential-less requests is generally quite safe since it essentially only allows things that curl already allows, and so exposes the website to very little risk.

In order to add features for requests with credentials, I think it needs to be shown that web developers will understand the implications of those features and not opt in to the wrong thing.

Adobe's crossdomain.xml feature was a good example of what happens if we make it too easy to opt in to lots of functionality for requests with credentials. After that feature was released, lots of websites ended up with security problems. I have not heard anything similar for CORS.

[roryhewitt]

@sicking, surely curl can also do credentialed requests? It can certainly pass cookies...

I think we basically agree on the problem, but I am not convinced that the security risks are either too great or too difficult to document to outweigh the usability benefits.

As it is, web developers can do all kinds of stupid things. We can't stop them - all we can do is document the right way to implement CORS. If we restrict new features to non-credentialed requests only, I think they will just find other ways to screw up, trying to implement workarounds.

So I tend to think that we should provide comprehensive documentation (examples, scenarios, do's and don'ts) and allow them maximum flexibility, in terms of functionality..

The trouble with trying really hard to avoid a _footgun_, is that you often end up with _no gun_. Which is no good if you're faced with a bear.

[sicking]

Yes, curl can pass credentials. But only the credentials of the person invoking curl. If website A makes a CORS-with-credentials request to website B, then it's the users cookies to website B that are sent. That is something that the webdeveloper of website A can't accomplish with curl.

It's pretty clear that we're not in a "no gun" situation here. All we're talking about here are APIs for making things that are already possible easier. I.e. these proposals are just grease for existing guns.

I feel like we're just going in circles at this point. Unless new information is presented I will stay silent.

[roryhewitt]

@sicking, well the last thing I want is for this to get bogged down (even if it's my doing). So I will back off on trying to get _everything_ I want, and will defer to you on this.

If we agree to allow Access-Control-Allow-Headers: *, Access-Control-Allow-Methods: * (and potentially Access-Control-Expose-Headers: *?) on non-credentialed requests only (which seems to be pretty much a consensus), how do we go about allowing that (adding it to the spec and getting buy-in for any other parties)?

[annevk]

These issues have become rather messy. Let's make this one about allowing * for Allow-Headers and Allow-Methods on non-credentialed requests.

It seems we have rough consensus on that. @tyoshino concerns with allowing that?

(What's needed after this is fixed is tests, best contributed to https://github.com/w3c/web-platform-tests as per the README.)

[tyoshino]

I'm fine with allowing the wildcarded ACAH and/or wildcarded ACAM for non-credentialed requests.

The client would recall and iterates on the headers it has sent to create new entries (or updating expiration of) instead of iterating on the result of parsing ACAH / ACAM which is done when they're not wildcarded?

---

set up measurements and find out to figure out how much CORS credentialed requests there are vs non-credentialed.

We can measure the ratio between w/ credentials vs w/o credentials, but no idea for studying how many of them are mis-configured.

[annevk]

Yeah, we'd use the request headers/method for the cache.

@tyoshino I think it'd be interesting to know the ratio, as well as overall usage, to know how much we should invest in improvements.

[mozfreddyb]

I strongly agree with @sicking's reasoning.

I want to re-iterate that * should still not allow sending forbidden headers or forbidden methods.

:+1: to allowing * for on Access-Control-Allow-Headers and Access-Control-Allow-Methods non-credentialed requests

[sicking]

We might also want to prevent * from allowing sending the authorization header (nor the cookie header if #268 is fixed).

Allowing authorization to be set allows distributed brute-forcing of credentials, so that's probably something that we should require more explicit opt-in for.

[tyoshino]

@annevk going to gather data https://bugs.chromium.org/p/chromium/issues/detail?id=602925

[tyoshino]

I want to re-iterate that * should still not allow sending forbidden headers or forbidden methods.

Yea. This change wouldn't relax forbidden headers / methods.

@sicking What are you proposing? To extend the forbidden header name list for cross origin fetching by adding Authorization to it? Are you suggesting that we realize it by changing Access-Control-* header logic?

[sicking]

I'm not proposing that we move authorization to the forbidden-header-name list. Access-control-allow-headers: authorization currently allows the authorization header to be set, and I think we should let that continue to be the case. Changing that would risk breaking existing content and I see little reason to do that.

I'm suggesting that Access-control-allow-headers: * should _not_ allow the authorization header to be sent to the server. Since it's very easy for developers to miss the fact that Access-control-allow-headers: * would allow distributed brute-force of credentials.

We should figure out what syntax should be used if a website _really_ want to allow all headers, including the authorization header, to be set. One obvious proposal would be Access-control-allow-headers: *, authorization, but I'm open to other proposals too.

[tyoshino]

Oh, got it. Thanks Jonas for elaboration.

So, the step 6 of CORS-preflight fetch:

1. If one of request's header list' names is not in headerNames and its corresponding header is not a simple header, return a network error.

will be modified e.g. like as follows:

---

A non-wildcarded header is a header whose name is one of

• Authorization

...

1. ...
2. If headerNames is wildcard,
   
   
   
   1. If one of request's header list' names is a non-wildcarded header, return a network error.
   
   
3. Otherwise,
   
   
   
   1. If one of request's header list' names is not in headerNames and its corresponding header is not a simple header, return a network error.
   
   
4. ...
5. If headerNames is wildcard,
   
   
   
   1. For each headerName in request's header list' which is not a simple header and for which there is a header-name cache match using ...
   
   
   2. For each headerName in request's header list' which is not a simple header and for which there is no header-name cache match using ...
   
   
6. Otherwise,
   
   
   
   1. For each ...
   
   

---

Note that it's guaranteed that headerName is not any of the non-wildcarded header at the point we're modifying preflight cache.

[annevk]

Requiring Authorization to be explicitly listed seems fine. Allowing * as one of the header names listed also seems fine (so you can still specify Authorization).

I'm going to assume we'll have agreement on this. I don't quite have the bandwidth right now to update the specification, so if anyone else wants to take a crack it, go ahead, but I'll try to get to it soonish.

[tyoshino]

I'm fine with the requirement.

[tyoshino]

See the result of the measurement at https://bugs.chromium.org/p/chromium/issues/detail?id=602925#c6. About 30% of XHRs are both cross-origin and withCredentials set on Chrome. Note that:

• as roryhewitt pointed out at https://bugs.chromium.org/p/chromium/issues/detail?id=602925#c3, this value may include the cases where any credentials were actually not sent
• this early result is only from canary releases of Chrome whose usage includes more developers than stable/beta/dev

[tyoshino]

The above comment was reporting the fact too roughly. I'd like to correct.

(The number of pages Chrome users visited that used sync or async XHRs that are both cross-origin and with withCredentials set) / ((The number of pages Chrome users visited that used async XHRs) + (The number of pages Chrome users visited that used sync XHRs)) = ~30%

[wuliang142857]

my solution:

app.use(function(request, response, next) {
    response.header("Access-Control-Allow-Origin", request.headers.origin);
    response.header("Access-Control-Allow-Headers", "Content-Type, Access-Control-Allow-Headers, Authorization, X-Requested-With");
    response.header("Access-Control-Allow-Methods", "GET,HEAD,POST,PUT,DELETE,OPTIONS");
    response.header("Access-Control-Allow-Credentials", "true");
    next();
});

[annevk]

Bugs:

• https://bugzilla.mozilla.org/show_bug.cgi?id=1309358
• https://bugs.chromium.org/p/chromium/issues/detail?id=698721
• https://bugs.webkit.org/show_bug.cgi?id=169194
• https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/11169249/