Kong: Supporting alternate Host header for XMLHttpRequest requests

[subnetmarco]

XMLHttpRequest doesn't allow to explicitly set a Host header, so we would need to support an alternate header like X-Host.

http://www.w3.org/TR/XMLHttpRequest/#the-setrequestheader()-method

[rsdevigo]

It has a problem, from the moment you create a custom header, for example X-Host, the browser makes this request in a request with the OPTIONS method so that it seeks to know whether the server supports this header, therefore the value this header is not visible in "resolver.access" and so kong not know which address to send this api options requisition.

It has two solutions:

• The kong treat this request the OPTIONS type, only way for all apis he owns.
• Besides you can put the public_dns the Host header, be possible to access the api via: http: // kong: 8000 / api_id

[GuilhermeBenites]

+1

[subnetmarco]

@rsdevigo Kong should handle the OPTIONS request and automatically add X-Host to the Access-Control-Allow-Headers header.

[thibaultcha]

side note: this is not a request because it is something we need to do, we are not just considering it. Otherwise any issue is a request. This is closer to a bug.

[rsdevigo]

So @thefosk, I think if the kong handle the OPTIONS request, it´s dangerous, cause sometimes the request may come with custom header besides X-Host, and sometimes I want that my API handle this request. Or I mistaken somewhere ?

[subnetmarco]

If we decide to implement the X-Host solution, Kong will append the header to the list of headers that the API might return, so any scenario will be handled. So that everything can happen in one request.

[rsdevigo]

So, the idea is:

• Client -> do a GET|POST|PUT|DELETE request with X-Host, the browser do a preflight request with Method: OPTIONS and just a custom header list without their values is sent.
• Kong -> Receive this OPTIONS request and answer Access-Control-Allow-Headers : "X-Host".
  Client Browser -> Do the GET request with the X-Host value.
• Kong -> Receive the GET|POST|PUT|DELETE and get the X-Host value and proxy to the API target_url the request ?

Is it ?

[subnetmarco]

• The client makes an HTTP request to the API setting an additional X-Host header. The browser will execute an OPTIONS requests before executing the real request.
• Kong will proxy the OPTIONS request to the final API, and before returning the response to the browser it will append X-Host to whatever Access-Control-Allow-Headers the API returned.
• The browser now knows that the request conditions are valid (including the X-Host header), and executes the HTTP request.

[rsdevigo]

I got it, but how the kong know the api public_dns if the x-host value it does not go with the OPTIONS request ? Or go ?

[saaslink]

Not able to use Postman to test Kong calls. Though it allows "Host" header it actually doesn't get parsed since Chrome strips restricted headers including "Host".

[subnetmarco]

@rsdevigo I will post more info on this issue once I think about the implementation

[montanaflynn]

Could Kong just look for both Host and X-Host-Override headers and replace the Host header going upstream with the value from X-Host-Override if it exists?

[thibaultcha]

246 supports Host or X-Host-Override. Sounds good?

[rsdevigo]

Yes, sounds good, but u test with XMLHttpRequest and angularJS preflight requests ?
When the angularJS application send a OPTIONS request the value of X-Host-Override does not exist, just the name of custom header.
How the Kong get this value to proxy the OPTIONS request ?

[thibaultcha]

I'm not really sure if this issue is about allowing a custom header for restricted headers on browsers or about the OPTIONS request. What is really needed, here?

[rsdevigo]

I agree that now there is a way to make a XMLHttpRequest without using the browser for it, with this branch the kong can redirect the request to the API. But when the matter came up in the chat was to use the kong with a client written in angularjs.
I do not know if the issue was made thinking about it. But without a solution to this kind of request the kong can not work with libraries as angularjs and browsers.
I see that there should be another issue to deal with CORS and then XMLHttpRequests requests.

[thibaultcha]

From what I've read we want several things here:

1. @thefosk Be able to set a non-restricted header from the browser that can allow Kong to know what API we are proxying (X-Host or X-Host-Override).
2. @thefosk + @rsdevigo Proxy preflight requests to upstream. This means finding a way for Kong to know what API is being used in an OPTIONS request.
3. @montanaflynn When a preflight request is happening, allow to replace the upstream request's Host with a custom value.

As of today:

1. This is implemented in #246.
2. I don't see a way to do that, except with the path resolver coming in 0.3, see #192.
3. It requires 2, and I'm not sure why we would want to do that? Currently the requests going to upstream have Host set to the upstream's domain name, as expected. The issue is more in 2.

On a side note, keep in mind that if you want to use Kong from your browser you can use the CORS plugin. It will not proxy preflight requests to the end API by default (see kong/plugins/cors/access.lua#L63), but this issue raises concerns about it since like @rsdevigo said, if one uses the CORS plugin with preflight_continue=true from the browser, I don't see how Kong gets the public_dns of the API, I need to test this.

To conclude: we support a custom header, I do not see a way to proxy preflight requests with this resolver, but one can use the CORS plugin to perform browser calls (needs to be tested but probably still no preflight requests).

Please feel free to give me more infos on this if I'm missing anything.

[montanaflynn]

@thibaultCha I think you misunderstood me.

Could Kong just look for both Host and X-Host-Override headers and replace the Host header going upstream with the value from X-Host-Override if it exists?

Meaning if I curl 127.0.0.1:8000 -H "x-host-override: publicapis.com" and I have an API with the public_dns of publicapis.com it would still work. This is what I believe #246 did.

[thibaultcha]

Oh ok, then yes it is what @thefosk originally described in the first comment of this issue, and it is #246.

[subnetmarco]

I now fully understand what @rsdevigo was trying to say, I completely misunderstood his point. As he was trying to explain, the problem here is that the preflight OPTIONS request that the browser sends to Kong doesn't support any additional header, not even X-Host-Override.

Because of this, Kong will never know what is the API consumed, unless the API has a path resolver configured. The options we have here are:

• The Kong resolver itself handles CORS, and it will just accept any API regardless of the CORS plugin configuration (but what if I don't want CORS on that API? Why do we need a CORS plugin at this point if it is handled by the Kong resolver?)
• CORS is only supported when:
  
  
  
  • A path resolver is being used (no need to send additional headers, Kong will detect the API by the requested endpoint)
  
  
  • A fully configured public DNS is being used (which will make it work in production, but not when developing). Not really an acceptable solution, unless both the path and the public DNS are being set, and the path is used in development, and the public DNS in production.
  
  
• [DIRTY TRICK] Or the only other possibility is to send the public_dns as one of the Access-Control-Request-Headers values, that will then be handled by Kong. This will work because the only thing a browser can change is the value of Access-Control-Request-Headers, the OPTIONS request will look like:

OPTIONS / HTTP/1.1
Host: localhost:8000
Access-Control-Request-Headers: accept, test.com
Access-Control-Request-Method: GET
Origin: website.com

Kong will then parse the header, and try to resolve an API based on those values.

I may be wrong but basically whatever we decide to do, the X-Host-Override is just useless and can be removed.

[subnetmarco]

I have reopened the issue because clearly the previous solution is not satisfactory anymore.

[thibaultcha]

The problem was understood and discussed here. See:

I don't see a way to do that, except with the path resolver coming in 0.3,

The path resolver (now available) is still the only way. I don't like the dirty hack though.

However supporting X-Host-Override does cover one use case:

1. Browser makes a request to Kong
2. Kong has no idea which API is requested at this point, correct, this has been discussed here
3. But this Kong instance was configured to allow CORS for all its APIs (not currently possible)
4. Kong replies 204 to the preflight request and specifies X-Host-Override as an allowed header
5. Browser makes the request with X-Host-Override
6. Kong knows which API the browser is requesting and proxies the request

If we only keep Host, then one can only query Kong with the DNS resolver (the public_dns value) when the domain name has an A record pointing to the Kong instance. Which is not an ideal development setup.

That being said, I am fine with removing X-Host-Override.

[EwanValentine]

Should this not be and issue any longer? I'm attempting to send form data from a React application via Kong and I'm getting OPTIONS 405 when I do a POST request. The Host value isn't what I'm sending in the original request either.

[EwanValentine]

Anyone?

[subnetmarco]

@EwanValentine if you are using the path resolver this issue doesn't apply to your use-case.

[EwanValentine]

@thefosk I'm not using the path resolver. My API endpoints follow the following scheme:

/api/v1/login (api.auth.v1) 
/api/v1/signup (api.auth.v1)
/api/v1/artists (api.artists.v1)
/api/v1/orders (api.store.v1)
/api/v1/products (api.store.v1) 

etc... 

So I'm not sure how the path resolver would fit into that?

[EwanValentine]

No worries, I've sorted it! I had no idea you could just call auth.myapi.dev/api/v1/login. My bad!