Fasthttp: CORS: allow every origin with credentials

[lus]

I have a simple question. I want to set "Access-Control-Allow-Credentials" to "true" and "Access-Control-Allow-Origin" to the current request origin, so every origin is allowed to access my API. Which method should I use on the RequestCtx to retrieve the current request origin which is eligible to show up in the CORS header? I tried different ones like ctx.RemoteAddr.String() or ctx.Request.URI().Host() but none of them worked.

I want that because I want to achieve a JWT authentication using the Authorization header, which is only received if I allow credentials.

Greetings

[erikdubbelboer]

When a browser does a cross origin request it will always add an Origin header containing the origin that is making the request. So this should work:

ctx.Response.Header.Set("Access-Control-Allow-Credentials", "true")
ctx.Response.Header.SetBytesV("Access-Control-Allow-Origin", ctx.Request.Peek("Origin"))

[lus]

Okay. Basically, I want every http library to connect to my API too. Do these send an "Origin" header or is this a Browser-only thing?

Thanks for your reply anyways.

[erikdubbelboer]

It's a browser thing because CORS is a browser thing. Other libraries don't care about CORS.

[lus]

Okay, that's strange because a friend uses C# WebRequest and definitely sends a valid Authorization header in his request. He sends this request to a simple test server written in Java and this server receives it. But if he sends it to a fasthttp server, ctx.Request.Header.Peek("Authorization") just returns an empty string, but if I send the same request using Postman, the header is filled with valid data. We both don't know how that's possible so I thought about a CORS error which I had in the past.

[erikdubbelboer]

Can you show your code and an example request made by C# WebRequest?

[cerus]

Hello, I'm the friend he's talking about. The code to the FastHttp server can be found here.
I'm actually using the RestSharp library for the requests. The code thats making the request (please excuse the mess) can be found here and the code that's calling that method can be found here.
As Lukaesebrot already said, when I send those requests to my test server the Authorization header is totally valid, but it doesnt seem to exist when I send the requests to the FastHttp server.

[erikdubbelboer]

What does this server print if you send the request to this simple server?:

package main

import (
    "github.com/valyala/fasthttp"
    "os"
)

func main() {
    fasthttp.ListenAndServe(":8080", func(ctx *fasthttp.RequestCtx) {
        ctx.Request.WriteTo(os.Stdout)
    })
}

[cerus]

The server prints the following:

GET /auth/check/ HTTP/1.1
User-Agent: VSpedSync/DevBuild
Host: localhost:8080
Content-Length: 0
Authorization: Bearer testtoken
Test: f
Accept: */*
Cache-Control: No-Cache
Accept-Encoding: gzip, deflate

[erikdubbelboer]

Looks to me like there is an Authorization header in there. Now try:

package main

import (
    "github.com/valyala/fasthttp"
    "fmt"
)

func main() {
    fasthttp.ListenAndServe(":8080", func(ctx *fasthttp.RequestCtx) {
        fmt.Println(string(ctx.Request.Header.Peek("Authorization")))
    })
}

If that works there must be something else wrong with your server code. Which endpoint are you testing exactly? https://github.com/VirtualSped/backend/search?q=Authorization&unscoped_q=Authorization

[lus]

Seems like this works pretty well. But I can't find a point in my code which would affect something like this. If you can, I would be thankful if you may tell me what exactly affects this.

[cerus]

I'm testing this endpoint: https://github.com/VirtualSped/backend/blob/master/api/endAuthCheck.go

[erikdubbelboer]

What is the response you are getting? Everything seems fine up to authentication.ValidateToken which is harder for me to check without the token and keys you are using. I suggest you try adding some fmt.Println("here") statements in your code to see where it goes wrong.

[cerus]

I modified the code on my machine and I get stuck at this section:

if authHeader == "" || !strings.Contains(authHeader, " ") {
    ctx.SetStatusCode(fasthttp.StatusBadRequest)
    ctx.SetBodyString("Invalid authorization header")
    return
}

[erikdubbelboer]

Try this code:

package main

import (
    "fmt"
    "github.com/fasthttp/router"
    "github.com/valyala/fasthttp"
    "strings"
)

func main() {
    fasthttp.ListenAndServe(":8080", func(ctx *fasthttp.RequestCtx) {
        webRouter := router.New()
        webRouter.HandleOPTIONS = true

        authGroup := webRouter.Group("/auth")
        authGroup.GET("/check", endAuthCheck)

        webRouter.Handler(ctx)
    })
}

func endAuthCheck(ctx *fasthttp.RequestCtx) {
    authHeader := string(ctx.Request.Header.Peek("Authorization"))
    if authHeader == "" || !strings.Contains(authHeader, " ") {
        ctx.SetStatusCode(fasthttp.StatusBadRequest)
        ctx.SetBodyString("Invalid authorization header")
        return
    }
    authSplit := strings.Split(authHeader, " ")
    if authSplit[0] != "Bearer" {
        ctx.SetStatusCode(fasthttp.StatusBadRequest)
        ctx.SetBodyString("Invalid authorization header")
        return
    }

    fmt.Println(authSplit[1])
}

[cerus]

I get stuck here again:

if authHeader == "" || !strings.Contains(authHeader, " ") {
    ctx.SetStatusCode(fasthttp.StatusBadRequest)
    ctx.SetBodyString("Invalid authorization header")
    return
}

[erikdubbelboer]

Are you sure you are sending the same request as above? When I send that to the Go server code from above it just prints testtoken as expected:

go run test.go
testtoken

curl -v localhost:8080/auth/check -H 'Authorization: Bearer testtoken' \
  -H 'User-Agent: VSpedSync/DevBuild' \
  -H 'Content-Length: 0' \
  -H 'Test: f' \
  -H 'Accept: */*' \
  -H 'Cache-Control: No-Cache' \
  -H 'Accept-Encoding: gzip, deflate'

*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 8080 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /auth/check HTTP/1.1
> Host: localhost:8080
> Authorization: Bearer testtoken
> User-Agent: VSpedSync/DevBuild
> Content-Length: 0
> Test: f
> Accept: */*
> Cache-Control: No-Cache
> Accept-Encoding: gzip, deflate
> 
< HTTP/1.1 200 OK
< Server: fasthttp
< Date: Sat, 11 Jan 2020 19:37:56 GMT
< Content-Length: 0
< 
* Connection #0 to host localhost left intact
* Closing connection 0

[cerus]

Yes, I'm sending the same request.

[erikdubbelboer]

What if you change the code like this:

if authHeader == "" || !strings.Contains(authHeader, " ") {
    ctx.Request.WriteTo(os.Stdout)
    ctx.SetStatusCode(fasthttp.StatusBadRequest)
    ctx.SetBodyString("Invalid authorization header")
    return
}

What does it print then?

[cerus]

It prints the following:

GET /auth/check HTTP/1.1
User-Agent: VSpedSync/DevBuild
Host: localhost:8080
Content-Length: 0
Test: f
Accept: */*
Cache-Control: No-Cache
Accept-Encoding: gzip, deflate

[erikdubbelboer]

As you can see that request does not contain an Authorization header. There must be something you're doing differently here. Could it be you're handling the token differently?

[cerus]

I'm sending the same request, nothing has changed. It's really weird.

[erikdubbelboer]

Can you use something like tcpdump to intercept the request that is being send just to be sure? If fasthttp doesn't print the Authorization header it means it didn't receive any.

[erikdubbelboer]

Can you give an example of a request including a valid token? with this server code: https://github.com/valyala/fasthttp/issues/723#issuecomment-573344741

Is there maybe some proxy in between when you test it with the real server, that isn't there when you test with these test go servers?

[cerus]

Sorry for my late reply! I was a bit busy.
I made a little video that shows what happens when I send the request to the "real" server, to my test server and to your test server (the one from this comment).
And no, I dont have any proxy.

[erikdubbelboer]

If I make this modification to your server, because I don't have mongodb:

diff --git a/main.go b/main.go
index 0c14956..1e80bd6 100644
--- a/main.go
+++ b/main.go
@@ -5,7 +5,6 @@ import (
        "github.com/VirtualSped/backend/api"
        "github.com/VirtualSped/backend/concommands"
        "github.com/VirtualSped/backend/config"
-       "github.com/VirtualSped/backend/database"
        "os"
 )

@@ -16,13 +15,6 @@ func main() {
                panic(err)
        }

-       // Connect to the MongoDB database
-       err = database.Connect()
-       if err != nil {
-               panic(err)
-       }
-       defer database.Close()
-
        // Serve REST API
        api.Serve()

Use this config.json:

{
    "api": {
        "host": "localhost:8080"
    }
}

And then send a valid request using this command:

 curl -v localhost:8080/auth/check -H 'Authorization: Bearer testtoken' \
  -H 'User-Agent: VSpedSync/DevBuild' \
  -H 'Content-Length: 0' \
  -H 'Test: f' \
  -H 'Accept: */*' \
  -H 'Cache-Control: No-Cache' \
  -H 'Accept-Encoding: gzip, deflate'

I get this response:

{"authorized":false}

Which to me indicates that everything on the server side seems to work fine. I don't have a windows machine or something that runs C# so I'm afraid I can't help with that part.

I see that in the request to the real server you use a real token. Can you try replacing that with your testtoken as well (so the requests are actually the same like I requested). My guess is there is something wrong with token.token that messes up the request to the real server.

[cerus]

Same thing happens when I replace the token with a test token.

[cerus]

I will try to run the go server on a virtual server in a bit and see if that makes a difference somehow.

[lus]

Little enhancement: Inside the api section of the config.json I also use a value named jwtSecret. I don't think that this is making a difference because it is just used to parse JWTs and without it it will parse them with an empty string, but just for your information.

[erikdubbelboer]

Can you try sending a request to http://dubbelboer.com:8090/auth/check so I can see what headers I receive?

[cerus]

I've sent three requests accidentally, the last one is the right one. Still doesnt work.

[erikdubbelboer]

I only see one request to /auth/check and it only contained these headers:

"test: f\r\nAccept: */*\r\nCache-Control: No-Cache\r\nUser-Agent: VSpedSync/DevBuild\r\nAccept-Encoding: gzip, deflate\r\nHost: dubbelboer.com:8090\r\n\r\n"

There is 100% something going wrong in your C# code.

[cerus]

That is really strange. Why do I receive the header in my Java test server though?

[erikdubbelboer]

Can you try sending another request to http://dubbelboer.com:8090/auth/check. I'm logging raw TCP packets now to make 100% sure.

[cerus]

I've sent a request

[erikdubbelboer]

Ah you are sending the request to /auth/check/ instead of /auth/check. So http://github.com/fasthttp/router automatically redirects the request to /auth/check (because RedirectTrailingSlash is true by default). And your C# library follows this redirect but removes the Authorization header for some reason.

Here are the TCP packets:

###
T 87.123.192.205:2799 -> 178.62.208.208:8090 [A]
......
#
T 87.123.192.205:2799 -> 178.62.208.208:8090 [AP]
GET /auth/check/ HTTP/1.1.
Authorization: Bearer testtoken.
test: f.
Accept: */*.
Cache-Control: No-Cache.
User-Agent: VSpedSync/DevBuild.
Host: dubbelboer.com:8090.
Accept-Encoding: gzip, deflate.
Connection: Keep-Alive.
.

##
T 178.62.208.208:8090 -> 87.123.192.205:2799 [AP]
HTTP/1.1 301 Moved Permanently.
Server: fasthttp.
Date: Sat, 11 Jan 2020 22:10:38 GMT.
Content-Length: 0.
Access-Control-Allow-Origin: .
Access-Control-Allow-Headers: Authorization, Content-Type, Set-Cookie, Cookie, Server.
Access-Control-Allow-Credentials: true.
Access-Control-Allow-Methods: POST, GET, OPTIONS.
Allow: POST, GET, OPTIONS.
Location: http://dubbelboer.com:8090/auth/check.
.

#
T 87.123.192.205:2799 -> 178.62.208.208:8090 [AP]
GET /auth/check HTTP/1.1.
test: f.
Accept: */*.
Cache-Control: No-Cache.
User-Agent: VSpedSync/DevBuild.
Accept-Encoding: gzip, deflate.
Host: dubbelboer.com:8090.
.

#
T 178.62.208.208:8090 -> 87.123.192.205:2799 [AP]
HTTP/1.1 400 Bad Request.
Server: fasthttp.
Date: Sat, 11 Jan 2020 22:10:38 GMT.
Content-Type: text/plain; charset=utf-8.
Content-Length: 28.
Access-Control-Allow-Origin: .
Access-Control-Allow-Headers: Authorization, Content-Type, Set-Cookie, Cookie, Server.
Access-Control-Allow-Credentials: true.
Access-Control-Allow-Methods: POST, GET, OPTIONS.
Allow: POST, GET, OPTIONS.
.
Invalid authorization header
#

[cerus]

Okay, let me test if it works when I remove the trailing slash

[cerus]

Oh god it finally works, thank you so much for your help. @Lukaesebrot and I spent so long trying to figure it out, thank you.

[lus]

Okay. At this point I want to thank you very much because of the things you've done to help us. Many thanks for the time and effort you put into that!

[erikdubbelboer]

No problem.

[Fenny]

Not all heroes wear capes :muscle: @erikdubbelboer