Thehive: OAuth2 not working : Authentication failure

[softybear]

Request Type

Bug

Work Environment

| Question | Answer
|---------------------------|--------------------
| OS version (server) | CentOS 7
| OS version (client) | Windows 10
| TheHive version | 3.3.0
| Package Type | RPM
| Browser type & version | Chrome, Firefox, Microsoft Edge

Problem Description

Using @saibot94 's OAuth2.0 and SSO implementation, TheHive fails to log in and/or create account using OAuth2.0.

The configuration in the application.conf is as such :

auth {

  provider = [local, oauth2]

  oauth2 {
    clientId = "hive-app-client-id"
    clientSecret = "client-secret"
    redirectUri = "https://my-hive-instance.com/index.html#login"
    responseType = "code"
    grantType = "authorization_code"
    authorizationUrl = "https://auth-site.com/OAuth/Authorize"
    tokenUrl = "https://auth-site.com/OAuth/Token"
    userUrl = "https://auth-site.com/api/User"
    scope = "scope_field"
  }

   sso {
        mapper = group
        autocreate = true
        defaultRoles = ["read"]
        #autologin = true
        attributes {
                login = "user_id"
                name = "displayName"
                groups = "role_fields"
                }
        groups.mappings {
                role1_field= ["read", "write", "admin"]
                role2_field= ["read", "write"]
                }
        }

Once I click on "Sign in with SSO", I am well redirected to the Federated Identity platform and can successfully login. Once I'm logged on, it brings me back to thehive's login page. I do get a code in the url which works sucessfully in postman.

However, on the client side, nothings happens and I am left with these logs.

2019-04-02 14:40:20,011 [ERROR] from org.elastic4play.controllers.Authenticated in application-akka.actor.default-dispatcher-17 - Authentication failure:
        session: AuthenticationError User session not found
        pki: AuthenticationError Certificate authentication is not configured
        key: AuthenticationError Authentication header not found
        basic: AuthenticationError Authentication header not found
        init: AuthenticationError Use of initial user is forbidden because users exist in database
2019-04-02 14:40:20,020 [INFO] from org.elastic4play.ErrorHandler in application-akka.actor.default-dispatcher-20 - GET /api/user/current returned 401
org.elastic4play.AuthenticationError: Authentication failure
2019-04-02 14:40:20,020 [INFO] from org.elastic4play.ErrorHandler in application-akka.actor.default-dispatcher-17 - GET /api/list/ui_settings returned 401
org.elastic4play.AuthenticationError: Authentication failure

Steps to Reproduce

1. Go to the login page
2. "Sign in with SSO"
3. Checking logs in /var/log/thehive/application.log

Complementary information

TheHive is running as HTTP. I did not set up SSL nor reverse proxy on the server.

From an authorization code grant flow point of view :

• Redirect : OK

• Authorization code request & response : OK

• Exchange code for Access Token : No logs or requests to the Token Url

• Call API with Access Token : No logs, or requests about it

Am I missing some steps in order to successfully implement OAuth2.0 and SSO?

Maybe the configuration of the application.conf is not right also?

Thank you so much for your help,
softybear

[nadouani]

This feature is really hard to troubleshoot because it depends on the authentication platform APIs. when the Federated Identity platform redirects to TheHive login page, do you see the code on the URL?

From there, TheHive will call the tokenUrl with that code to get the token.

What do you see in your browser's console after redirection to login page?

[softybear]

Hi @nadouani,

Thank you for such a quick answer.

I do receive a code in the URL https://my-hive-instance.com/index.html#/login?code=[received_code].
This code is valid, as I can use postman to retrieve the access token to then call the user information successfully.

Unfortunately, I do not see much information when being redirected in my browser's console.

Before logging on with SSO, on the url https://my-hive-instance.com/index.html#/login, I see these two errors.

Once I sign in with sso, successfully login on fedid page and get redirected with a code,
I still see URL as such : https://my-hive-instance.com/index.html#/login?code=[received_code] and absolutely nothing really interesting in the console in both firefox and chrome.

Here is a screenshot

I may have given some more information at this link already but I am not sure that it gives much more.

[nadouani]

What's the details required by the tokenUrl API call to validate the code and get the token? What headers? What format? etc...

If TheHive doesn't call that API as expected then in fact it you will not be able to get the token, and then 401 is your response.

[softybear]

To validate the code and the token, a POST request should be sent to tokenUrl in this way :

##POST request to tokenUrl
##Header
"Authorization":"base64_encoded_credentials"
"Content-type":"application/x-www-form-urlencoded"
##Body
"grant_type":"authorization_code"
"client_id":"thehive"
"redirect_uri":"https://my-hive-instance.com/index.html#login"
"code":"returned_code_in_the_url_by_the_hive"

The response type is as such with a JSON:

##Response from the request
{
  "access_token":"access_token_string",
  "id_token":"id_token_string",
  "token_type":"Bearer"
}

Then I can just GET the userUrl with a Bearer access_token_string to retrieve successfully retrieve userinfo.

You are right, TheHive is not calling the API at all from what I see.

[nadouani]

I think that TheHive calls the tokenUrl with an « Authorization: token AUTHORIZATION_TOKEN » header, not with a basic auth.

[softybear]

Hi @nadouani,

Thank you a lot for helping me troubleshooting this issue. I am going to take a look and see if I can conform to what TheHive expects on my side. (an authorization token)

I'll keep you updated once I get further news.

[nadouani]

Here are the details of how the oauth apis are called:

https://github.com/TheHive-Project/TheHive/blob/master/thehive-backend/app/services/OAuth2Srv.scala#L90

PS: it's scala, good luck :p

[softybear]

Actually, I've been looking at the code and the OAuth2Srv.scala seems fine to me. From what I see and understand, it looks like it works the way I expect it and waits to catch the code to then use the Bearer Access Token to retrieve user informations.

However, I am never going inside getAuthTokenAndAuthenticate function as I do not get any Getting user token with the code from the response! in the logs. It just doesn't appear in my application.log and it should appear there if getAuthTokenAndAuthenticate is being called. see this piece of code

So I suppose the OAuth2 service manages to successfully redirect me to the redirectUri, then maybe it fails to store the code to process and execute the getAuthTokenAndAuthenticate function.

In the end, this leaves the client with an empty header. Hence why I get Authentication failure in the logs, coming from this service : KeyAuthSrv.scala.

I do not get any further logs apart from Authentication failure, though I should be getting all of the (many) logs from the getAuthTokenAndAuthenticate function.

Could this be considered a bug?

Maybe there are some prerequisites parameters to add in the application.conf that I missed or weren't explicitly documented. I've tried commands such as auth.method.basic = true and a few probably unrelated without success just in case the problem lies somewhere there.

[nadouani]

For sure, the oauth feature can be improved by adding more configuration options, because each oauth service implement its own way of things. There is no standard way. If you have ideas of options we can add, then your welcome ;).

I would consider that as an improvement.

[softybear]

Just as an update for transparency, in case anybody else is following the issue, I've enabled DEBUG logging by adding <logger name="org.elastic4play" level="DEBUG"/> to the logback.xml file.

2019-04-08 12:24:21,577 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-18 - key AuthenticationError SSO authentication is not supported 
2019-04-08 12:24:21,577 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-18 - local AuthenticationError SSO authentication is not supported
2019-04-08 12:24:21,577 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-18 - oauth2 OAuth2Redirect authorization_URL

Still no signs of getAuthTokenAndAuthenticate logs.

[saibot94]

Hey @softybear!

As I worked on the OAuth2 feature a while ago, I'll try to have a look on the issue one of these days, maybe I can clarify the situation.

[softybear]

Hi @saibot94,

Thank you so much for jumping in, I am looking forward to hear your input on the issue!

[1earch]

Hi @softybear,

Could you try to configure TheHive and your IDP to redirect you to https://hive-instance.com/index.html? This should exec the SSO login front-end code (which will call the backend SSO login code).

[softybear]

Hi @1earch,

Thank you for your input! I haven't fixed my problem with SSO. So, the redirect works, I can login on the IDP and then I am redirected back to thehive instance. I do get an authorization code in the url.

From this step, then, nothing happens. No log, no network request, nothing.

sso {
        mapper = group
        autocreate = true
        defaultRoles = ["read"]
        #autologin = true
        attributes {
                login = "user_id"
                name = "displayName"
                groups = "role_fields"
                }
        groups.mappings {
                role1_field= ["read", "write", "admin"]
                role2_field= ["read", "write"]
                }
        }

I think I just didn't manage to get the application.conf right. Have you managed to make oauth2 authentication work on your side?

I'm willing to put some more efforts to make it work! :))

EDIT : I've just seen you've worked on the issue and provided many additionnal features. Thank you for your work and I cannot wait to test it!

[1earch]

Could you try to configure your auth.oauth2.redirectUri as https://your-hive-instance.com/index.html (without the #login)?

auth {
  ...
  oauth2 {
    ...
    redirectUri = "https://my-hive-instance.com/index.html"

Note: you may need to reconfigure your IDP to allow this redirect URI.

[softybear]

Hi @1earch,

On current latest thehive version (3.3.0), it seems like it doesn't work for me.

Here are the logs :

 [INFO] from org.elastic4play.ErrorHandler in application-akka.actor.default-dispatcher-2 - GET /api/list/ui_settings returned 401
org.elastic4play.AuthenticationError: Authentication failure
        at org.elastic4play.controllers.Authenticated.$anonfun$getContext$4(Authenticated.scala:261)
        at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:303)
        at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
        at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
        at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
        at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
        at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2019-09-04 12:28:28,466 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-5 - key AuthenticationError SSO authentication is not supported
2019-09-04 12:28:28,466 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-5 - local AuthenticationError SSO authentication is not supported
2019-09-04 12:28:28,466 [DEBUG] from services.OAuth2Srv in application-akka.actor.default-dispatcher-5 - Getting user token with the code from the response!

The first authentication failure seems to happen in any cases where I am not logging in. (Might not be SSO/OAuth2.0 related)

My config is as such :

  provider = [local, oauth2]                                                                                                                                    

  oauth2 {                                                                                                                       
                clientId = "clientid"                                                                                                       
                clientSecret = "clientsecret"
                redirectUri = "https://hive-instance-url.com/index.html"                                                                    
                responseType = "code"
                grantType = "authorization_code"
                authorizationUrl = "autorizationurl"
                authorizationUrl = "https://auth-site.com/OAuth/Authorize"
                tokenUrl = "https://auth-site.com/OAuth/Token"
                userUrl = "https://auth-site.com/api/User"
                scope = "scope_field_for_user_info"
        }    

   sso {
        mapper = group
        defaultRoles = ["read"]
        attributes {
                login = "user_id"
                name = "displayName"
                groups = "user_login_field"
                }
        groups.mappings {
                user1= ["read", "write", "admin"]
                user2= ["read", "write"]
                }
        }        

With the redirectUri configured as you said, it still works and I do get a token :
https://hive-instance.com/index.html?code=PevLEROsCAuefWZcyNBMssCFBl2FbeqKccEAAAAp#/login

Surprisingly, I've noticed after some time that a POST request from https://hive-instance.com/api/ssoLogin?code=PevLEROsCAuefWZcyNBMssCFBl2FbeqKccEAAAAp times-out and shows a 502 Bad Gateway. I've tested all the steps of authorization code workflow manually with curl from the server and it works, both with and without proxy. I've also checked my URLs.

[1earch]

Concerning your logs:

 [INFO] from org.elastic4play.ErrorHandler in application-akka.actor.default-dispatcher-2 - GET /api/list/ui_settings returned 401
org.elastic4play.AuthenticationError: Authentication failure
 ...

This could be normal: you aren't logged in, but the UI (frontend) seems trying to access to a protected resource (/api/list/ui_settings).

For the other errors:

2019-09-04 12:28:28,466 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-5 - key AuthenticationError SSO authentication is not supported
2019-09-04 12:28:28,466 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-5 - local AuthenticationError SSO authentication is not supported
2019-09-04 12:28:28,466 [DEBUG] from services.OAuth2Srv in application-akka.actor.default-dispatcher-5 - Getting user token with the code from the response!

The first debug message explain that authentication by API key doesn't support SSO authentication (logical). The second one explain that authentication using local credentials doesn't support neither SSO. Finally, the real SSO authentication service is called, and TheHive "gets user token with the code from the response".

Thus, all of this seems normal for me.

Can you confirm that modifying the redirectUri caused the last debug message? This will resolve a part of your issue: getting the getAuthTokenAndAuthenticate method executed.

[1earch]

In addition, your token endpoint waits for a POST request to tokenUrl using Basic Auth:

##Header
"Authorization":"base64_encoded_credentials"
"Content-type":"application/x-www-form-urlencoded"
##Body
"grant_type":"authorization_code"
"client_id":"thehive"
"redirect_uri":"https://my-hive-instance.com/index.html"
"code":"returned_code_in_the_url_by_the_hive"

but TheHive's implementation doesn't support this. The request will be:

##Header
"Content-type":"application/x-www-form-urlencoded"
##Body
"grant_type":"authorization_code"
"client_id":"thehive"
"client_secret":"thehivesecret"
"redirect_uri":"https://my-hive-instance.com/index.html"
"code":"returned_code_in_the_url_by_the_hive"

Thus, I think you need to configure your token endpoint to accept request without HTTP basic auth. Then, could you try to get access token (with curl) then user infos using this second format? If this work, it may work with TheHive 😉

[softybear]

I confirm that modifying the redirectUri caused the last debug message.

> 2019-09-05 09:39:43,947 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-16 - oauth2 OAuth2Redirect https://idp-url.com/authorization.oauth2

this is what I get with redirectUri = "https://hive-instance-url.com/index.html#login"

Getting rid of #login in the redirectUri fixes it and calls the getAuthTokenAndAuthenticate method as supposed and shows Getting user token with the code from the response!

You are actually so right about what my endpoint expects, I didn't know TheHive would make the request that way. You just troubleshooted my issue so well, thanks a lot!

2019-09-05 09:46:12,598 [ERROR] from services.OAuth2Srv in application-akka.actor.default-dispatcher-3 - Token verification failure

I am right now taking actions to fix this, this is clearly a problem linked to my environment and hopefully the administrator can comply to these requests!

[hkelley]

I may have opened a duplicate of this issue yesterday - #1294 . Like @softybear, my IDP only allows HTTP Basic authentication at the token URL.

Is that ultimately the fix here? If so, I think it's only a few lines of code to add a configuration option and an if statement that toggled between Bearer and Basic.

[softybear]

I may have opened a duplicate of this issue yesterday - #1294 . Like @softybear, my IDP only allows HTTP Basic authentication at the token URL.

Is that ultimately the fix here? If so, I think it's only a few lines of code to add a configuration option and an if statement that toggled between Bearer and Basic.

Hi @hkelley,

I haven't looked up in the issue for SSO since then, but in the end thanks to @1earch, I found out that the main blocking point was the redirect URI. My company's IdP doesn't support redirect uri with '#' characters and I guess it's the same for you.

I've seen some ways documented by @1earch
where you would use apache as a reverse proxy to force a redirect in order to bypass that '#login' restriction.

I haven't checked for the new 3.4.1 yet, but it implements léarch's code and improved SSO, maybe it fixes that.

Edit : I also discovered that my company's IdP also accepted both Bearer and Basic Auth, even though only one was documented.

[shortstack]

having similar issues.. version 3.4.2-1

authing with OIDC sends me to the following, provides the code, says authenticated
https://hive_url/index.html?code=$thecode&locale=en_US&userState=Authenticated#!/login

but i'm always sent back to the hive login screen, where i auth with my identity provider credentials, and i get

2020-05-05 03:06:44,062 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-5 - key AuthenticationError SSO authentication is not supported
2020-05-05 03:06:44,062 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-5 - local AuthenticationError SSO authentication is not supported
2020-05-05 03:06:44,063 [DEBUG] from org.elastic4play.services.auth.MultiAuthSrv in application-akka.actor.default-dispatcher-5 - oauth2 OAuth2Redirect https://$oauth_endpoint/oauth2/authorize
2020-05-05 03:06:44,335 [ERROR] from org.elastic4play.controllers.Authenticated in application-akka.actor.default-dispatcher-18 - Authentication failure:
    session: AuthenticationError User session not found
    pki: AuthenticationError Certificate authentication is not configured
    key: AuthenticationError Authentication header not found
    init: AuthenticationError Use of initial user is forbidden because users exist in database
2020-05-05 03:06:44,336 [INFO] from org.elastic4play.ErrorHandler in application-akka.actor.default-dispatcher-18 - GET /api/list/ui_settings returned 401
org.elastic4play.AuthenticationError: Authentication failure
    at org.elastic4play.controllers.Authenticated.$anonfun$getContext$4(Authenticated.scala:272)
    at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:303)
    at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
    at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
    at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2020-05-05 03:06:44,336 [ERROR] from org.elastic4play.controllers.Authenticated in application-akka.actor.default-dispatcher-4 - Authentication failure:
    session: AuthenticationError User session not found
    pki: AuthenticationError Certificate authentication is not configured
    key: AuthenticationError Authentication header not found
    init: AuthenticationError Use of initial user is forbidden because users exist in database
2020-05-05 03:06:44,337 [INFO] from org.elastic4play.ErrorHandler in application-akka.actor.default-dispatcher-4 - GET /api/user/current returned 401
org.elastic4play.AuthenticationError: Authentication failure
    at org.elastic4play.controllers.Authenticated.$anonfun$getContext$4(Authenticated.scala:272)
    at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:303)
    at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
    at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
    at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

if i enable autologin, i get sent into an infinite loop of it authing and sending me to the login page

i've tried redirect URIs for index.html, with and without the #!/login

config:

auth {

  provider = [local,oauth2]

  oauth2 {
    clientId = "REDACTED"
    clientSecret = "REDACTED"
    redirectUri = "https://$hive_url/index.html"
    responseType = "code"
    grantType = "authorization_code"
    authorizationUrl = "https://$oauth_endpoint/oauth2/authorize"
    tokenUrl = "https://$oauth_endpoint/oauth2/token"
    userUrl = "https://$oauth_endpoint/oauth2/userinfo"
    scope = "openid"
  }

   sso {
        mapper = groups
        autocreate = true
        autoupdate = true
        defaultRoles = ["read","write"]
        autologin = true
        attributes {
            login = "username"
            name = "username"
            groups = "groups"
            #roles = "roles"
        }
        defaultRoles = ["read","write"]
        groups.mappings {
            analyst = ["read", "write"]
            admin = ["read", "write", "admin"]
        }
    }
}

[nadouani]

Hello folks, this looks like a hot and active issue.

Can anyone here provide us:

• An oauth provider and an account on it
• Credentials for a testing user
• The configuration part that fails

The foal here is to help us debug the issue :)

[nadouani]

And of course, we talk about TheHive 3 here, right?

[hkelley]

My issue is not the same as @shortstack's but I may be able to help set up a sandbox. @nadouani , how can we DM some URLs and credentials? I dropped an email to the contact@ address I found in your github profile.

[shortstack]

fusionauth has a really simple one line install on linux, https://fusionauth.io/docs/v1/tech/5-minute-setup-guide, if anyone is interested

i've got it working with other integrations via oidc, and would love to get thehive working with it. it's so close!

[shortstack]

this might be helpful, or shed some light on where the disconnect might be - https://github.com/FusionAuth/fusionauth-issues/issues/620#issuecomment-626873333