Graylog2-server: Graylog 3.2: login via Active Directory integration fails with valid credentials

Created on 10 Feb 2020 · 8Comments · Source: Graylog2/graylog2-server

Our Graylog setup uses LDAP/AD integration feature for access management. Prior to Graylog 3.2, users were able to login with their AD credentials with no problem.

After upgrading to Graylog 3.2/3.2.1, they get "Invalid credentials, please verify them and retry." in login screen.

Further notes/observations:
1) When investigating the issue, I tried to do a fresh deploy of Graylog 3.2.1 + Elasticsearch + MongoDB, but the issue persists even after that. Our solution is hosted in Docker Swarm so fresh deploy means removal of all volumes to purge previously hold data, deploy our logging Docker Swarm stack and then perform series of Graylog API calls with Admin credentials to setup Inputs, Streams, LDAP/AD integration and Roles.
2) After fresh deploy, I tried to remove our custom Roles, but the issue persists.
3) AD integration is setup correctly - both 'Connection Test' and 'Login test' with valid user credentials pass in 'LDAP Settings' view. (Moreover the same setup works in previous versions)
4) AD setup looks as follows

   {
     "enabled": true,
     "system_username": "[email protected]",
     "ldap_uri": "ldaps://ADCONTROLLER.mydomain.company.com:636/",
     "use_start_tls": false,
     "trust_all_certificates": false,
     "active_directory": true,
     "search_base": "dc=mydomain,dc=company,dc=com",
     "search_pattern": "(&(objectClass=user)(sAMAccountName={0}))",
     "display_name_attribute": "displayName",
     "default_group": "Reader",
     "group_mapping": {
       "Operation_Admin": "Admin"
     },
     "group_search_base": "dc=mydomain,dc=company,dc=com",
     "group_id_attribute": "cn",
     "additional_default_groups": [
       null
     ],
     "group_search_pattern": "(objectClass=group)",
     "system_password_set": true
   }

5) I have set org.graylog2.security and org.graylog2.users to debug and this is a sample run when login fails.

2020-02-07 12:44:38,849 DEBUG: org.graylog2.security.MongoDbSessionDAO - Created session 5e3d5bb6b7a23c0592017c7b
2020-02-07 12:44:38,852 DEBUG: org.graylog2.security.MongoDbSessionDAO - Updating session org.apache.shiro.session.mgt.SimpleSession,id=ee009b5f-d0e6-4c80-8571-dcbfa98d6b22
2020-02-07 12:44:38,886 INFO : org.apache.directory.api.ldap.codec.osgi.DefaultLdapCodecService - Registered pre-bundled control factory: 1.3.6.1.4.1.18060.0.0.1
2020-02-07 12:44:38,887 INFO : org.apache.directory.api.ldap.codec.osgi.DefaultLdapCodecService - Registered pre-bundled control factory: 2.16.840.1.113730.3.4.7
2020-02-07 12:44:38,887 INFO : org.apache.directory.api.ldap.codec.osgi.DefaultLdapCodecService - Registered pre-bundled control factory: 2.16.840.1.113730.3.4.2
2020-02-07 12:44:38,888 INFO : org.apache.directory.api.ldap.codec.osgi.DefaultLdapCodecService - Registered pre-bundled control factory: 2.16.840.1.113730.3.4.18
2020-02-07 12:44:38,888 INFO : org.apache.directory.api.ldap.codec.osgi.DefaultLdapCodecService - Registered pre-bundled control factory: 1.2.840.113556.1.4.319
2020-02-07 12:44:38,889 INFO : org.apache.directory.api.ldap.codec.osgi.DefaultLdapCodecService - Registered pre-bundled control factory: 2.16.840.1.113730.3.4.3
2020-02-07 12:44:38,889 INFO : org.apache.directory.api.ldap.codec.osgi.DefaultLdapCodecService - Registered pre-bundled control factory: 1.3.6.1.4.1.4203.1.10.1
2020-02-07 12:44:38,889 INFO : org.apache.directory.api.ldap.codec.osgi.DefaultLdapCodecService - Registered pre-bundled control factory: 1.2.840.113556.1.4.473
2020-02-07 12:44:38,890 INFO : org.apache.directory.api.ldap.codec.osgi.DefaultLdapCodecService - Registered pre-bundled control factory: 1.2.840.113556.1.4.474
2020-02-07 12:44:38,891 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.3.6.1.4.1.18060.0.0.1
2020-02-07 12:44:38,891 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 2.16.840.1.113730.3.4.7
2020-02-07 12:44:38,891 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 2.16.840.1.113730.3.4.2
2020-02-07 12:44:38,891 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 2.16.840.1.113730.3.4.18
2020-02-07 12:44:38,891 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.2.840.113556.1.4.319
2020-02-07 12:44:38,891 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 2.16.840.1.113730.3.4.3
2020-02-07 12:44:38,892 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.3.6.1.4.1.4203.1.10.1
2020-02-07 12:44:38,892 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.3.6.1.4.1.42.2.27.8.5.1
2020-02-07 12:44:38,892 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 2.16.840.1.113730.3.4.9
2020-02-07 12:44:38,893 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 2.16.840.1.113730.3.4.10
2020-02-07 12:44:38,893 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.3.6.1.4.1.4203.1.9.1.3
2020-02-07 12:44:38,894 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.3.6.1.4.1.4203.1.9.1.4
2020-02-07 12:44:38,894 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.3.6.1.4.1.4203.1.9.1.1
2020-02-07 12:44:38,895 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.3.6.1.4.1.4203.1.9.1.2
2020-02-07 12:44:38,895 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.2.840.113556.1.4.473
2020-02-07 12:44:38,895 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.2.840.113556.1.4.474
2020-02-07 12:44:38,895 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.2.840.113556.1.4.841
2020-02-07 12:44:38,896 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.2.840.113556.1.4.417
2020-02-07 12:44:38,897 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.2.840.113556.1.4.1413
2020-02-07 12:44:38,897 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled control factory: 1.2.840.113556.1.4.528
2020-02-07 12:44:38,898 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled extended operation factory: 1.3.6.1.1.8
2020-02-07 12:44:38,899 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled extended operation factory: 1.3.6.1.4.1.18060.0.1.8
2020-02-07 12:44:38,900 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled extended operation factory: 1.3.6.1.4.1.18060.0.1.3
2020-02-07 12:44:38,901 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled extended operation factory: 1.3.6.1.4.1.18060.0.1.6
2020-02-07 12:44:38,902 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled extended operation factory: 1.3.6.1.4.1.18060.0.1.5
2020-02-07 12:44:38,903 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled extended operation factory: 1.3.6.1.4.1.4203.1.11.1
2020-02-07 12:44:38,904 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled extended operation factory: 1.3.6.1.4.1.4203.1.11.3
2020-02-07 12:44:38,905 INFO : org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - Registered pre-bundled extended operation factory: 1.3.6.1.4.1.1466.20037
2020-02-07 12:44:39,187 DEBUG: org.graylog2.users.UserServiceImpl - Loading user testusr
2020-02-07 12:44:39,198 DEBUG: org.graylog2.security.realm.LdapUserAuthenticator - User testusr: Mapping ldap group <Operation_Admin> to role <Admin>
2020-02-07 12:44:39,199 DEBUG: org.graylog2.security.realm.LdapUserAuthenticator - User testusr: No group mapping for ldap group <ADMIN>
2020-02-07 12:44:39,201 ERROR: org.graylog2.security.realm.LdapUserAuthenticator - Error during LDAP user account sync. Cannot log in user testusr
java.lang.IllegalArgumentException: null
    at org.bson.types.ObjectId.isValid(ObjectId.java:91) ~[graylog.jar:?]
    at org.bson.types.ObjectId.parseHexString(ObjectId.java:548) ~[graylog.jar:?]
    at org.bson.types.ObjectId.<init>(ObjectId.java:239) ~[graylog.jar:?]
    at org.graylog2.database.StringObjectIdFunction.apply(StringObjectIdFunction.java:28) ~[graylog.jar:?]
    at org.graylog2.database.StringObjectIdFunction.apply(StringObjectIdFunction.java:24) ~[graylog.jar:?]
    at com.google.common.collect.Iterators$6.transform(Iterators.java:785) ~[graylog.jar:?]
    at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47) ~[graylog.jar:?]
    at java.util.AbstractCollection.toArray(AbstractCollection.java:141) ~[?:1.8.0_242]
    at java.util.ArrayList.<init>(ArrayList.java:178) ~[?:1.8.0_242]
    at org.graylog2.users.UserImpl.setRoleIds(UserImpl.java:314) ~[graylog.jar:?]
    at org.graylog2.security.realm.LdapUserAuthenticator.updateFromLdap(LdapUserAuthenticator.java:306) ~[graylog.jar:?]
    at org.graylog2.security.realm.LdapUserAuthenticator.syncFromLdapEntry(LdapUserAuthenticator.java:233) ~[graylog.jar:?]
    at org.graylog2.security.realm.LdapUserAuthenticator.doGetAuthenticationInfo(LdapUserAuthenticator.java:125) [graylog.jar:?]
    at org.apache.shiro.realm.AuthenticatingRealm.getAuthenticationInfo(AuthenticatingRealm.java:571) [graylog.jar:?]
    at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doMultiRealmAuthentication(ModularRealmAuthenticator.java:219) [graylog.jar:?]
    at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doAuthenticate(ModularRealmAuthenticator.java:269) [graylog.jar:?]
    at org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:198) [graylog.jar:?]
    at org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(AuthenticatingSecurityManager.java:106) [graylog.jar:?]
    at org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:274) [graylog.jar:?]
    at org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:260) [graylog.jar:?]
    at org.graylog2.shared.security.SessionCreator.create(SessionCreator.java:77) [graylog.jar:?]
    at org.graylog2.rest.resources.system.SessionsResource.newSession(SessionsResource.java:129) [graylog.jar:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_242]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_242]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_242]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_242]
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
    at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
    at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
    at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
2020-02-07 12:44:39,206 DEBUG: org.graylog2.security.realm.PasswordAuthenticator - Retrieving authc info for user testusr
2020-02-07 12:44:39,206 DEBUG: org.graylog2.users.UserServiceImpl - Loading user testusr
2020-02-07 12:44:39,207 INFO : org.graylog2.shared.security.SessionCreator - Invalid credentials in session create request. Actor: "urn:graylog:user:testusr"

Expected Behavior

After setting up LDAP/AD integration, users should be able to login with their AD credentials.

Current Behavior

After setting up LDAP/AD integration, users are not be able to login with their AD credentials (even though 'Login test' in 'LDAP Settings' view passes OK).

Steps to Reproduce (for bugs)

Setup AD/LDAP integration in 'LDAP Settings' view.
Pass 'Connection Test'
Pass 'Login test' valid username and password from AD.
Try log in with valid username and password from AD -> fails.

Your Environment

The solution is hosted within single node Docker Swarm.

Graylog Version: 3.2.1 (https://hub.docker.com/layers/graylog/graylog/3.2.1-1/images/sha256-c63f9097635a80e680b188c737f11e0d508771e2d82c520590ee7111f0bac965)
Elasticsearch Version: 6.8.6 (https://hub.docker.com/layers/elasticsearch/library/elasticsearch/6.8.6/images/sha256-ae5006dce6042a234c0bc96ec1634925aae5d1ef590dab62f93d67761f36d694)
MongoDB Version: 4.2.1 (https://hub.docker.com/layers/mongo/library/mongo/4.2.1/images/sha256-37ed0ce402c453d82362c8674a95a84ccdc02c9864bfa4fc8170d49d42584a92)
Operating System: Debian docker image
Browser version: Chrome 79.0.3945.130

bug ldap triaged

Source

rsirny

All 8 comments

Have you checked the case of the OU and usernames? It appears graylog is currently case-sensitive(while ldap/ad authentication should be case-insensitive)

iankoetter on 10 Feb 2020

We are having what looks like this exact issue after upgrading from 3.1 to 3.2 using RPMs, authenticating with 389-directory LDAP. We verified logon succeeds when testing on the Authentication page but fails when attempting to log on. The LDAP server logs indicates the bind was successful, and this configuration worked with version 3.1.

ia-cirt on 10 Feb 2020

@gimmic the case is correct. Moreover, based on the stack trace the error takes place after the user is found within AD if I read the code correctly.

It's rather weird. Looking at the classes from the stack trace, it seems that there hasn't been many changes for quite some time in LdapUserAuthenticator or UserImp. There has been some work on authentication system, but I am not sure if it could break the user sync in db (which seems to be happening).

rsirny on 11 Feb 2020

@rsirny I think the culprit of the issue you described is this null value inside additional_default_groups:

     "additional_default_groups": [
       null
     ],

Could you please replace the value for additional_default_groups with [] and see if it helps with the issue?

It looks like when a role was deleted from the database but was still used in additional_default_groups, the backend cannot resolve the role name and defaults to null.

edmundoa on 14 Feb 2020

@edmundoa Thanks! It was the source of the issue. :)

The root cause of how it got there can hit anybody who is using automation over REST API to setup LDAP:

1) It seems Views User role was removed in 3.2.
2) It was still used in our deployment scripts which uses PUT /api/system/ldap/settings.
3) The endpoint happily accepts non-existing group in additional_default_groups and returns 204 ok.
4) The non-existing seems to be translated into null and used internally.

rsirny on 14 Feb 2020

🎉1

@edmundoa in our case it was due to an LDAP group that was deleted from our LDAP server that had a mapped role in Graylog. Going back into the LDAP group mappings and just saving the mappings fixed it. Thanks!

ia-cirt on 14 Feb 2020

👍1

@ia-cirt thank you for the information! I will try to reproduce that and see if we can manage to fix it too.

edmundoa on 15 Feb 2020

@ia-cirt I tried deleting a group from LDAP that was previously mapped into Graylog but users could still log in, even if they belonged to that group before. Since that seems to be another issue, please open a new issue if that happens again so we can investigate it. Thank you!

edmundoa on 17 Feb 2020

Was this page helpful?

0 / 5 - 0 ratings