Presto: Issue while using Secure Internal Communication

[vikash32]

Hi,

I am trying to set up secure internal communication in presto with below config.properties. But it is throwing 401 unauthorised. As far as i understand it is happening cause it is asking for authentication from ldap and not able to authenticate itself from it.

Not sure but can we use LDAP and secure internal communication at the same time. I have not seen anything mention in the document itself.
https://prestodb.io/docs/current/security/ldap.html
https://prestodb.io/docs/current/security/internal-communication.html

coordinator=true
node-scheduler.include-coordinator=true
http-server.http.enabled=false
query.max-memory=5GB
query.max-memory-per-node=1GB
query.max-total-memory-per-node=2GB
node.internal-address=presto-coordinator.xyz.com
http-server.authentication.type=PASSWORD
http-server.https.enabled=true
http-server.https.port=8443
http-server.https.keystore.path=/presto.jks
http-server.https.keystore.key=vikash
discovery-server.enabled=false
discovery.uri=https://presto-coordinator.xyz.com:8443
internal-communication.https.required=true
internal-communication.https.keystore.path=/presto.jks
internal-communication.https.keystore.key=vikash

my password-authenticator.properties file are below.

password-authenticator.name=ldap
ldap.url=ldaps://ldapserver.xyz.com:636
ldap.user-bind-pattern=uid=${USER},ou=Employees,dc=xyz,dc=com
ldap.user-base-dn=DC=xyz,DC=com

Error Stacktrace are

2018-12-12T15:54:12.400+0530    INFO    main    com.facebook.presto.server.PrestoServer ======== SERVER STARTED ========
2018-12-12T15:54:12.400+0530    ERROR   Announcer-0 io.airlift.discovery.client.Announcer   Cannot connect to discovery server for announce: Announcement failed with status code 401:
2018-12-12T15:54:12.400+0530    ERROR   Announcer-0 io.airlift.discovery.client.Announcer   Service announcement failed after 33.49ms. Next request will happen within 0.00s
2018-12-12T15:54:12.408+0530    ERROR   Announcer-2 io.airlift.discovery.client.Announcer   Service announcement failed after 4.86ms. Next request will happen within 1.00ms

Do not know what i am doing wrong. can somebody help..

[findepi]

@vikash32 i think your question has already been answered: https://github.com/prestodb/presto/issues/12032#issuecomment-446488405

[vikash32]

@findepi

I used certificate for secure internal communication and it was working fine when i am not using LDAP authentication on presto co-ordinator node .
But Once i am using LDAP i.e adding http-server.authentication.type=PASSWORD to authenticate presto co-ordinator node and certificate as well for secure internal communication at the same time. It started giving 401.

i hope this time i am clear what i am facing.

[vikash32]

After one and half week banging my head on the wall, i finally made it working by adding below properties under config.properties.

http-server.authentication.type=PASSWORD,CERTIFICATE

[findepi]

@vikash32 i am glad to see this working for you.

[johnwhumphreys]

@vikash32 - You are my hero for the week. If I manage to get open source posting permission in this company I'm going to add a doc on this to the repository. I was doing it with a custom password authenticator but it was still destroying my brain :).

[johnwhumphreys]

@vikash32 - Actually, I don't think this works. It makes it run but then you can connect over JDBC without the user/password if you have the cert. It's like either work instead of both are required.

[johnwhumphreys]

@findepi - Any idea how to work around this and get a custom password authenticator working with internal TLS? :( Strong requirement for me that I can't break.

[vikash32]

@johnwhumphreys are you using LDAP/Kerberos for authentication and certificate of secure internal communication, If you are using any of these and you are using http-server.authentication.type=PASSWORD,CERTIFICATE then you must have to use 'Built in system access control', without it user will be able to login without username and password if they have certificate.

Please check the "Principal Rules" on this link. It will short out the issues. https://prestosql.io/docs/current/security/built-in-system-access-control.html

[findepi]

@findepi - Any idea how to work around this and get a custom password authenticator working with internal TLS? :( Strong requirement for me that I can't break.

@johnwhumphreys you can more conveniently reach me on Presto Slack (https://prestosql.io/community.html) or https://github.com/prestosql/presto

[johnwhumphreys]

@vikash32 - Thanks for the quick reply; digging in now. I'm using a custom password authenticator (so, similar to LDAP, but not the same).

Can you please help me understand how this prevents the cert from being used for specifically the external (e.g. JDBC) connections and not the internal ones? Having trouble seeing how this relates to the cert/etc, and my first couple tries with this haven't worked, so I think I am missing something.

Still trying though; thanks again for the help!

[tooptoop4]

@vikash32 with PASSWORD,CERTIFICATE how does it let login without username and password if they have certificate?

@johnwhumphreys are you using LDAP/Kerberos for authentication and certificate of secure internal communication, If you are using any of these and you are using http-server.authentication.type=PASSWORD,CERTIFICATE then you must have to use 'Built in system access control', without it user will be able to login without username and password if they have certificate.

Please check the "Principal Rules" on this link. It will short out the issues. https://prestosql.io/docs/current/security/built-in-system-access-control.html

[johnwhumphreys]

I got this all working with help from the google groups forum. Here's a write-up for anyone else coming in. https://coding-stream-of-consciousness.com/2019/08/04/presto-internal-tls-password-login-removing-private-key-from-jks-file/.

@vikash32 - Once I extracted the cert into a trust-store and only gave that to the JDBC client, it worked fine. SSL was used and the password login was forced. Originally, I naively gave out the JKS file which had the private key, then the JDBC client was logging in with the JKS file the same way the internal presto nodes were authenticating to each other. That completely bypassed/ignored the password login.

[tooptoop4]

if internal-communication.https.required=false can the password login be bypassed?

[johnwhumphreys]

@tooptoop4 - If you are not using secure internal communication, then a standard password login mechanism should work fine (LDAP or custom).

Without secure internal communication, you just have http-server.authentication.type=PASSWORD rather than http-server.authentication.type=PASSWORD,CERTIFICATE. In that case, certificates will not be used for authentication, so it doesn't matter if the client has one.

The client will still need a JKS file or trust store for HTTPS/TLS as secure transport is required to make any password plugins work in Presto. But the JKS file or trust store will not affect login and a password login will always be required.

[tooptoop4]

I have:
http-server.http.port=4321
discovery-server.enabled=true
discovery.uri=http://dns:4321
http-server.authentication.type=PASSWORD,CERTIFICATE
http-server.https.enabled=true
http-server.https.port=6789
http-server.https.keystore.path=keystore.jks
http-server.https.keystore.key=keystorepass

But I can't reproduce what you mentioned about logging in via JDBC with just cert and no credentials. I am on prestodb 0.220

If I don't pass credentials I get:
2019-08-16 18:02:47 ERROR Error retrieving catalog information Authentication failed: Unauthorized
java.sql.SQLException: Authentication failed: Unauthorized
at com.facebook.presto.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:276)
at com.facebook.presto.jdbc.PrestoStatement.execute(PrestoStatement.java:228)
at com.facebook.presto.jdbc.PrestoStatement.executeQuery(PrestoStatement.java:77)
at com.facebook.presto.jdbc.PrestoDatabaseMetaData.select(PrestoDatabaseMetaData.java:1414)
at com.facebook.presto.jdbc.PrestoDatabaseMetaData.getCatalogs(PrestoDatabaseMetaData.java:945)
at workbench.db.DbMetadata.getCatalogInformation(DbMetadata.java:2653)
at workbench.gui.dbobjects.objecttree.TreeLoader.loadCatalogs(TreeLoader.java:433)
at workbench.gui.dbobjects.objecttree.TreeLoader.load(TreeLoader.java:317)
at workbench.gui.dbobjects.objecttree.DbObjectsTree.load(DbObjectsTree.java:443)
at workbench.gui.dbobjects.objecttree.DbTreePanel.doConnect(DbTreePanel.java:399)
at workbench.gui.dbobjects.objecttree.DbTreePanel.access$400(DbTreePanel.java:110)
at workbench.gui.dbobjects.objecttree.DbTreePanel$3.run(DbTreePanel.java:344)
Caused by: com.facebook.presto.jdbc.internal.client.ClientException: Authentication failed: Unauthorized
at com.facebook.presto.jdbc.internal.client.StatementClientV1.requestFailedException(StatementClientV1.java:454)
at com.facebook.presto.jdbc.internal.client.StatementClientV1.(StatementClientV1.java:135)
at com.facebook.presto.jdbc.internal.client.StatementClientFactory.newStatementClient(StatementClientFactory.java:24)
at com.facebook.presto.jdbc.QueryExecutor.startQuery(QueryExecutor.java:46)
at com.facebook.presto.jdbc.PrestoConnection.startQuery(PrestoConnection.java:701)
at com.facebook.presto.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:240)
... 11 more