Janusgraph: Add support for AWS Keyspaces

[adric-work]

Describe the feature:
Add support for AWS Keyspaces' "DefaultPartitioner"

https://stackoverflow.com/questions/62220244/run-janusgraph-with-aws-keyspace-storage-backend

At the moment it looks like there is only support for
"RandomPartitioner" and "Murmur3Partitioner"

Describe a specific use case for the feature:
I would like to use AWS's managed Cassandra DB "Keyspaces" as my database for The Hive 4. I was able to connect via cqlsh from my The Hive instance, but was unable to get Janusgraph to connect.

Older issue where people are asking for the same feature in the comments:
https://github.com/TheHive-Project/TheHive/issues/1286

[farodin91]

I could think we could easily add an option to force usage one of three support partitioner.

[li-boxuan]

Seems duplicate of https://github.com/JanusGraph/janusgraph/issues/2291

Not sure but I suspect the DataStax driver we are using does not support AWS's DefaultPartitioner. https://community.datastax.com/questions/2989/trying-to-connect-to-aws-cassandra-with-datastaxsp.html

[bezalel]

I could think we could easily add an option to force usage one of three support partitioner.

I hope to use AWS managed Cassandra (Keyspaces) as JanusGraph storage backend!

is it simple issue to be fixed by adding "com.amazonaws.cassandra.DefaultPartitioner" support on CQLStoreManager.java?
https://github.com/JanusGraph/janusgraph/blob/master/janusgraph-cql/src/main/java/org/janusgraph/diskstorage/cql/CQLStoreManager.java#L217-L235

[li-boxuan]

It would be helpful if someone can confirm that the DataStax driver can be used to connect to AWS managed Cassandra first. Otherwise, there is not much JanusGraph can do at the moment.

[andreacardaropoli]

It would be helpful if someone can confirm that the DataStax driver can be used to connect to AWS managed Cassandra first. Otherwise, there is not much JanusGraph can do at the moment.

By the looks of it, it should be possible according to AWS documentation: https://docs.aws.amazon.com/keyspaces/latest/devguide/using_java_driver.html#java_tutorial

[li-boxuan]

I tested today by building a custom janusgraph version. No luck yet. I encountered the same problems reported at https://forums.aws.amazon.com/thread.jspa?messageID=942152 which seems to be a general problem with using Datastax Driver. Unfortunately, the solution on that thread seems to be switching to other drivers.

UPDATE: I overcame the previous problem. Now I got this error: "com.datastax.oss.driver.api.core.servererrors.InvalidQueryException: TOKEN is not yet supported". According to https://forums.aws.amazon.com/thread.jspa?messageID=942371 and https://forums.aws.amazon.com/thread.jspa?messageID=947841 seems AWS keyspace does not support token keyword. Thus, JanusGraph cannot be used on top of Amazon keyspace before they add token support.

UPDATE2: I overcame the previous problem by modifying the janusgraph code further. I Will continue testing and come back later

UPDATE3 (Mar 20, 2021): I was able to connect to AWS keyspace and load the GraphOfTheGods toy graph. Basic graph operations worked:

gremlin> g = graph.traversal()
==>graphtraversalsource[standardjanusgraph[cql:[cassandra.ap-east-1.amazonaws.com]], standard]
gremlin> g.V().valueMap()
12:24:00 WARN  org.janusgraph.graphdb.transaction.StandardJanusGraphTx  - Query requires iterating over all vertices [()]. For better performance, use indexes
==>[name:[tartarus]]
==>[name:[cerberus]]
==>[name:[sky]]
==>[name:[jupiter],age:[5000]]
==>[name:[nemean]]
==>[name:[hercules],age:[30]]
==>[name:[alcmene],age:[45]]
==>[name:[hydra]]
==>[name:[neptune],age:[4500]]
==>[name:[saturn],age:[10000]]
==>[name:[pluto],age:[4000]]
==>[name:[sea]]

There are some changes needed in the JanusGraph source code which I will clean up when I have some time. In case anyone wants to try now, apply this patch and do a janusgraph build.

Below are the configs needed:

storage.hostname=cassandra.<your-datacenter, e.g. ap-east-1>.amazonaws.com
storage.port=9142
storage.username=<your-service-username>
storage.password=<your-service-password>

storage.cql.keyspace=janusgraph
storage.cql.local-datacenter=<your-datacenter, e.g. ap-east-1>
storage.cql.ssl.enabled=true
storage.cql.ssl.truststore.location=<your-trust-store-location>
storage.cql.ssl.truststore.password=<your-trust-store-password>

storage.cql.only-use-local-consistency-for-system-operations=true
storage.cql.read-consistency-level=LOCAL_QUORUM
storage.cql.write-consistency-level=LOCAL_QUORUM

log.janusgraph.key-consistent=true
log.tx.key-consistent=true