Janusgraph: ScyllaDB support for Thrift and CQL backends

[apscomp]

Is it possible to get some links or add documentation of Storing JanusGraph on a ScyllaDB ?

If it is already in the works... no issues....

[analytically]

FYI https://github.com/scylladb/scylla/issues/693

[dylanht]

Depending on JanusGraph/janusgraph#35

As per the discussion there, it is reasonable to expect ScyllaDB to work as a drop in replacement for Apache Cassandra, both via Thrift and CQL as the protocol used to communicate with the backend. This follows from the assertion in the thread linked by @analytically above (scylladb/scylla#693) that integration via Thrift is working with outstanding concerns about incorrect results of OLAP queries addressed by recent patches applied to ScyllaDB master, which will become the 1.6 release. As CQL support was already implemented in ScyllaDB, should be a drop in replacement as well. @twilmes and @pluradj would be good to get your input on this guys as you took point on this and got a functional integration running.

While the focus will remain supporting Apache Cassandra, there could be significant return on investment here as this is theoretically low hanging fruit - possibly as simple as adding tests for ScyllaDB integration. The purpose of this issue is to attempt such, and if successful write some docs and open the door for users who want to deploy JanusGraph over ScyllaDB.

[penberg]

@dylanht ScyllaDB 1.6-rc1 is already out, which has all the relevant Titan/JanusGraph fixes. There really should be no need for a separate ScyllaDB backend for JanusGraph. If we have Apache Cassandra compatibility bugs, we'll address them in ScyllaDB.

[dylanht]

@penberg Good to know, thanks for the correction and authoritative statement about compatibility. Looking forward to testing Scylla with JanusGraph.

[sjudeng]

I ran through JanusGraph Thrift and CQL unit tests with Scylla (see #302). Results from CQL testing look especially good with the exception of TTL support. The results from Thrift testing show more issues beyond TTL. I don't think we need to spend much time on the Thrift issues if we're going to deprecate it but maybe others can weigh in with opinions on this as well. Note this doesn't include SSL tests, which would require some additional work to convert and pass the test server cert/key to Scylla.

Still to do:

1. Resolve the TTL test failures. This might just be filing an issue with Scylla and seeing if the test failures are eough to go on or if they need more.
2. SSL testing. Ideally the Scylla Docker image would support passing the server cert/key directly when running the container.
3. Integration testing. Based on discussion in https://github.com/scylladb/scylla/issues/693 it might be nice to run tests against different cluster configurations (number of nodes/replication factor).

CQL test results

| Module | Tests | Failures | Errors |
| ---- | ---- | ---- | ---- |
| cql | 245 | 8 | 0 |

All CQL test failures are associated with TTL support in JanusGraphTest.

CQLGraphTest>JanusGraphTest.testUnsettingTTL:4983 expected:<0> but was:<1>
CQLGraphTest>JanusGraphTest.testEdgeTTLTiming:4675 null
CQLGraphTest>JanusGraphTest.testVertexTTLWithCompositeIndex:4843 expected null, but was:<v[4324]>
CQLGraphTest>JanusGraphTest.testEdgeTTLWithTransactions:4723 null
CQLGraphCacheTest>JanusGraphTest.testUnsettingTTL:4983 expected:<0> but was:<1>
CQLGraphCacheTest>JanusGraphTest.testEdgeTTLTiming:4675 null
CQLGraphCacheTest>JanusGraphTest.testVertexTTLWithCompositeIndex:4843 expected null, but was:<v[4300]>
CQLGraphCacheTest>JanusGraphTest.testEdgeTTLWithTransactions:4723 null

Thrift test results

| Module | Tests | Failures | Errors |
| ---- | ---- | ---- | ---- |
| thrift (cassandra) | 241 | 13 | 13 |

In addition to TTL failures there are four additional failures associated with JanusGraphTest.simpleLogTest[WithFailures].

ThriftGraphTest>JanusGraphTest.simpleLogTest:3547->JanusGraphTest.simpleLogTest:3839 expected:<4> but was:<2>
ThriftGraphTest>JanusGraphTest.testUnsettingTTL:4983 expected:<0> but was:<1>
ThriftGraphTest>JanusGraphTest.testEdgeTTLTiming:4675 null
ThriftGraphTest>JanusGraphTest.simpleLogTestWithFailure:3552->JanusGraphTest.simpleLogTest:3839 expected:<4> but was:<2>
ThriftGraphTest>JanusGraphTest.testVertexTTLWithCompositeIndex:4843 expected null, but was:<v[4172]>
ThriftGraphTest>JanusGraphTest.testEdgeTTLWithTransactions:4723 null
ThriftGraphCacheTest>JanusGraphTest.simpleLogTest:3547->JanusGraphTest.simpleLogTest:3839 expected:<4> but was:<2>
ThriftGraphCacheTest>JanusGraphTest.testUnsettingTTL:4983 expected:<0> but was:<1>
ThriftGraphCacheTest>JanusGraphTest.testEdgeTTLTiming:4675 null
ThriftGraphCacheTest>JanusGraphTest.simpleLogTestWithFailure:3552->JanusGraphTest.simpleLogTest:3839 expected:<4> but was:<2>
ThriftGraphCacheTest>JanusGraphTest.testVertexTTLWithCompositeIndex:4843 expected null, but was:<v[4292]>
ThriftGraphCacheTest>JanusGraphTest.testEdgeTTLWithTransactions:4723 null
ThriftDistributedStoreManagerTest>DistributedStoreManagerTest.testGetDeployment:39 expected:<LOCAL> but was:<REMOTE>

Thrift test errors are shown below.

ThriftOLAPTest>OLAPTest.removeGhostVertices:189->JanusGraphBaseTest.getV:404 » NullPointer
ThriftPartitionGraphTest>JanusGraphPartitionGraphTest.testPartitionSpreadFlushBatch:331->JanusGraphPartitionGraphTest.testPartitionSpread:359 » NullPointer
ThriftPartitionGraphTest>JanusGraphPartitionGraphTest.testPartitionSpreadNoFlushNoBatch:346->JanusGraphPartitionGraphTest.testPartitionSpread:359 » NullPointer
ThriftPartitionGraphTest>JanusGraphPartitionGraphTest.testVertexPartitioning:180 » NullPointer
ThriftPartitionGraphTest>JanusGraphPartitionGraphTest.testPartitionSpreadNoFlushBatch:341->JanusGraphPartitionGraphTest.testPartitionSpread:359 » NullPointer
ThriftPartitionGraphTest>JanusGraphPartitionGraphTest.testPartitionSpreadFlushNoBatch:336->JanusGraphPartitionGraphTest.testPartitionSpread:359 » NullPointer
ThriftPartitionGraphTest>JanusGraphPartitionGraphTest.testKeybasedGraphPartitioning:467 » NullPointer
ThriftGraphTest.testVertexCentricQuery » NullPointer
ThriftGraphTest>JanusGraphTest.testBasic:203->JanusGraphBaseTest.assertMissing:400 » NullPointer
ThriftGraphTest>JanusGraphTest.testVertexRemoval:249->JanusGraphBaseTest.assertMissing:400 » NullPointer
ThriftGraphCacheTest.testVertexCentricQuery » NullPointer
ThriftGraphCacheTest.testBasic » NullPointer
ThriftGraphCacheTest.testVertexRemoval » NullPointer

[amcp]

@sjudeng @dylanht I agree we do not need to focus on Thrift in the ScyllaDB integration if it is on the deprecation path.

[sjudeng]

I created https://github.com/scylladb/scylla/issues/2497 to track the issue with TTL when using ScyllaDB. The underlying issue is actually just that JanusGraph is expecting entities are expired immediately after TTL has elapsed whereas ScyllaDB currently treats TTL as a minimum and in particular doesn't expire cells until 1 second after TTL.

[sjudeng]

The TTL fix in ScyllaDB was merged last August and targets version 2.1. The first ScyllaDB 2.1 release candidate was added last month and with this all TTL and other default tests are passing in janusgraph-cql. See https://github.com/scylladb/scylla-cluster-tests/issues/423.

Once ScyllaDB 2.1 is released we can update to run the tests automatically in Travis, then document ScyllaDB support and close this issue.

• [x] ScyllaDB 2.1 release
• [ ] Include janusgraph-cql with ScyllaDB in Travis build matrix
• [ ] Document support for ScyllaDB

[mbrukman]

See also this mailing list post by @pluradj for how to configure ScyllaDB as a backend for JanusGraph.

[porunov]

Thrift support was deprecated in 0.4.1 version of JanusGraph. Thrift is going to be completely removed in 0.6.0 version of JanusGraph due to deprecation. The PR is #1800
I am closing this issue as an obsolete issue. Feel free to reopen if you think it is still wise to support Thrift.