Janusgraph: Multiplicity.MANY2ONE doesn't work

[porunov]

Edge multiplicity Multiplicity.MANY2ONE doesn't work.

After I created an edge label:

janusGraph.openManagement().makeEdgeLabel("connectedTo").multiplicity(Multiplicity.MANY2ONE).make();
janusGraph.tx().commit();

I can easily add two outgoing edges:

myVertex1.addEdge("connectedTo", myVertex2);
myVertex1.addEdge("connectedTo", myVertex3);
janusGraph.tx().commit();

I think that either old vertex has to be deleted when we try to create a second outgoing edge with Multiplicity.MANY2ONE or an exception has to be thrown.

[pluradj]

Your example doesn't commit the schema management transaction correctly. janusGraph.tx().commit() commits the graph transaction (vertex/edge mutations), but not the schema transactions. What's happening in your example is that since the edge label definition was not committed, the new edge was created with the default multiplicity Multiplicity.MULTI.

There is a code example in the Edge Label Multiplicity docs and below is an example from the Gremlin Console.

gremlin> janusGraph = JanusGraphFactory.open("inmemory");
==>standardjanusgraph[inmemory:[127.0.0.1]]
gremlin> mgmt = janusGraph.openManagement();
==>org.janusgraph.graphdb.database.management.ManagementSystem@522bf64e
gremlin> connectedTo = mgmt.makeEdgeLabel('connectedTo').multiplicity(Multiplicity.MANY2ONE).make();
==>connectedTo
gremlin> mgmt.commit();
==>null
gremlin> myVertex1 = janusGraph.addVertex("name", "myVertex1");
==>v[4208]
gremlin> myVertex2 = janusGraph.addVertex("name", "myVertex2");
==>v[8304]
gremlin> myVertex3 = janusGraph.addVertex("name", "myVertex3");
==>v[4224]
gremlin> janusGraph.tx().commit();
==>null
gremlin> myVertex1.addEdge("connectedTo", myVertex2);
==>e[1zi-38w-t1-6eo][4208-connectedTo->8304]
gremlin> myVertex1.addEdge("connectedTo", myVertex3);
An edge with the given label already exists on the out-vertex and the label [connectedTo] is out-unique
Type ':help' or ':h' for help.
Display stack trace? [yN]

[porunov]

@pluradj Sorry for the misspell. Actually, I did:
mgmt.commit();
Also, I've checked the multiplicity of the created edge after that and it was Multiplicity.MANY2ONE.
Still, it doesn't work.

[pluradj]

Can you provide a full code example that demonstrates the error? The example I shared previously shows the multiplicity working as documented.

[porunov]

@pluradj
I've just checked my code. I am using OGM Ferma graph. And the problem is there. Not in JanusGraph.
I tried to reproduce this problem with JanusGraph and I could not. JanusGraph works as expected.
Sorry for troubling.

[porunov]

@pluradj Sorry for confusion. I've found the same problem in JanusGraph.
When we use an edge label with Multiplicity.MANY2ONE the creation of a new outgoing vertex removes the old edges only from outgoing vertex but leaves the edge in ingoing vertex.
For example:
We create an edge from vertex A to vertex B and then we create an edge from vertex A to vertex C.
As a result, we will have the outgoing edge from vertex A to vertex C and ingoing edges to vertex B from vertex A and to vertex C from vertex A.
I.e.:

A -> C
B <- A
C <- A

The vertex B <- A has to be removed, but it leaves there.

Here is an example:

JanusGraphManagement mgmt = janusGraph.openManagement();
if(!mgmt.containsEdgeLabel("connectedTo")) {
    mgmt.makeEdgeLabel("connectedTo").multiplicity(Multiplicity.MANY2ONE).make();
    mgmt.commit();
    mgmt = janusGraph.openManagement();
}
System.out.println(mgmt.getEdgeLabel("connectedTo").multiplicity().name());

Vertex parentLocation1 = janusGraph.addVertex("location");
parentLocation1.property("location_id", 100000000);
parentLocation1.property("location_type", (short) 0);
parentLocation1.property("name", "ParentLocation1");

Vertex parentLocation2 = janusGraph.addVertex("location");
parentLocation2.property("location_id", 100000001);
parentLocation2.property("location_type", (short) 0);
parentLocation2.property("name", "ParentLocation2");

Vertex childLocation = janusGraph.addVertex("location");
childLocation.property("location_id", 100000002);
childLocation.property("location_type", (short) 1);
childLocation.property("name", "childLocation");

childLocation.addEdge("connectedTo", parentLocation1);

janusGraph.tx().commit();

childLocation.addEdge("connectedTo", parentLocation2);

janusGraph.tx().commit();

try {
    Thread.sleep(5000);
} catch (InterruptedException e) {
    e.printStackTrace();
}

parentLocation1 = janusGraph.traversal().V().hasLabel("location")
        .has("location_id", 100000000).next();

parentLocation2 = janusGraph.traversal().V().hasLabel("location")
        .has("location_id", 100000001).next();

childLocation = janusGraph.traversal().V().hasLabel("location")
        .has("location_id", 100000002).next();

Iterator<Edge> it = childLocation.edges(Direction.OUT, "connectedTo");

while (it.hasNext()){
    System.out.println("Child edge connected to: "+it.next().inVertex().<String>value("name"));
}

it = parentLocation2.edges(Direction.IN, "connectedTo");

while (it.hasNext()){
    System.out.println("Parent2 includes: "+it.next().outVertex().<String>value("name"));
}

it = parentLocation1.edges(Direction.IN, "connectedTo");

while (it.hasNext()){
    System.out.println("Parent1 includes: "+it.next().outVertex().<String>value("name"));
}

Output:

MANY2ONE
Child edge connected to: ParentLocation2
Parent2 includes: childLocation
Parent1 includes: childLocation

As we see Parent1 still includes childLocation even so it had to be removed.

[robertdale]

@porunov What version of JanusGraph and what backend are you using?

[porunov]

@robertdale Hello,
I am using a compiled version of JanusGraph from the master branch (0.3.0 version).
The version was built with the commit:
2cc4276213471adf50a109b7cb234ca60051e136 Mar 27, 2018

Storage backed: ScyllaDB 2.1.1
Index backend: Elasticsearch 6.2.3

[pluradj]

I tried your code on master against inmemory, berkeleyje, and cql. It throws an exception when creating the second edge.

gremlin> childLocation.addEdge("connectedTo", parentLocation2);
An edge with the given label already exists on the out-vertex and the label [connectedTo] is out-unique

[porunov]

@pluradj I am using ScyllaDB and I don't have any exceptions.
I started thinking about the realization of it. Maybe the way it is working in ScyllaDB is OK because the database is eventually consistent. So, it is possible that two edges would be created at the same time and added to the database. Because there is no guarantee of consistency, it is possible that
instance A and B see that there is no edges in vertex Child. Then both instances create ingoing part of the edge in both vertices Parent1 and Parent2. Then both instances write outgoing part of the edge in the vertex Child. As Cassandra (or ScyllaDB) is an eventual consistent database, it will save the latest write for the Child edge.
It is just my thoughts. I may be wrong. What do you think?
If so, I think the issue is resolved. May be it is good to add this information into the documentation.

[pluradj]

I've tried it now on Scylla also. I wasn't able to reproduce your error (that is, 2 edges get created despite Multiplicity.MANY2ONE) with Cassandra or Scylla (single node in Docker). With both backends, I encountered the _expected exception_ is thrown when the second edge is added. I tried with the all of the Cassandra adapters: cql, cassandra, cassandrathrift.

I don't think this is an eventual consistency issue.

What are your graph connection properties/janusgraph.properties? I didn't override any of the defaults, that is:

janusGraph = JanusGraphFactory.build().set("storage.backend", "cql").open();

[porunov]

@pencal But I also wasn't able to reproduce the error with ScyllaDB. That is why I assume that it is normal behaviour with such type of storage backend but I am not sure.

My janusgraph.properties is the next:

##############################################################
# General configuration
##############################################################
#gremlin.fermaGraph=org.janusgraph.core.JanusGraphFactory
query.batch=true
#query.fast-property=true
storage.batch-loading=true
storage.buffer-size=10240
storage.cql.atomic-batch-mutate=true
storage.cql.batch-statement-size=200
##############################################################
# Storage configuration
##############################################################
storage.backend=cassandra
storage.hostname=graph.storage.com
storage.username=cassandra
storage.password=cassandra
storage.cassandra.astyanax.local-datacenter=USA_east_1
##############################################################
# Index search configuration
##############################################################
index.search.backend=elasticsearch
index.search.hostname=index.search.com

##############################################################
# Ids allocation configuration
##############################################################
ids.block-size=7000000

My ScyllaDB and Elasticsearch are running under development mode on a local virtual machine. Both ScyllaDB and Elasticsearch has 2 GB of RAM available. There is only one instance of ScyllaDB, one instance of Elasticsearch and one instance of an application with JanusGraph.

[pluradj]

Thank you for the properties.

1. Adding storage.batch-loading=true recreates the problem, and the problem is not specific to any storage backend. When batch loading is enabled, it does not appear that the Multiplicity.MANY2ONE constraint is being enforced, thus allowing two edges to be created.

2. If storage.batch-loading=false (the default if nothing specified), the second edge throws a SchemaViolationException which is the expected behavior.

I posted Gremlin Console sessions of both scenarios in this gist.

[robertdale]

This is not an issue. See also http://docs.janusgraph.org/latest/bulk-loading.html

Enabling batch loading disables JanusGraph internal consistency checks in a number of places. Most importantly, it disables locking. In other words, JanusGraph assumes that the data to be loaded into JanusGraph is consistent with the graph and hence disables its own checks in the interest of performance.

[pluradj]

Thanks, Robert, you beat me to the doc reference. See also:

Important: Enabling storage.batch-loading requires the user to ensure that the loaded data is internally consistent and consistent with any data already in the graph.

[porunov]

Thank you for the help! Seems that I didn't take attention on it when I was reading the documentation.
I'm closing this issue as I think that it is resolved.