Orientdb: 2 Servers directly connected don't sync in distributed mode

Created on 9 Jul 2018 · 9Comments · Source: orientechnologies/orientdb

OrientDB Version: 3.0.2

Java Version: 1.8

OS: linux

Expected behavior

When launching 2 instances of ./dserver.sh on the same network, they should detect each other and sync.

Actual behavior

They don't detect each other and act as standalone.

Steps to reproduce

Launch one instance of ./dserver.sh on 2 computers.
The two computers can ping each other. IPtable is set on ACCEPT all INPUT/OUTPUT/FORWARD.
The traceroute reveals no jump in between.
The two configurations are mostly out of the boxe packages.

here is what the server terminal display :


2018-07-09 14:42:54:451 INFO  Detected limit of amount of simultaneously open files is 4096,  limit of open files for disk cache will be set to 1536
2018-07-09 14:42:54:457 WARNI Value of limit of simultaneously open files is too small, recommended value is 262144
2018-07-09 14:42:54:470 INFO  Loading configuration from: /data/orientdb-community-3.0.2/config/orientdb-server-config.xml...
2018-07-09 14:42:54:666 INFO  OrientDB Server v3.0.2 - Veloce (build e47e693f1470a7a642461be26983d4eca70777fd, branch develop) is starting up...
2018-07-09 14:42:54:685 INFO  8156430336 B/7778 MB/7 GB of physical memory were detected on machine
2018-07-09 14:42:54:685 INFO  Soft memory limit for this process is set to -1 B/-1 MB/-1 GB
2018-07-09 14:42:54:686 INFO  Hard memory limit for this process is set to -1 B/-1 MB/-1 GB
2018-07-09 14:42:54:687 INFO  Path to 'memory' cgroup is '/user.slice'
2018-07-09 14:42:54:689 INFO  Mounting path for memory cgroup controller is '/sys/fs/cgroup/memory'
2018-07-09 14:42:54:690 INFO  cgroup soft memory limit is 9223372036854771712 B/8796093022207 MB/8589934591 GB
2018-07-09 14:42:54:690 INFO  cgroup hard memory limit is 9223372036854771712 B/8796093022207 MB/8589934591 GB
2018-07-09 14:42:54:691 INFO  Detected memory limit for current process is 8156430336 B/7778 MB/7 GB
2018-07-09 14:42:54:692 INFO  JVM can use maximum 3925MB of heap memory
2018-07-09 14:42:54:692 INFO  Because OrientDB is running outside a container 2g of memory will be left unallocated according to the setting 'memory.leftToOS' not taking into account heap memory
2018-07-09 14:42:54:694 INFO  OrientDB auto-config DISKCACHE=1 805MB (heap=3 925MB os=7 778MB)
2018-07-09 14:42:54:717 INFO  Databases directory: /data/orientdb-community-3.0.2/databases
2018-07-09 14:42:55:037 INFO  Storage 'plocal:/data/orientdb-community-3.0.2/databases/OSystem' is opened under OrientDB distribution : 3.0.2 - Veloce (build e47e693f1470a7a642461be26983d4eca70777fd, branch develop)
2018-07-09 14:42:55:297 WARNI Port 0.0.0.0:2424 busy, trying the next available...
2018-07-09 14:42:55:298 INFO  Listening binary connections on 0.0.0.0:2425 (protocol v.37, socket=default)
2018-07-09 14:42:55:300 WARNI Port 0.0.0.0:2480 busy, trying the next available...
2018-07-09 14:42:55:301 INFO  Listening http connections on 0.0.0.0:2481 (protocol v.10, socket=default)
2018-07-09 14:42:55:308 INFO  Installing dynamic plugin 'orientdb-studio-3.0.2.zip'...
2018-07-09 14:42:55:310 INFO  Installing dynamic plugin 'orientdb-neo4j-importer-plugin-3.0.2-dist.jar'...
2018-07-09 14:42:55:314 INFO  Installing dynamic plugin 'orientdb-etl-3.0.2.jar'...
2018-07-09 14:42:55:317 INFO  Installing dynamic plugin 'orientdb-teleporter-3.0.2.jar'...
2018-07-09 14:42:55:323 INFO  ODefaultPasswordAuthenticator is active
2018-07-09 14:42:55:324 INFO  OServerConfigAuthenticator is active
2018-07-09 14:42:55:325 INFO  OSystemUserAuthenticator is active
2018-07-09 14:42:55:520 INFO  Configuring Hazelcast from '/data/orientdb-community-3.0.2/config/hazelcast.xml'. [FileSystemXmlConfig]
2018-07-09 14:42:55:560 WARNI Name of the hazelcast schema location is incorrect, using default [AbstractXmlConfigHelper]
2018-07-09 14:42:55:748 INFO  [LOCAL] [orientdb] [3.10.1] Prefer IPv4 stack is true. [AddressPicker]
2018-07-09 14:42:55:755 INFO  [LOCAL] [orientdb] [3.10.1] Picked [172.18.0.1]:2434, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=2434], bind any local is true [AddressPicker]
2018-07-09 14:42:55:764 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Hazelcast 3.10.1 (20180521 - 66f881d) starting at [172.18.0.1]:2434 [system]
2018-07-09 14:42:55:765 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Copyright (c) 2008-2018, Hazelcast, Inc. All Rights Reserved. [system]
2018-07-09 14:42:55:765 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Configured Hazelcast Serialization version: 1 [system]
2018-07-09 14:42:55:765 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] A non-empty group password is configured for the Hazelcast member. Starting with Hazelcast version 3.8.2, members with the same group name, but with different group passwords (that do not use authentication) form a cluster. The group password configuration will be removed completely in a future release. [Node]
2018-07-09 14:42:55:972 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Backpressure is disabled [BackpressureRegulator]
2018-07-09 14:42:55:983 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Running with 2 response threads [InboundResponseHandlerSupplier]
2018-07-09 14:42:56:351 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Creating MulticastJoiner [Node]
2018-07-09 14:42:56:489 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Starting 1 partition threads and 2 generic threads (1 dedicated for priority tasks) [OperationExecutorImpl]
2018-07-09 14:42:56:501 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments. [Diagnostics]
2018-07-09 14:42:56:509 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] [172.18.0.1]:2434 is STARTING [LifecycleService]
2018-07-09 14:42:58:608 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Cluster version set to 3.10 [system]
2018-07-09 14:42:58:610 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] 

Members {size:1, ver:1} [
    Member [172.18.0.1]:2434 - 8f82d593-7357-4cb5-8a9f-a6f960bb307b this
]
 [ClusterService]
2018-07-09 14:42:58:643 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] [172.18.0.1]:2434 is STARTED [LifecycleService]
2018-07-09 14:42:58:643 INFO  Starting distributed server 'node1531137572939' (hzID=8f82d593-7357-4cb5-8a9f-a6f960bb307b)...
2018-07-09 14:42:58:648 INFO  Distributed cluster time=Mon Jul 09 14:42:58 CEST 2018 (delta from local node=0)...
2018-07-09 14:42:58:693 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Initializing cluster partition table arrangement... [PartitionStateManager]
2018-07-09 14:42:58:772 INFO  [node1531137572939] Registered local server with nodeId=0
2018-07-09 14:42:58:789 INFO  Distributed Lock Manager server is 'node1531137572939'
2018-07-09 14:42:58:797 INFO  [node1531137572939] Loaded configuration for database 'OSystem' from disk: /data/orientdb-community-3.0.2/config/default-distributed-db-config.json
2018-07-09 14:42:58:805 INFO  [node1531137572939] Servers in cluster: [node1531137572939]
2018-07-09 14:42:58:811 INFO  [node1531137572939] Opening database 'demodb'...
2018-07-09 14:42:58:812 INFO  [node1531137572939] Loaded configuration for database 'demodb' from disk: /data/orientdb-community-3.0.2/databases/demodb/distributed-config.json
2018-07-09 14:42:58:823 INFO  [node1531137572939] Distributed servers status (*=current @=lockmgr[node1531137572939]):

+-----------------------+--------+---------+-----+---------+-----------------+-----------------+-----------------------+
|Name                   |Status  |Databases|Conns|StartedOn|Binary           |HTTP             |UsedMemory             |
+-----------------------+--------+---------+-----+---------+-----------------+-----------------+-----------------------+
|node1531137572939(*)(@)|STARTING|         |0    |14:42:55 |10.31.127.23:2425|10.31.127.23:2481|263,83MB/3,83GB (6,72%)|
+-----------------------+--------+---------+-----+---------+-----------------+-----------------+-----------------------+

2018-07-09 14:42:58:824 INFO  [node1531137572939] Current node started as MASTER for database 'demodb'
2018-07-09 14:42:59:037 INFO  Storage 'plocal:/data/orientdb-community-3.0.2/databases/demodb' is opened under OrientDB distribution : 3.0.2 - Veloce (build e47e693f1470a7a642461be26983d4eca70777fd, branch develop)
2018-07-09 14:42:59:091 WARNI [node1531137572939] Node 'node1531137572939' is NOT_AVAILABLE, the command 'sql.select from OUser where name = ? limit 1' against database 'demodb' will be executed only on local server with the possibility to have partial result
2018-07-09 14:42:59:093 INFO  [node1531137572939] Reassigning ownership of clusters for database demodb...
2018-07-09 14:42:59:095 INFO  [node1531137572939] Reassignment of clusters for database 'demodb' completed (classes=45)
2018-07-09 14:42:59:099 INFO  [node1531137572939] Publishing ONLINE status for database node1531137572939.demodb...
2018-07-09 14:42:59:102 INFO  [node1531137572939] Updated node status to 'ONLINE'
2018-07-09 14:42:59:105 INFO  [node1531137572939] Distributed servers status (*=current @=lockmgr[node1531137572939]):

+-----------------------+------+----------------------+-----+---------+-----------------+-----------------+-----------------------+
|Name                   |Status|Databases             |Conns|StartedOn|Binary           |HTTP             |UsedMemory             |
+-----------------------+------+----------------------+-----+---------+-----------------+-----------------+-----------------------+
|node1531137572939(*)(@)|ONLINE|demodb=ONLINE (MASTER)|0    |14:42:55 |10.31.127.23:2425|10.31.127.23:2481|335,51MB/3,83GB (8,55%)|
+-----------------------+------+----------------------+-----+---------+-----------------+-----------------+-----------------------+

2018-07-09 14:42:59:106 WARNI Authenticated clients can execute any kind of code into the server by using the following allowed languages: [sql]
2018-07-09 14:42:59:107 INFO  OrientDB Studio available at http://10.31.127.23:2481/studio/index.html
2018-07-09 14:42:59:107 INFO  OrientDB Server is active v3.0.2 - Veloce (build e47e693f1470a7a642461be26983d4eca70777fd, branch develop).

and the same on the other side (except IP@ ) :


018-07-09 14:42:51:879 INFO  Detected limit of amount of simultaneously open files is 4096,  limit of open files for disk cache will be set to 1536
2018-07-09 14:42:51:881 WARNI Value of limit of simultaneously open files is too small, recommended value is 262144
2018-07-09 14:42:51:890 INFO  Loading configuration from: /data/orientdb-community-3.0.2/config/orientdb-server-config.xml...
2018-07-09 14:42:52:064 INFO  OrientDB Server v3.0.2 - Veloce (build e47e693f1470a7a642461be26983d4eca70777fd, branch develop) is starting up...
2018-07-09 14:42:52:081 INFO  8156430336 B/7778 MB/7 GB of physical memory were detected on machine
2018-07-09 14:42:52:082 INFO  Soft memory limit for this process is set to -1 B/-1 MB/-1 GB
2018-07-09 14:42:52:082 INFO  Hard memory limit for this process is set to -1 B/-1 MB/-1 GB
2018-07-09 14:42:52:083 INFO  Path to 'memory' cgroup is '/user.slice'
2018-07-09 14:42:52:086 INFO  Mounting path for memory cgroup controller is '/sys/fs/cgroup/memory'
2018-07-09 14:42:52:086 INFO  cgroup soft memory limit is 9223372036854771712 B/8796093022207 MB/8589934591 GB
2018-07-09 14:42:52:087 INFO  cgroup hard memory limit is 9223372036854771712 B/8796093022207 MB/8589934591 GB
2018-07-09 14:42:52:087 INFO  Detected memory limit for current process is 8156430336 B/7778 MB/7 GB
2018-07-09 14:42:52:088 INFO  JVM can use maximum 3925MB of heap memory
2018-07-09 14:42:52:088 INFO  Because OrientDB is running outside a container 2g of memory will be left unallocated according to the setting 'memory.leftToOS' not taking into account heap memory
2018-07-09 14:42:52:091 INFO  OrientDB auto-config DISKCACHE=1 805MB (heap=3 925MB os=7 778MB)
2018-07-09 14:42:52:101 INFO  Databases directory: /data/orientdb-community-3.0.2/databases
2018-07-09 14:42:52:404 INFO  Storage 'plocal:/data/orientdb-community-3.0.2/databases/OSystem' is opened under OrientDB distribution : 3.0.2 - Veloce (build e47e693f1470a7a642461be26983d4eca70777fd, branch develop)
2018-07-09 14:42:52:661 WARNI Port 0.0.0.0:2424 busy, trying the next available...
2018-07-09 14:42:52:662 INFO  Listening binary connections on 0.0.0.0:2425 (protocol v.37, socket=default)
2018-07-09 14:42:52:664 WARNI Port 0.0.0.0:2480 busy, trying the next available...
2018-07-09 14:42:52:665 INFO  Listening http connections on 0.0.0.0:2481 (protocol v.10, socket=default)
2018-07-09 14:42:52:673 INFO  Installing dynamic plugin 'orientdb-etl-3.0.2.jar'...
2018-07-09 14:42:52:677 INFO  Installing dynamic plugin 'orientdb-studio-3.0.2.zip'...
2018-07-09 14:42:52:678 INFO  Installing dynamic plugin 'orientdb-neo4j-importer-plugin-3.0.2-dist.jar'...
2018-07-09 14:42:52:681 INFO  Installing dynamic plugin 'orientdb-teleporter-3.0.2.jar'...
2018-07-09 14:42:52:686 INFO  ODefaultPasswordAuthenticator is active
2018-07-09 14:42:52:687 INFO  OServerConfigAuthenticator is active
2018-07-09 14:42:52:688 INFO  OSystemUserAuthenticator is active
2018-07-09 14:42:52:857 INFO  Configuring Hazelcast from '/data/orientdb-community-3.0.2/config/hazelcast.xml'. [FileSystemXmlConfig]
2018-07-09 14:42:52:899 WARNI Name of the hazelcast schema location is incorrect, using default [AbstractXmlConfigHelper]
2018-07-09 14:42:53:091 INFO  [LOCAL] [orientdb] [3.10.1] Prefer IPv4 stack is true. [AddressPicker]
2018-07-09 14:42:53:098 INFO  [LOCAL] [orientdb] [3.10.1] Picked [172.18.0.1]:2434, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=2434], bind any local is true [AddressPicker]
2018-07-09 14:42:53:107 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Hazelcast 3.10.1 (20180521 - 66f881d) starting at [172.18.0.1]:2434 [system]
2018-07-09 14:42:53:108 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Copyright (c) 2008-2018, Hazelcast, Inc. All Rights Reserved. [system]
2018-07-09 14:42:53:108 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Configured Hazelcast Serialization version: 1 [system]
2018-07-09 14:42:53:108 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] A non-empty group password is configured for the Hazelcast member. Starting with Hazelcast version 3.8.2, members with the same group name, but with different group passwords (that do not use authentication) form a cluster. The group password configuration will be removed completely in a future release. [Node]
2018-07-09 14:42:53:308 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Backpressure is disabled [BackpressureRegulator]
2018-07-09 14:42:53:319 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Running with 2 response threads [InboundResponseHandlerSupplier]
2018-07-09 14:42:53:691 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Creating MulticastJoiner [Node]
2018-07-09 14:42:53:830 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Starting 1 partition threads and 2 generic threads (1 dedicated for priority tasks) [OperationExecutorImpl]
2018-07-09 14:42:53:840 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments. [Diagnostics]
2018-07-09 14:42:53:844 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] [172.18.0.1]:2434 is STARTING [LifecycleService]
2018-07-09 14:42:55:883 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Cluster version set to 3.10 [system]
2018-07-09 14:42:55:884 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] 

Members {size:1, ver:1} [
    Member [172.18.0.1]:2434 - 79669db5-3c2f-4685-bc1e-2e744da15e04 this
]
 [ClusterService]
2018-07-09 14:42:55:909 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] [172.18.0.1]:2434 is STARTED [LifecycleService]
2018-07-09 14:42:55:909 INFO  Starting distributed server 'node1531137618719' (hzID=79669db5-3c2f-4685-bc1e-2e744da15e04)...
2018-07-09 14:42:55:913 INFO  Distributed cluster time=Mon Jul 09 14:42:55 CEST 2018 (delta from local node=0)...
2018-07-09 14:42:55:952 INFO  [172.18.0.1]:2434 [orientdb] [3.10.1] Initializing cluster partition table arrangement... [PartitionStateManager]
2018-07-09 14:42:56:021 INFO  [node1531137618719] Registered local server with nodeId=0
2018-07-09 14:42:56:037 INFO  Distributed Lock Manager server is 'node1531137618719'
2018-07-09 14:42:56:044 INFO  [node1531137618719] Loaded configuration for database 'OSystem' from disk: /data/orientdb-community-3.0.2/config/default-distributed-db-config.json
2018-07-09 14:42:56:058 INFO  [node1531137618719] Servers in cluster: [node1531137618719]
2018-07-09 14:42:56:065 INFO  [node1531137618719] Opening database 'demodb'...
2018-07-09 14:42:56:066 INFO  [node1531137618719] Loaded configuration for database 'demodb' from disk: /data/orientdb-community-3.0.2/databases/demodb/distributed-config.json
2018-07-09 14:42:56:076 INFO  [node1531137618719] Distributed servers status (*=current @=lockmgr[node1531137618719]):

+-----------------------+--------+---------+-----+---------+-----------------+-----------------+-----------------------+
|Name                   |Status  |Databases|Conns|StartedOn|Binary           |HTTP             |UsedMemory             |
+-----------------------+--------+---------+-----+---------+-----------------+-----------------+-----------------------+
|node1531137618719(*)(@)|STARTING|         |0    |14:42:52 |10.31.127.24:2425|10.31.127.24:2481|284,43MB/3,83GB (7,25%)|
+-----------------------+--------+---------+-----+---------+-----------------+-----------------+-----------------------+

2018-07-09 14:42:56:077 INFO  [node1531137618719] Current node started as MASTER for database 'demodb'
2018-07-09 14:42:56:247 INFO  Storage 'plocal:/data/orientdb-community-3.0.2/databases/demodb' is opened under OrientDB distribution : 3.0.2 - Veloce (build e47e693f1470a7a642461be26983d4eca70777fd, branch develop)
2018-07-09 14:42:56:304 WARNI [node1531137618719] Node 'node1531137618719' is NOT_AVAILABLE, the command 'sql.select from OUser where name = ? limit 1' against database 'demodb' will be executed only on local server with the possibility to have partial result
2018-07-09 14:42:56:306 INFO  [node1531137618719] Reassigning ownership of clusters for database demodb...
2018-07-09 14:42:56:307 INFO  [node1531137618719] Reassignment of clusters for database 'demodb' completed (classes=45)
2018-07-09 14:42:56:311 INFO  [node1531137618719] Publishing ONLINE status for database node1531137618719.demodb...
2018-07-09 14:42:56:313 INFO  [node1531137618719] Updated node status to 'ONLINE'
2018-07-09 14:42:56:316 INFO  [node1531137618719] Distributed servers status (*=current @=lockmgr[node1531137618719]):

+-----------------------+------+----------------------+-----+---------+-----------------+-----------------+-----------------------+
|Name                   |Status|Databases             |Conns|StartedOn|Binary           |HTTP             |UsedMemory             |
+-----------------------+------+----------------------+-----+---------+-----------------+-----------------+-----------------------+
|node1531137618719(*)(@)|ONLINE|demodb=ONLINE (MASTER)|0    |14:42:52 |10.31.127.24:2425|10.31.127.24:2481|339,04MB/3,83GB (8,64%)|
+-----------------------+------+----------------------+-----+---------+-----------------+-----------------+-----------------------+

2018-07-09 14:42:56:316 WARNI Authenticated clients can execute any kind of code into the server by using the following allowed languages: [sql]
2018-07-09 14:42:56:316 INFO  OrientDB Studio available at http://10.31.127.24:2481/studio/index.html
2018-07-09 14:42:56:317 INFO  OrientDB Server is active v3.0.2 - Veloce (build e47e693f1470a7a642461be26983d4eca70777fd, branch develop).

Any Idea what might cause this ?

Thx

question

Source

edmondboulet

Most helpful comment

Hi @luigidellaquila, thanks for helping me again.

my network was indeed blocking multicast services. I added the IP manually before deploying and it fixed the issue.

Now this is on display when I start one instance :

+--------------------+------+----------------------+-----+---------+-----------------+-----------------+------------------------+
|Name                |Status|Databases             |Conns|StartedOn|Binary           |HTTP             |UsedMemory              |
+--------------------+------+----------------------+-----+---------+-----------------+-----------------+------------------------+
|node1531207589052(@)|ONLINE|demodb=ONLINE (MASTER)|0    |09:26:26 |10.31.127.23:2425|10.31.127.23:2481|105,58MB/3,97GB (2,60%) |
|node1531208719133(*)|ONLINE|demodb=ONLINE (MASTER)|4    |10:23:06 |10.31.127.24:2424|10.31.127.24:2480|544,58MB/3,83GB (13,87%)|
+--------------------+------+----------------------+-----+---------+-----------------+-----------------+------------------------+

But now, despite the fact that my nodes detect each other, they still don't sync.
I inserted a Vertex into one node with insert into V set prop = "test"
And typed select from V where prop ="test" into the other one but it didn't find anything.

Also when I type into the console CLUSTER STATUS
I get

Cluster status:

!ERROR: The database instance is not set in the current thread. Be sure to set it with: ODatabaseRecordThreadLocal.instance().set(db);

Thanks again for the help.

edmondboulet on 10 Jul 2018

👍2

All 9 comments

Hi @edmondboulet

Some networks do not support multicast, in such cases the auto-discovery does not work and you have to set IPs manually in the Hazelcast configuration. See https://orientdb.com/docs/2.2.x/Distributed-Configuration.html for a reference

Thanks

Luigi

luigidellaquila on 9 Jul 2018

Hi @luigidellaquila, thanks for helping me again.

my network was indeed blocking multicast services. I added the IP manually before deploying and it fixed the issue.

Now this is on display when I start one instance :

+--------------------+------+----------------------+-----+---------+-----------------+-----------------+------------------------+
|Name                |Status|Databases             |Conns|StartedOn|Binary           |HTTP             |UsedMemory              |
+--------------------+------+----------------------+-----+---------+-----------------+-----------------+------------------------+
|node1531207589052(@)|ONLINE|demodb=ONLINE (MASTER)|0    |09:26:26 |10.31.127.23:2425|10.31.127.23:2481|105,58MB/3,97GB (2,60%) |
|node1531208719133(*)|ONLINE|demodb=ONLINE (MASTER)|4    |10:23:06 |10.31.127.24:2424|10.31.127.24:2480|544,58MB/3,83GB (13,87%)|
+--------------------+------+----------------------+-----+---------+-----------------+-----------------+------------------------+

Also when I type into the console CLUSTER STATUS
I get

Cluster status:

!ERROR: The database instance is not set in the current thread. Be sure to set it with: ODatabaseRecordThreadLocal.instance().set(db);

Thanks again for the help.

edmondboulet on 10 Jul 2018

👍2

Hi @edmondboulet

How do you do the insert operation? From Studio or from an application API?

Thanks

Luigi

luigidellaquila on 10 Jul 2018

From the console. ( ./console.sh in bin folder )

I use Connect remote:localhost/demodb root root on each Node.
Also I have only 2 Nodes

edmondboulet on 10 Jul 2018

Uhm... that's interesting.
I'd say it is an already fixed problem, I'm testing now the new 3.0.4 (it will be released in a few hours) and I couldn't reproduce the issue.
The easiest thing I can suggest is to wait for next release (or try v 3.0.3) and upgrade

Thanks

Luigi

luigidellaquila on 10 Jul 2018

Ok, I was working with the tp3-3.0.2.
I will try with 3.0.3 and 3.0.4 if it doesn't work. Will kup.

edmondboulet on 10 Jul 2018

Update : just moved to 3.0.3 and the issue is still here.

here is more info on my cluster configuration :


CLUSTER CONFIGURATION [wQuorum: true] (LEGEND: X = Owner, o = Copy)
+---------------------+----+-----------+----------+------+------+
|                     |    |           |          |MASTER|MASTER|
|                     |    |           |          |ONLINE|ONLINE|
|                     |    |           |          |static|static|
+---------------------+----+-----------+----------+------+------+
|CLUSTER              |  id|writeQuorum|readQuorum|node1 |node3 |
+---------------------+----+-----------+----------+------+------+
|*                    |    |     2     |    1     |  X   |  o   |
|_studio_1            | 226|     2     |    1     |  o   |  X   |
|_studio_5            | 230|     2     |    1     |  o   |  X   |
|_studio_6            | 231|     2     |    1     |  o   |  X   |
|_studio_7            | 232|     2     |    1     |  o   |  X   |
|archaeologicalsites  | 113|     2     |    1     |  o   |  X   |
|archaeologicalsites_3| 116|     2     |    1     |  o   |  X   |
|archaeologicalsites_4| 117|     2     |    1     |  o   |  X   |
|archaeologicalsites_6| 119|     2     |    1     |  o   |  X   |
|attractions          |  81|     2     |    1     |  o   |  X   |
|attractions_3        |  84|     2     |    1     |  o   |  X   |
......

server status :

+-----------+------+----------------------+-----+---------+-----------------+-----------------+------------------------+
|Name       |Status|Databases             |Conns|StartedOn|Binary           |HTTP             |UsedMemory              |
+-----------+------+----------------------+-----+---------+-----------------+-----------------+------------------------+
|node3      |ONLINE|demodb=ONLINE (MASTER)|4    |11:32:16 |10.31.127.23:2425|10.31.127.23:2481|497,23MB/3,83GB (12,67%)|
|node1(*)(@)|ONLINE|demodb=ONLINE (MASTER)|2    |11:31:29 |10.31.127.24:2424|10.31.127.24:2480|518,92MB/3,83GB (13,22%)|
+-----------+------+----------------------+-----+---------+-----------------+-----------------+------------------------+

The two Nodes are definitely connected but whether in add vertex in node and query it in the other or the other way around, it doesn't find it (see screenshot)