Janusgraph: hbase install error could not be instantiated

Created on 25 Oct 2017 · 16Comments · Source: JanusGraph/janusgraph

hbase config （is sign server）

hbase.rootdir
hdfs://bigdata.skycomm.com:9000/hbase

hbase.cluster.distributed
true

hbase.zookeeper.quorum
bigdata.skycomm.com

hbase.master
10.20.8.126

hbase.master.info.port
60010

hbase.regionserver.port
60020

error

[main] WARN org.apache.tinkerpop.gremlin.server.GremlinServer - Graph [graph] configured at [/usr/local/src/janusgraph/conf/janusgraph-hbase-es.properties] could not be instantiated and will not be available in Gremlin Server. GraphFactory message: Configuration must contain a valid 'gremlin.graph' setting
java.lang.RuntimeException: Configuration must contain a valid 'gremlin.graph' setting
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:57)
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:104)
at org.apache.tinkerpop.gremlin.server.GraphManager.lambda$new$0(GraphManager.java:55)
at java.util.LinkedHashMap$LinkedEntrySet.forEach(LinkedHashMap.java:671)
at org.apache.tinkerpop.gremlin.server.GraphManager.(GraphManager.java:53)
at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.(ServerGremlinExecutor.java:83)
at org.apache.tinkerpop.gremlin.server.GremlinServer.(GremlinServer.java:110)
at org.apache.tinkerpop.gremlin.server.GremlinServer.main(GremlinServer.java:344)
393 [main] INFO org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor - Initialized Gremlin thread pool. Threads in pool named with pattern gremlin-*
763 [main] INFO org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines - Loaded gremlin-groovy ScriptEngine

kinquestion

Source

444728248

Most helpful comment

@dzgYY — glad to hear you've solved your issue!

Can you please clarify which JARs were the issue, and what you mean by "some jars in janusgraph can't fit my cluster"? Was it a version conflict? Was there another issue?

More details will help other users solve their problem if they come across this issue. Thanks for your help!

mbrukman on 16 Nov 2017

👍6

All 16 comments

Configuration must contain a valid 'gremlin.graph' setting

add gremlin.graph=org.janusgraph.core.JanusGraphFactory to your properties file.

pluradj on 25 Oct 2017

👍1

An easy enhancement would be to add gremlin.graph=org.janusgraph.core.JanusGraphFactory to all the properties files shipped with the distribution.

pluradj on 25 Oct 2017

Also could mention in server docs https://github.com/JanusGraph/janusgraph/issues/452

pluradj on 25 Oct 2017

Hello my hbase startup, when I expect janusgraph will automatically kill the reginserver
error
org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: node1/10.20.8.128:60020

Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions:
Thu Oct 26 02:36:50 EDT 2017, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=68454: row 'janusgraph,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=node1,60020,1508998836149, seqNum=0

tks！！！

444728248 on 26 Oct 2017

@444728248 I'm not clear what your question or scenario was in your last comment. Do you have more details on what was happening when you saw that exception? Are you still able to recreate it? Sharing more of the log might be helpful (if it is really long, best if you do it as a gist then share the link here).

pluradj on 1 Nov 2017

same questions! Did you solve it?@444728248

dzgYY on 13 Nov 2017

I just run the GraphOftheGodhistory on gremlin shell.I used Janusgraph-0.2.0,the log is like that:

17:14:26 WARN org.janusgraph.diskstorage.hbase.HBaseStoreManager - Unexpected exception during getDeployment() java.lang.RuntimeException: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getDeployment(HBaseStoreManager.java:364) at # org.janusgraph.diskstorage.hbase.HBaseStoreManager.getFeatures(HBaseStoreManager.java:403) at org.janusgraph.diskstorage.common.AbstractStoreManager.getMetaDataSchema(AbstractStoreManager.java:58) at org.janusgraph.diskstorage.hbase.HBaseKeyColumnValueStore.<init>(HBaseKeyColumnValueStore.java:83) at org.janusgraph.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:463) at org.janusgraph.diskstorage.keycolumnvalue.KeyColumnValueStoreManager.openDatabase(KeyColumnValueStoreManager.java:43) at org.janusgraph.diskstorage.Backend.getStandaloneGlobalConfiguration(Backend.java:453) at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1378) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:80) at org.janusgraph.core.JanusGraphFactory$open.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at groovysh_evaluate.run(groovysh_evaluate:3) at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232) at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:70) at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:191) at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy) at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132) at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72) at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122) at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95) at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152) at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124) at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59) at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152) at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:83) at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232) at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:166) at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232) at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:478) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:739) at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getLocalKeyPartition(HBaseStoreManager.java:524) at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getDeployment(HBaseStoreManager.java:361) ... 56 more Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Mon Nov 13 17:14:26 CST 2017, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=68697: row 'janusns:janus-god,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=node159a78,60020,1504755205668, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:276) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:210) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327) at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302) at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167) at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:162) at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797) at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602) at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406) at org.janusgraph.diskstorage.hbase.HBaseAdmin1_0.tableExists(HBaseAdmin1_0.java:111) at org.janusgraph.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:716) ... 58 more Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=68697: row 'janusns:janus-god,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=node159a78,60020,1504755205668, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:169) at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:65) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at java.io.DataOutputStream.flush(DataOutputStream.java:123) at org.apache.hadoop.hbase.ipc.IPCUtil.write(IPCUtil.java:284) at org.apache.hadoop.hbase.ipc.IPCUtil.write(IPCUtil.java:272) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:920) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:34094) at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:400) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:204) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:65) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:364) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:338) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136) ... 4 more Could not open global configuration

tks! @pluradj @444728248

dzgYY on 13 Nov 2017

The caveat is that I run it on a hadoop cluster with kerberos authorization. But I gave the user access permissions on hbase。It doesn't seem to work. Do I need any other configurations?

dzgYY on 14 Nov 2017

I solved it .It's just because some jars in janusgraph can't fit my cluster.I changed them then it's ok!

dzgYY on 16 Nov 2017

@dzgYY — glad to hear you've solved your issue!

Can you please clarify which JARs were the issue, and what you mean by "some jars in janusgraph can't fit my cluster"? Was it a version conflict? Was there another issue?

More details will help other users solve their problem if they come across this issue. Thanks for your help!

mbrukman on 16 Nov 2017

👍6

@dzgYY I currently have the exact same stacktrace on a CDH kerberized cluster, which JARs did you have to switch ? I guess the HBase client ones ?

I tried switching the hbase-shaded-client-1.2.6.jar for cloudera's but I had the same result

Thx

cberez on 30 Nov 2017

I can confirm that as of 10/15/2018 this is still an issue. Lost almost an hour trying to work through this. Will try to help out here.

MikeThomsen on 15 Oct 2018

To prevent confusion we have recently added a default template for new issues containing the guidelines as to what belongs in issues. Usage, configuration, and general questions should be asked in gitter, stackoverflow, or the janusgraph-users google group. Github issues are for reporting bugs, requesting new features, and tracking the development of JanusGraph. If your issue is still outstanding please consult one of the communities mentioned. If you still feel like your issue belongs here and was closed in error please feel free to repoen it.