Deeplearning4j: ParagraphVectors freeze

[dimparf]

ParagraphVectors is freeze on Starting learning process...
My code:

val paragraphVectors = new ParagraphVectors.Builder()
              .learningRate(0.025)            
              .minLearningRate(0.001)         
              .batchSize(1000)
              .epochs(20)
              .iterate(iterator)              
              .trainWordVectors(true)         
              .tokenizerFactory(tokenizerFactory)
              .allowParallelTokenization(false)
              .build()

paragraphVectors.fit()
Dataset is small.

[raver119]

What's your dl4j version? Show your pom.xml please

[dimparf]

1.0.0-beta2

[raver119]

Post your full pom.xml, source code and output from the console please.

[dimparf]

Ок,
05:40:05 INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [CpuBackend] backend
05:40:06 INFO org.nd4j.nativeblas.NativeOpsHolder - Number of threads used for NativeOps: 1
05:40:06 INFO org.nd4j.nativeblas.Nd4jBlas - Number of threads used for BLAS: 1
05:40:06 INFO o.n.l.a.o.e.DefaultOpExecutioner - Backend used: [CPU]; OS: [Linux]
05:40:06 INFO o.n.l.a.o.e.DefaultOpExecutioner - Cores: [2]; Memory: [0,7GB];
05:40:06 INFO o.n.l.a.o.e.DefaultOpExecutioner - Blas vendor: [MKL]
05:40:06 INFO o.d.m.s.SequenceVectors - Starting vocabulary building...
05:40:07 INFO o.d.m.w.wordstore.VocabConstructor - Sequences checked: [491], Current vocabulary size: [515]; Sequences/sec: [593,00];
05:40:07 INFO o.d.m.e.loader.WordVectorSerializer - Projected memory use for model: [0,39 MB]
05:40:07 INFO o.d.m.e.inmemory.InMemoryLookupTable - Initializing syn1...
05:40:07 INFO o.d.m.s.SequenceVectors - Building learning algorithms:
05:40:07 INFO o.d.m.s.SequenceVectors - building SequenceLearningAlgorithm: [PV-DBOW]
05:40:07 INFO o.d.m.s.SequenceVectors - building ElementsLearningAlgorithm: [SkipGram]
05:40:07 INFO o.d.m.s.SequenceVectors - Starting learning process...

[dimparf]

And full sbt (no pom, this is Scala project):

lazy val versions = new {
  val ScalatraVersion = "2.6.3"
  val dl4j = "1.0.0-beta2"
}
libraryDependencies ++= Seq(
 //dl4j deps
  "org.deeplearning4j" % "deeplearning4j-core" % versions.dl4j,
  "org.deeplearning4j" % "deeplearning4j-nlp" % versions.dl4j,
  "org.nd4j" % "nd4j-native" % versions.dl4j,
  "org.nd4j" % "nd4j-native-platform" % versions.dl4j,
  "info.debatty" % "java-string-similarity" % "1.1.0",
  "com.github.tototoshi" %% "scala-csv" % "1.3.5",
  "org.scalatest" %% "scalatest" % "3.0.1",
  "org.json4s" %% "json4s-jackson" % "3.5.0",
  "org.typelevel" %% "cats-core" % "1.2.0",
  "org.jsoup" % "jsoup" % "1.11.3",
  "com.github.wookietreiber" %% "scala-chart" % "latest.integration" % "test",

  //Scalatra
  "org.scalatra" %% "scalatra" % versions.ScalatraVersion,
  "org.scalatra" %% "scalatra-json" % versions.ScalatraVersion,
  "org.scalatra" %% "scalatra-swagger"  % versions.ScalatraVersion,
  "org.scalatra" %% "scalatra-scalatest" % versions.ScalatraVersion % "test",
  "ch.qos.logback" % "logback-classic" % "1.2.3" % "runtime",
  "org.eclipse.jetty" % "jetty-webapp" % "9.4.6.v20170531" % Compile,
  //"org.eclipse.jetty.websocket" % "websocket-server" % "9.4.6.v20170531" % "compile;provided",
  "javax.servlet" % "javax.servlet-api" % "3.1.0"
)

[raver119]

Deps looks ok. Post threads dump please. I.e. using jps/jstack or if you're running from Intellij - there's button available for that

[dimparf]

Ok, I will analyze the situation and write the result here.

[dimparf]

My code in async env.
Lock in VocabConstructor.buildJointVocabulary

        int numThreads = Math.max(numProc / 2, 2);
        PriorityScheduler executorService = new PriorityScheduler(numThreads);
        VocabRunnable runnable = new VocabRunnable(tempHolder, document, finCounter, loopCounter);

                executorService.execute(runnable);
...
  // if we're not in parallel mode - wait till this runnable finishes
                if (!allowParallelBuilder) {
                    try {
                        runnable.awaitDone();
                    } catch (InterruptedException e) {
                        Thread.currentThread().interrupt();
                        throw new RuntimeException(e);
                    }
                }

[MarcelloSoltea]

Same problem also with 1.0.0-beta3, I'm freeezed at :

                    runnable.awaitDone();

[raver119]

Fixed now.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.