Deeplearning4j: QuickStart problem

[willishf]

Following the guide at https://deeplearning4j.org/docs/latest/deeplearning4j-quickstart get an error on maven clean install can't resolve dependency

Issue Description

PS C:\deeplearning\deeplearning4j-examples\dl4j-examples> mvn install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.deeplearning4j:dl4j-examples >------------------
[INFO] Building DeepLearning4j Examples 1.0.0-beta5
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The POM for org.deeplearning4j.examples:shared-utilities:jar:1.0.0-beta5 is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.701 s
[INFO] Finished at: 2019-10-21T12:22:06-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project dl4j-examples: Could not resolve dependencies for project org.deeplearning4j:dl4j-examples:jar:1.0.0-beta5: Failure to find org.deeplearning4j.examples:shared-utilities:jar:1.0.0-beta5 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

Please describe our issue, along with:

• expected behavior
• encountered behavior

Version Information

Please indicate relevant versions, including, if relevant:

• Deeplearning4j version
• Platform information (OS, etc)
• CUDA version, if used
• NVIDIA driver version, if in use

Additional Information

Where applicable, please also provide:

• Full log or exception stack trace (ideally in a Gist: gist.github.com)
• pom.xml file or similar (also in a Gist)

Contributing

If you'd like to help us fix the issue by contributing some code, but would
like guidance or help in doing so, please mention it!

[Bfzanchetta]

I have been trying to build DL4J for one week, with no luck.

It would be great to mention that all "build from Source" documentation is out of date or the web link is down.
Really hard.

[AlexDBlack]

@willishf

PS C:\deeplearning\deeplearning4j-examples\dl4j-examples> mvn install

You're trying to build only one of the modules, without building the examples' shared utilities module first.
So the error you're getting is reasonable/correct. Install the shared module first, or build the entire examples project.

@Bfzanchetta Most people don't need to build from source, and should use the releases on Maven central via a build such as maven or gradle.
Alternatively if you need fixes implemented after the last release, you can use snapshots: https://deeplearning4j.org/docs/latest/deeplearning4j-config-snapshots

The only reason to build from source is adding custom functionality excluding things like new layers, etc, which you can just do in your own project. Is this what you are doing? If so - provide some more details, and let us know what are you stuck on.

[eraly]

@willishf The quickstart needs to be updated. Sorry about that. We are in midst of a big transition. Do the following and things should work fine! Thank you.

git clone https://github.com/eclipse/deeplearning4j-examples
cd shared-utilities
mvn install
cd ../dl4j-examples
mvn install

@Bfzanchetta The team is in the midst of some big transitions. We will be updating our docs and such soon. I don't see you on our gitter channel. Feel free to join us there. In general we discourage users from building from source. But if you absolutely you can either bring them up in the gitter channel or file an issue describing it. I am not sure if our CI is back up yet. If it's not it will be soon and there will be snapshots available of the latest master. Thank you for your patience!

[eraly]

Whoops crossed paths with Alex.

[willishf]

Same error but but looks resolvable. Any magic commands to run to force
recognition of the file.

PS C:\deeplearning\deeplearning4j-examples\shared-utilities> mvn install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------< org.deeplearning4j.examples:shared-utilities

---

[INFO] Building shared-utilities 1.0.0-beta5
[INFO] --------------------------------[ jar
]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.0.1:enforce (enforce-default) @
shared-utilities ---
[INFO]
[INFO] --- lint-maven-plugin:0.0.11:check (pom-lint) @ shared-utilities ---
[INFO] Writing summary report
[INFO] [LINT] Completed with no violations
[INFO] Writing xml report
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
shared-utilities ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
C:\deeplearning\deeplearning4j-examples\shared-utilities\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.1:compile (default-compile) @
shared-utilities ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to
C:\deeplearning\deeplearning4j-examples\shared-utilities\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources)
@ shared-utilities ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
C:\deeplearning\deeplearning4j-examples\shared-utilities\src\test\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
shared-utilities ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @
shared-utilities ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ shared-utilities ---
[INFO] Building jar:
C:\deeplearning\deeplearning4j-examples\shared-utilities\target\shared-utilities-1.0.0-beta5.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @
shared-utilities ---
[INFO] Installing
C:\deeplearning\deeplearning4j-examples\shared-utilities\target\shared-utilities-1.0.0-beta5.jar
to C:\Users\Scooter
Willis.m2\repository\org\deeplearning4j\examples\shared-utilities\1.0.0-beta5\shared-utilities-1.0.0-beta5.jar
[INFO] Installing
C:\deeplearning\deeplearning4j-examples\shared-utilities\pom.xml to
C:\Users\Scooter
Willis.m2\repository\org\deeplearning4j\examples\shared-utilities\1.0.0-beta5\shared-utilities-1.0.0-beta5.pom

[INFO]

[INFO] BUILD SUCCESS

[INFO]

[INFO] Total time: 3.478 s
[INFO] Finished at: 2019-10-21T19:14:24-04:00

[INFO]

PS C:\deeplearning\deeplearning4j-examples\shared-utilities> cd ..
PS C:\deeplearning\deeplearning4j-examples> cd .\dl4j-examples\
PS C:\deeplearning\deeplearning4j-examples\dl4j-examples> ls

Directory: C:\deeplearning\deeplearning4j-examples\dl4j-examples

Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 10/21/2019 11:42 AM src
-a---- 10/21/2019 11:42 AM 827 .gitignore
-a---- 10/21/2019 11:42 AM 2615 build.gradle
-a---- 10/21/2019 11:42 AM 11308 pom.xml
-a---- 10/21/2019 11:42 AM 557 README.md

PS C:\deeplearning\deeplearning4j-examples\dl4j-examples> mvn install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.deeplearning4j:dl4j-examples

---

[INFO] Building DeepLearning4j Examples 1.0.0-beta5
[INFO] --------------------------------[ jar
]---------------------------------
Downloading from central:
https://repo.maven.apache.org/maven2/org/deeplearning4j/deeplearning4j-examples-parent/1.0.0-beta5/deeplearning4j-examples-parent-1.0.0-beta5.pom

[INFO]

[INFO] BUILD FAILURE

[INFO]

[INFO] Total time: 2.961 s
[INFO] Finished at: 2019-10-21T19:15:02-04:00

[INFO]

[ERROR] Failed to execute goal on project dl4j-examples: Could not resolve
dependencies for project org.deeplearning4j:dl4j-examples:jar:1.0.0-beta5:
Failed to collect dependencies at
org.deeplearning4j.examples:shared-utilities:jar:1.0.0-beta5: Failed to
read artifact descriptor for
org.deeplearning4j.examples:shared-utilities:jar:1.0.0-beta5: Could not
find artifact
org.deeplearning4j:deeplearning4j-examples-parent:pom:1.0.0-beta5 in
central (https://repo.maven.apache.org/maven2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

On Mon, Oct 21, 2019 at 7:11 PM Susan Eraly [e-mail]
wrote:

@willishf https://github.com/willishf The quickstart needs to be
updated. Sorry about that. We are in midst of a big transition. Do the
following and things should work fine! Thank you.

git clone https://github.com/eclipse/deeplearning4j-examples
cd shared-utilities
mvn install
cd ../dl4j-examples
mvn install

@Bfzanchetta https://github.com/Bfzanchetta The team is in the midst of
some big transitions. We will be updating our docs and such soon. I don't
see you on our gitter channel. Feel free to join us there
https://gitter.im/deeplearning4j/deeplearning4j. In general we
discourage users from building from source. But if you absolutely you can
either bring them up in the gitter channel or file an issue describing it.
I am not sure if our CI is back up yet. If it's not it will be soon and
there will be snapshots available of the latest master. Thank you for your
patience!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/eclipse/deeplearning4j/issues/8307?email_source=notifications&email_token=AA653HWDIAKILBKU5XCX3X3QPYZKBA5CNFSM4JDAKDF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB4CQWI#issuecomment-544745561,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA653HSBVSYM3EVVASV5WMTQPYZKBANCNFSM4JDAKDFQ
.

[eraly]

I would just try again. It should work.. Check your .m2 repo for the shared-utilities jar.

[willishf]

The jar file appears to be there

[image: image.png]

Details on MVN install

mvn -version
Apache Maven 3.6.2 (40f52333136460af0dc0d7232c0dc0bcf0d9e117;
2019-08-27T11:06:16-04:00)
Maven home: C:\java\apache-maven-3.6.2\bin..
Java version: 1.8.0_231, vendor: Oracle Corporation, runtime: C:\Program
Files\Java\jdk1.8.0_231\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

Still getting the same error message where not familiar with maven enough
to interpret error messages.

PS C:\deeplearning\deeplearning4j-examples\dl4j-examples> mvn install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.deeplearning4j:dl4j-examples

---

[INFO] Building DeepLearning4j Examples 1.0.0-beta5
[INFO] --------------------------------[ jar
]---------------------------------

[INFO]

[INFO] BUILD FAILURE

[INFO]

[INFO] Total time: 1.869 s
[INFO] Finished at: 2019-10-22T19:24:03-04:00

[INFO]

[ERROR] Failed to execute goal on project dl4j-examples: Could not resolve
dependencies for project org.deeplearning4j:dl4j-examples:jar:1.0.0-beta5:
Failed to collect dependencies at
org.deeplearning4j.examples:shared-utilities:jar:1.0.0-beta5: Failed to
read artifact descriptor for
org.deeplearning4j.examples:shared-utilities:jar:1.0.0-beta5: Failure to
find org.deeplearning4j:deeplearning4j-examples-parent:pom:1.0.0-beta5 in
https://repo.maven.apache.org/maven2 was cached in the local repository,
resolution will not be reattempted until the update interval of central has
elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

On Tue, Oct 22, 2019 at 3:30 PM Susan Eraly [e-mail]
wrote:

I would just try again. It should work.. Check your .m2 repo for the
shared-utilities jar.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/eclipse/deeplearning4j/issues/8307?email_source=notifications&email_token=AA653HQLHEWBNEY4NXYVORDQP5IF5A5CNFSM4JDAKDF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB65EXQ#issuecomment-545116766,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA653HUHRTNGPUBYIIYD33DQP5IF5ANCNFSM4JDAKDFQ
.

[eraly]

Can you paste the image?

[willishf]

[image: image.png]

PS C:\deeplearning\deeplearning4j-examples> cd .\dl4j-examples\
PS C:\deeplearning\deeplearning4j-examples\dl4j-examples> mvn install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.deeplearning4j:dl4j-examples

---

[INFO] Building DeepLearning4j Examples 1.0.0-beta5
[INFO] --------------------------------[ jar
]---------------------------------

[INFO]

[INFO] BUILD FAILURE

[INFO]

[INFO] Total time: 1.869 s
[INFO] Finished at: 2019-10-22T19:24:03-04:00

[INFO]

[ERROR] Failed to execute goal on project dl4j-examples: Could not resolve
dependencies for project org.deeplearning4j:dl4j-examples:jar:1.0.0-beta5:
Failed to collect dependencies at
org.deeplearning4j.examples:shared-utilities:jar:1.0.0-beta5: Failed to
read artifact descriptor for
org.deeplearning4j.examples:shared-utilities:jar:1.0.0-beta5: Failure to
find org.deeplearning4j:deeplearning4j-examples-parent:pom:1.0.0-beta5 in
https://repo.maven.apache.org/maven2 was cached in the local repository,
resolution will not be reattempted until the update interval of central has
elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

On Wed, Oct 23, 2019 at 1:57 PM Susan Eraly [e-mail]
wrote:

Can you paste the image?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/eclipse/deeplearning4j/issues/8307?email_source=notifications&email_token=AA653HQBIN5CDW3G26G2BM3QQCF7VA5CNFSM4JDAKDF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECCJ6GY#issuecomment-545562395,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA653HWN36MY56C76HFP3ZLQQCF7VANCNFSM4JDAKDFQ
.

[willishf]

Deleted the directory and tried again this time building the shared-utilities directory first. Same error.

[AlexDBlack]

So, following up here

I tested locally, turns out the problem here is just the missing parent project installation
try this:

mvn clean install -DskipTests -pl org.deeplearning4j:deeplearning4j-examples-parent
cd shared-utilities
mvn clean install -DskipTests
cd ../dl4j-examples
mvn clean package -DskipTests

Without doing the parent install (first line) the last line (dl4j-examples package) will fail.
Looks like a maven bug / edge case.

Only workaround that I can see for that is to get rid of the shared-utilities module. That would result in code duplication, but should make it easier to build just one module in isolation.
Though longer term I want to go to single module examples anyway...