Deeplearning4j: "ReflectionsException: could not create Vfs.Dir from url …" when restoring multilayer network

[beatngu13]

Using the following pom.xml configuration:

<build>
    <plugins>

        <plugin>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.5.1</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>

    </plugins>
</build>

<dependencies>

    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>1.7.25</version>
    </dependency>

    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-simple</artifactId>
        <version>1.7.25</version>
    </dependency>

    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-core</artifactId>
        <version>0.8.0</version>
    </dependency>

    <dependency>
        <groupId>org.nd4j</groupId>
        <artifactId>nd4j-native-platform</artifactId>
        <version>0.8.0</version>
    </dependency>

</dependencies>

And the following code:

ModelSerializer.restoreMultiLayerNetwork( Example.class.getResourceAsStream( "/model.zip" ) );

I'm getting a bunch of these warnings (revealed through SLF4J):

[main] WARN org.reflections.Reflections - could not create Vfs.Dir from url. ignoring the exception and continuing
org.reflections.ReflectionsException: could not create Vfs.Dir from url, no matching UrlType was found [file:/System/Library/Java/Extensions/libAppleScriptEngine.jnilib]
either use fromURL(final URL url, final List<UrlType> urlTypes) or use the static setDefaultURLTypes(final List<UrlType> urlTypes) or addDefaultURLTypes(UrlType urlType) with your specialized UrlType.
    at org.reflections.vfs.Vfs.fromURL(Vfs.java:109)
    at org.reflections.vfs.Vfs.fromURL(Vfs.java:91)
    at org.reflections.Reflections.scan(Reflections.java:237)
    at org.reflections.Reflections.scan(Reflections.java:204)
    at org.reflections.Reflections.<init>(Reflections.java:129)
    at org.deeplearning4j.nn.conf.NeuralNetConfiguration.registerSubtypes(NeuralNetConfiguration.java:431)
    at org.deeplearning4j.nn.conf.NeuralNetConfiguration.configureMapper(NeuralNetConfiguration.java:386)
    at org.deeplearning4j.nn.conf.NeuralNetConfiguration.initMapper(NeuralNetConfiguration.java:376)
    at org.deeplearning4j.nn.conf.NeuralNetConfiguration.<clinit>(NeuralNetConfiguration.java:123)
    at org.deeplearning4j.nn.conf.MultiLayerConfiguration.fromJson(MultiLayerConfiguration.java:124)
    at org.deeplearning4j.util.ModelSerializer.restoreMultiLayerNetwork(ModelSerializer.java:239)
    at org.deeplearning4j.util.ModelSerializer.restoreMultiLayerNetwork(ModelSerializer.java:267)
    at org.deeplearning4j.util.ModelSerializer.restoreMultiLayerNetwork(ModelSerializer.java:271)
    at com.github.beatngu13.dl4jplayground.Example.main(Example.java:10)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

I'm on macOS Sierra (10.12.5) and JDK 8 (1.8.0_121-b13).

[loretoparisi]

I'm having the same issue on macOS Sierra here.

[agibsonccc]

Latest version though? Also, this is a warning. In practice it shouldn't be stopping your jobs.

[beatngu13]

@agibsonccc yep, still happens with v0.9.1. But as you said, it is a warning and only pollutes the log file. I just thought it may be relevant internally or considered as a fix in the long term.

[loretoparisi]

@agibsonccc yep! latest version v0.9.1 and yes I confirm it does not stop anything. Every function works as expected, but tons of logging btw.

[AlexDBlack]

To confirm, this is just due to the reflections library, and can be ignored (or, just set your logging configuration so it doesn't show up).
The reflections library will be going away in the next release too.

[Dathiou]

Is this issue related to this as well? (it occurs right before the warning mentioned above)

17/10/15 16:48:48 WARN Reflections: could not create Dir using directory from url file:/usr/share/aws/emr/emrfs/auxlib/*. skipping.
java.lang.NullPointerException
    at org.reflections.vfs.Vfs$DefaultUrlTypes$3.matches(Vfs.java:239)
    at org.reflections.vfs.Vfs.fromURL(Vfs.java:98)
    at org.reflections.vfs.Vfs.fromURL(Vfs.java:91)
    at org.reflections.Reflections.scan(Reflections.java:237)
    at org.reflections.Reflections.scan(Reflections.java:204)
    at org.reflections.Reflections.<init>(Reflections.java:129)
    at org.reflections.Reflections.<init>(Reflections.java:170)
    at org.nd4j.versioncheck.VersionCheck.listGitPropertiesFiles(VersionCheck.java:201)
    at org.nd4j.versioncheck.VersionCheck.getVersionInfos(VersionCheck.java:220)
    at org.nd4j.versioncheck.VersionCheck.checkVersions(VersionCheck.java:92)
    at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:6098)
    at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:6087)
    at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:201)
    at org.nd4j.Nd4jRegistrator.registerClasses(Nd4jRegistrator.java:20)
    at org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(KryoSerializer.scala:125)
    at org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(KryoSerializer.scala:125)

I am having container failures and I wonder if it is not related to this warning. I filed:
https://github.com/deeplearning4j/deeplearning4j/issues/4181

[AlexDBlack]

Closing this - again, configure logging and you won't see it. Reflections library (that is the underlying cause) is still planned to be removed.

[loretoparisi]

@AlexDBlack ok thanks. Could you please explain to me how to properly configure the logging?
Thank you.

[hy-2013]

I have the similar bug as @Dathiou. @AlexDBlack can you explain it and solve the bug?

[agibsonccc]

@hy-2013 I'm blocking comments on this. This for the 3rd time now is not a bug. It's a warning. Your logging is relative to your slf4j implementation on the backend.