Testcontainers-java: Dependency conflict for jna (with testcontainers:1.15.0-rc2)

[gesellix]

Using the current testcontainers-core release candidate triggers the following dependency conflict:

./gradlew :dependencyInsight --configuration testCompileClasspath --dependency net.java.dev.jna:jna

> Task :dependencyInsight
Dependency resolution failed because of conflict(s) on the following module(s):
   - net.java.dev.jna:jna between versions 5.5.0 and 5.2.0

net.java.dev.jna:jna:5.5.0
   variant "compile" [
      org.gradle.status              = release (not requested)
      org.gradle.usage               = java-api
      org.gradle.libraryelements     = jar (compatible with: classes+resources)
      org.gradle.category            = library

      Requested attributes not found in the selected variant:
         org.gradle.dependency.bundling = external
         org.gradle.jvm.version         = 14
   ]
   Selection reasons:
      - By conflict resolution : between versions 5.5.0 and 5.2.0

net.java.dev.jna:jna:5.5.0
\--- com.github.docker-java:docker-java-transport-zerodep:3.2.5
     \--- org.testcontainers:testcontainers:1.15.0-rc2
          \--- org.testcontainers:spock:1.15.0-rc2
               \--- testCompileClasspath

net.java.dev.jna:jna:5.2.0 -> 5.5.0
\--- org.rnorth.visible-assertions:visible-assertions:2.1.2
     \--- org.testcontainers:testcontainers:1.15.0-rc2
          \--- org.testcontainers:spock:1.15.0-rc2
               \--- testCompileClasspath

I'm unsure whether the fix should be in docker-java-transport-zerodep to not expose jna or if jna should be excluded for visible-assertions. Maybe you have even better solutions. So this time no instant pull request, but I'm happy to create one for however and wherever you prefer the fix to be :)

[gesellix]

The change in #2882 seems to be the relevant bit: https://github.com/testcontainers/testcontainers-java/commit/1c97b978ac6ccfba0119986c22e4ac6d9ac955fa#diff-26fb43127aab27a5a6df14164a6f424aR173

[bsideup]

@gesellix since docker-java knows nothing about visible-assertions, I'd say the fix should go to Testcontainers. Although I'd prefer to see the fix coming together with some check that will verify such issues in future, because that's not something we were caring before (since it doesn't break anything), but if people _need_ such alignment then let's do it :)

[dmeiners88]

Maybe https://github.com/kordamp/enforcer-gradle-plugin is worth a look for automating the dependency convergence check in the future, specifically https://kordamp.org/enforcer-gradle-plugin/#_dependencyconvergence.

[gorodum]

In 1.15.0 also, not RC

Dependency convergence error for net.java.dev.jna:jna:5.2.0 paths to dependency are:
+-com.sap.fsm:cloud-poc:2.3.0
  +-org.testcontainers:testcontainers:1.15.0
    +-org.rnorth.visible-assertions:visible-assertions:2.1.2
      +-net.java.dev.jna:jna:5.2.0
and
+-com.sap.fsm:cloud-poc:2.3.0
  +-org.testcontainers:testcontainers:1.15.0
    +-com.github.docker-java:docker-java-transport-zerodep:3.2.5
      +-net.java.dev.jna:jna:5.5.0

[jonasfranz]

I'm currently using this workaround to prevent a failure of the maven enforcer plugin. It seems to work as far as I can see. But it would be nice if we can solve this directly in the library.

Add this to your parent project's pom.xml:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>net.java.dev.jna</groupId>
            <artifactId>jna</artifactId>
            <version>5.5.0</version>
        </dependency>
    </dependencies>
</dependencyManagement>

Warning: This will force the usage of jna 5.5.0 for all dependencies using jna.

[lgnashold]

I am also experiencing this issue. Is there a workaround for Gradle? I tried using a lower version of test containers (1.14.3), but I ran into a different issue.

[JaanaTwt]

I face this issue in testcontainers version 1.15.2. Any plans for a fix?

[gesellix]

@bsideup @rnorth is the org.rnorth.visible-assertions:visible-assertions dependency required on compile scope? I wouldn't like to go down the rabbit hole of managing transitive dependencies for downstream users, when it would be possible to keep the conflicting dependencies local to the Testcontainers build. As far as I can see from a quick glance, the visible-assertions dependency is only required for the Testcontainers checks, so we might change the scope to testCompile. Such a change might break existing users, though, if they use the visible-assertions lib themselves.

More explicitly speaking, I propose changes like below to the published pom.xml of testcontainers-core:

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.12</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
      <version>1.7.30</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>commons-compress</artifactId>
      <version>1.20</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.rnorth.duct-tape</groupId>
      <artifactId>duct-tape</artifactId>
      <version>1.0.8</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <groupId>org.jetbrains</groupId>
          <artifactId>annotations</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
-    <dependency>
-      <groupId>org.rnorth.visible-assertions</groupId>
-      <artifactId>visible-assertions</artifactId>
-      <version>2.1.2</version>
-      <scope>compile</scope>
-    </dependency>
    <dependency>
      <groupId>com.github.docker-java</groupId>
      <artifactId>docker-java-api</artifactId>
      <version>3.2.7</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>com.github.docker-java</groupId>
      <artifactId>docker-java-transport-zerodep</artifactId>
      <version>3.2.7</version>
      <scope>compile</scope>
    </dependency>
  </dependencies>

[gesellix]

... Such a change should be combined with a config like this:

configurations.all {
    resolutionStrategy {
+        failOnVersionConflict()
        // use lower Jackson version
        force("com.fasterxml.jackson.core:jackson-databind:2.8.8")
+        force("org.slf4j:slf4j-api:1.7.30")
+        force("net.java.dev.jna:jna:5.5.0")
    }
}

[gesellix]

Mh. Answering myself: visible-assertions is required in GenericContainer.

[dagguh]

We care about dependency conflicts, because hiding them (the default Maven/Gradle behavior) leads to runtime bugs like NoSuchMethodError or broken behavioral contracts. We use failOnVersionConflict in every Gradle build.

A few ways to reduce transitive conflicts:

• don't include unnecessary dependencies
• reduce the scope of dependencies as much as possible (e.g. provided vs compile in Maven or implementation vs api in Gradle)
• use version ranges for increased compatibility (this requires some compatibility contract from the dependency maintainers, preferably SemVer)

In this case, com.github.docker-java:docker-java-transport-zerodep:3.2.7 requires net.java.dev.jna:jna:5.5.0 while org.rnorth.visible-assertions:visible-assertions:2.1.2 requires net.java.dev.jna:jna:5.2.0. By default Gradle would pick the higher one: 5.5.0. So if this works, and it doesn't break visible-assertions, then the conflict is bogus. Therefore, the requirement inside visible-assertions is too strict. If jna declares their compatibility (e.g. they follow SemVer), then net.java.dev.jna:jna:[5.2.0, 6.0.0) and net.java.dev.jna:jna:[5.5.0, 6.0.0) would not generate a bogus conflict.