Testcontainers-java: Cannot start container due to ClassNotFoundException (JsonMerge)

[zabetak]

Version used:

<dependency>
  <groupId>org.testcontainers</groupId>
  <artifactId>testcontainers</artifactId>
  <version>1.15.0-rc2</version>
</dependency>

When I attempt to run the test, I get the following exception:

2020-11-04T11:21:36,470  INFO [main] dockerclient.DockerClientProviderStrategy: Loaded org.testcontainers.dockerclient.UnixSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first
2020-11-04T11:21:36,724  INFO [main] dockerclient.DockerMachineClientProviderStrategy: docker-machine executable was not found on PATH ([/opt/jdks/jdk1.8.0_261/bin, /home/stamatis/Programs/apache-maven-3.6.3/bin, /home/stamatis/Projects/Misc/Hive-scripts/, /home/stamatis/.local/bin, /usr/local/sbin, /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin, /usr/games, /usr/local/games, /snap/bin])
2020-11-04T11:21:36,754 ERROR [main] dockerclient.DockerClientProviderStrategy: Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
2020-11-04T11:21:36,754 ERROR [main] dockerclient.DockerClientProviderStrategy:     UnixSocketClientProviderStrategy: failed with exception NoClassDefFoundError (com/fasterxml/jackson/annotation/JsonMerge). Root cause ClassNotFoundException (com.fasterxml.jackson.annotation.JsonMerge)
2020-11-04T11:21:36,754 ERROR [main] dockerclient.DockerClientProviderStrategy:     UnixSocketClientProviderStrategy: failed with exception NoClassDefFoundError (Could not initialize class org.testcontainers.shaded.com.github.dockerjava.core.DefaultObjectMapperHolder)
2020-11-04T11:21:36,754 ERROR [main] dockerclient.DockerClientProviderStrategy: As no valid configuration was found, execution cannot continue
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.516 sec <<< FAILURE!
testQueries(path.to.my.Class)  Time elapsed: 0.308 sec  <<< ERROR!
org.testcontainers.containers.ContainerLaunchException: Container startup failed
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:331)
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:312)
    at org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:1025)
    at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image: RemoteDockerImage(imageName=com.company/mini:1.10.0, imagePullPolicy=DefaultPullPolicy())
    at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1282)
    at org.testcontainers.containers.GenericContainer.logger(GenericContainer.java:616)
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:321)
    ... 25 more
Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$7(DockerClientProviderStrategy.java:214)
    at java.util.Optional.orElseThrow(Optional.java:290)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:206)
    at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:132)
    at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:174)
    at org.testcontainers.LazyDockerClient.getDockerClient(LazyDockerClient.java:14)
    at org.testcontainers.LazyDockerClient.listImagesCmd(LazyDockerClient.java:12)
    at org.testcontainers.images.LocalImagesCache.maybeInitCache(LocalImagesCache.java:68)
    at org.testcontainers.images.LocalImagesCache.get(LocalImagesCache.java:32)
    at org.testcontainers.images.AbstractImagePullPolicy.shouldPull(AbstractImagePullPolicy.java:18)
    at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:65)
    at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:26)
    at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:17)
    at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:39)
    at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1280)
    ... 27 more

Obviously there is a dependency conflict of jackson-annotation that is used by other libraries used in the project. What worked for me was to declare explicitly jackson-annotations on my pom.xml and use a recent version.

<dependency>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-annotations</artifactId>
  <version>2.10.5</version>
</dependency>

I just submitted the issue just in case it is useful to somebody else.

[bsideup]

If your project forces (because normally build systems will prefer the newer version) an older version of any dependency that we're using it is obvious that things may break.

Since there is no action to be taken on our side, I am going to close this issue.

[zabetak]

Well maven for instance does not use the newer version for transitive deps but the one that is closer (https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html) so you can easily get an older version that's incompatible.

Anyways I agree that it is not problem on your side but it is useful to have the stacktrace and explanation here for people googling around.

[bsideup]

@zabetak thanks, I actually missed this piece of info! 👍

What can be done is lowering the Jackson dependency version requirement in docker-java from where it comes from - WDYT about submitting a UX issue there?

[zabetak]

In my case, I can use the newer Jackson dependency in the other projects so I don't find a strong reason to ask for a downgrade. Maven provides the mean to ask for which dependency you want to use so I consider the problem solved.

Many thanks for your time @bsideup !

[bsideup]

@zabetak thanks for reporting! I created https://github.com/docker-java/docker-java/issues/1485, just in case

[marcusphi]

I am getting same error:

org.testcontainers.dockerclient.DockerClientProviderStrategy - Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
org.testcontainers.dockerclient.DockerClientProviderStrategy -     UnixSocketClientProviderStrategy: failed with exception NoClassDefFoundError (com/fasterxml/jackson/annotation/JsonMerge). Root cause ClassNotFoundException (com.fasterxml.jackson.annotation.JsonMerge)
org.testcontainers.dockerclient.DockerClientProviderStrategy -     UnixSocketClientProviderStrategy: failed with exception NoClassDefFoundError (Could not initialize class org.testcontainers.shaded.com.github.dockerjava.core.DefaultObjectMapperHolder)
org.testcontainers.dockerclient.DockerClientProviderStrategy - As no valid configuration was found, execution cannot continue

In the description of this issue it's stated:

Obviously there is a dependency conflict of jackson-annotation that is used by other libraries used in the project.

Obviously! Really? Are you sure?

I do not think I have any other dependency pulling in another version of jackson-annotations. I think it's coming from testcontainers. I have run some maven commands to verify this. As far as I can tell the dependecy chain is: org.testcontainers:testcontainers:1.15.0 -> com.github.docker-java:docker-java-api:3.2.5 -> com.fasterxml.jackson.core:jackson-annotations:2.8.0. And jackson-annotations:2.8.0 does not have JsonMerge class, it seems to have been added in 2.9.

After adding

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-annotations</artifactId>
    <version>2.9.10</version>
</dependency>

to my <dependencyManagement> section it works.

I dopn't fully understand this, but as far as I can tell this means testcontainers doesn't work out of the box, in a plain vanilla project with no dependencies. Shouldn't testcontainers declare jackson-annotations:2.9 if it depends on it?

So I think this issue should be reopened and fixed.

[bsideup]

@marcusphi docker-java-api depends on jackson-annotations version 2.10.3:
https://mvnrepository.com/artifact/com.github.docker-java/docker-java-api/3.2.5

I dopnät fully understand this, but as far as I can tell this means testcontainers doesn't work out of the box, in a plain vanilla project with no dependencies. Shouldn't testcontainers declare jackson-annotations:2.9 if it depends on it?

Unless you can provide a reproducer, I don't think that's the case.

[marcusphi]

You are right @bsideup. I thought I knew how to use maven, but it seems not 😄 I really hate maven sometimes. But I guess not having it would be worse. Version was managed snuck in from a parent dependencyManagement section.

[bsideup]

@marcusphi well, the good thing about Maven is that it is declarative and it is more or less easy to find from where certain version comes from :)

We will consider lowering Jackson, so that even old projects work fine with it, since we don't need most of its new features. But, for now, you should be able to "fix" it by forcing the correct version of the dependency 👍

[energister]

Alternative solution would be to use testcontainers 1.14.3 which solves Ryuk container availability issue and depends on old enough jackson-annotations version at the same time.