Testcontainers-java: "Could not find a valid Docker environment" error while running tests from Jenkins on the Ubuntu

Created on 16 Mar 2018 · 22Comments · Source: testcontainers/testcontainers-java

The environment:

Host machine: Ubuntu 16.04 with Docker installed and running;
Jenkins which is run via Docker container on the same host machine.

I am using testcontainers-java in my project and trying to setup executing SeleniumWebdriver tests via Maven from the Jenkins CI. Example code is taken from the official documentation and works perfectly if I run tests from IDE installed on the same Linux host machine, or if I run tests from a terminal via Maven - testcontainers-java starts an appropriate container and I receive correct data from the browser under test.

However, when I try to run the same tests with the same set of Maven commands from a Jenkins job - I receive the following error:

java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$2(DockerClientProviderStrategy.java:146) at java.util.Optional.orElseThrow(Optional.java:290) at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:138) at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:99) at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:126) at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:166) at org.testcontainers.containers.BrowserWebDriverContainer.<init>(BrowserWebDriverContainer.java:61)

I suspect that there is maybe some issue because Jenkins itself is running inside another container and cannot get access to the installed Docker instance from inside. Is there any way to solve this issue? I was trying to search for a solution a lot but didn't succeed. This forum is my last hope

clienin-container olinux resolutioinvalid typquestion

Source

endless-qa

Most helpful comment

So, the final solution that works for me:

Add jenkins user to the docker group: sudo usermod -aG docker jenkins
Reboot a host machine. Optionally, you can open the terminal under the jenkins user and run the groups command to make sure that there is the docker group in the list.

After that Jenkins is able to deal with the docker.sock file and TestContainers can see the Docker installation.

endless-qa on 23 Mar 2018

👍10 ❤5

All 22 comments

Hi @endless-qa,

Did you perform the steps from here?
https://www.testcontainers.org/usage/inside_docker.html

bsideup on 16 Mar 2018

Thanks for quick response!
I've just tried mapping from your link. But additionally to the previous error, I got another one:

WARNING: Force-closing a channel whose registration task was not accepted by an event loop: [id: 0x253704aa] java.util.concurrent.RejectedExecutionException: event executor terminated at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:821) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:327) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:320) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:746) at org.testcontainers.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:475) at org.testcontainers.shaded.io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:80) at org.testcontainers.shaded.io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:74) at org.testcontainers.shaded.io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:86) at org.testcontainers.shaded.io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:332) at org.testcontainers.shaded.io.netty.bootstrap.Bootstrap.doResolveAndConnect(Bootstrap.java:163) at org.testcontainers.shaded.io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:145)

And then the same exception:

endless-qa on 16 Mar 2018

The command that I'm using to start Jenkins via Docker is a standard one:

docker run -p 8080:8080 -p 50000:50000 -v /your/home:/var/jenkins_home jenkins

So, I added volume mapping as suggested in the Documentation and now it looks like:

docker run -p 8080:8080 -p 50000:50000 -v $PWD:$PWD -w $PWD -v /var/run/docker.sock:/var/run/docker.sock -v /your/home:/var/jenkins_home jenkins

However, I get 2 exceptions as I described above.

endless-qa on 16 Mar 2018

@endless-qa could you please provide the logs?

bsideup on 16 Mar 2018

@bsideup
The only log I have is this one (Jenkins job output):

Running DemoTest SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Mar 16, 2018 6:54:35 AM org.testcontainers.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe register WARNING: Force-closing a channel whose registration task was not accepted by an event loop: [id: 0xe7241868] java.util.concurrent.RejectedExecutionException: event executor terminated at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:821) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:327) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:320) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:746) at org.testcontainers.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:475) at org.testcontainers.shaded.io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:80) at org.testcontainers.shaded.io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:74) at org.testcontainers.shaded.io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:86) at org.testcontainers.shaded.io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:332) at org.testcontainers.shaded.io.netty.bootstrap.Bootstrap.doResolveAndConnect(Bootstrap.java:163) at org.testcontainers.shaded.io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:145) at com.github.dockerjava.netty.NettyDockerCmdExecFactory$UnixDomainSocketInitializer.connect(NettyDockerCmdExecFactory.java:276) at com.github.dockerjava.netty.NettyDockerCmdExecFactory.connect(NettyDockerCmdExecFactory.java:220) at com.github.dockerjava.netty.NettyDockerCmdExecFactory.connect(NettyDockerCmdExecFactory.java:213) at com.github.dockerjava.netty.NettyDockerCmdExecFactory.access$000(NettyDockerCmdExecFactory.java:157) at com.github.dockerjava.netty.NettyDockerCmdExecFactory$1.getChannel(NettyDockerCmdExecFactory.java:183) at com.github.dockerjava.netty.InvocationBuilder.getChannel(InvocationBuilder.java:213) at com.github.dockerjava.netty.InvocationBuilder.get(InvocationBuilder.java:507) at com.github.dockerjava.netty.exec.PingCmdExec.execute(PingCmdExec.java:23) at com.github.dockerjava.netty.exec.PingCmdExec.execute(PingCmdExec.java:10) at com.github.dockerjava.netty.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21) at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35) at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$null$3(DockerClientProviderStrategy.java:169) at org.rnorth.ducttape.ratelimits.RateLimiter.getWhenReady(RateLimiter.java:51) at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$ping$4(DockerClientProviderStrategy.java:167) at org.rnorth.ducttape.unreliables.Unreliables.lambda$retryUntilSuccess$0(Unreliables.java:41) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 20.711 sec <<< FAILURE! demoTest(DemoTest) Time elapsed: 0.007 sec <<< ERROR! java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$2(DockerClientProviderStrategy.java:146) at java.util.Optional.orElseThrow(Optional.java:290) at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:138) at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:99) at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:126) at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:166) at org.testcontainers.containers.BrowserWebDriverContainer.<init>(BrowserWebDriverContainer.java:61) at DemoTest.<init>(DemoTest.java:9) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217) at org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Results : Tests in error: demoTest(DemoTest): Could not find a valid Docker environment. Please see logs and check configuration Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

There is nothing in the running Jenkins server logs as well as in the Docker logs for the Jenkins container. Could you please point me where I can find any additional logs that may be useful for investigation?

endless-qa on 16 Mar 2018

@endless-qa please configure the logs as recommended here:
https://www.testcontainers.org/usage.html#logging

Do not forget to add Slf4j-compatible logging implementation (we recommend Logback)

bsideup on 16 Mar 2018

@endless-qa Please also keep in mind, that the docker socket needs to be actually mounted into the Jenkins build slaves.

kiview on 16 Mar 2018

Sorry for the delay. Here is the log with added logging as per your suggestion:

[main] ERROR org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy - ping failed with configuration Environment variables, system properties and defaults. Resolved: dockerHost=unix:///var/run/docker.sock apiVersion='{UNKNOWN_VERSION}' registryUrl='https://index.docker.io/v1/' registryUsername='jenkins' registryPassword='null' registryEmail='null' dockerConfig='DefaultDockerClientConfig[dockerHost=unix:///var/run/docker.sock,registryUsername=jenkins,registryPassword=<null>,registryEmail=<null>,registryUrl=https://index.docker.io/v1/,dockerConfig=/var/jenkins_home/.docker,sslConfig=<null>,apiVersion={UNKNOWN_VERSION}]' due to org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:51) at org.testcontainers.dockerclient.DockerClientProviderStrategy.ping(DockerClientProviderStrategy.java:166) at org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy.test(EnvironmentAndSystemPropertyClientProviderStrategy.java:42) at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$1(DockerClientProviderStrategy.java:104) at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:302) at java.util.stream.Streams$ConcatSpliterator.tryAdvance(Streams.java:731) at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126) at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.findAny(ReferencePipeline.java:469) at
org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:137) at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:99) at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:126) at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:166) at org.testcontainers.containers.BrowserWebDriverContainer.<init>(BrowserWebDriverContainer.java:61) at DemoTest.<init>(DemoTest.java:9) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217) at org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) Caused by: org.testcontainers.shaded.io.netty.channel.AbstractChannel$AnnotatedConnectException: connect(..) failed: Permission denied: /var/run/docker.sock at org.testcontainers.shaded.io.netty.channel.unix.Errors.throwConnectException(Errors.java:107) at org.testcontainers.shaded.io.netty.channel.unix.Socket.connect(Socket.java:243) at org.testcontainers.shaded.io.netty.channel.epoll.AbstractEpollStreamChannel.doConnect(AbstractEpollStreamChannel.java:752) at org.testcontainers.shaded.io.netty.channel.epoll.EpollDomainSocketChannel.doConnect(EpollDomainSocketChannel.java:87) at org.testcontainers.shaded.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.connect(AbstractEpollStreamChannel.java:815) at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1266) at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:545) at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:530) at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.connect(CombinedChannelDuplexHandler.java:497) at org.testcontainers.shaded.io.netty.channel.ChannelOutboundHandlerAdapter.connect(ChannelOutboundHandlerAdapter.java:47) at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler.connect(CombinedChannelDuplexHandler.java:298) at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:545) at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:530) at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:512) at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline.connect(DefaultChannelPipeline.java:985) at org.testcontainers.shaded.io.netty.channel.AbstractChannel.connect(AbstractChannel.java:255) at org.testcontainers.shaded.io.netty.bootstrap.Bootstrap$3.run(Bootstrap.java:252) at org.testcontainers.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) at org.testcontainers.shaded.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:311) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) at org.testcontainers.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.ConnectException: connect(..) failed: Permission denied ... 23 more [pool-1-thread-1] WARN org.testcontainers.shaded.io.netty.channel.AbstractChannel - Force-closing a channel whose registration task was not accepted by an event loop: [id: 0xcb4f2d46] java.util.concurrent.RejectedExecutionException: event executor terminated at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:821) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:327) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:320) at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:746) at org.testcontainers.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:475) at org.testcontainers.shaded.io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:80) at org.testcontainers.shaded.io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:74) at org.testcontainers.shaded.io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:86) at org.testcontainers.shaded.io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:332) at org.testcontainers.shaded.io.netty.bootstrap.Bootstrap.doResolveAndConnect(Bootstrap.java:163) at org.testcontainers.shaded.io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:145) at com.github.dockerjava.netty.NettyDockerCmdExecFactory$UnixDomainSocketInitializer.connect(NettyDockerCmdExecFactory.java:276) at com.github.dockerjava.netty.NettyDockerCmdExecFactory.connect(NettyDockerCmdExecFactory.java:220) at com.github.dockerjava.netty.NettyDockerCmdExecFactory.connect(NettyDockerCmdExecFactory.java:213) at com.github.dockerjava.netty.NettyDockerCmdExecFactory.access$000(NettyDockerCmdExecFactory.java:157) at com.github.dockerjava.netty.NettyDockerCmdExecFactory$1.getChannel(NettyDockerCmdExecFactory.java:183) at com.github.dockerjava.netty.InvocationBuilder.getChannel(InvocationBuilder.java:213) at com.github.dockerjava.netty.InvocationBuilder.get(InvocationBuilder.java:507) at com.github.dockerjava.netty.exec.PingCmdExec.execute(PingCmdExec.java:23) at com.github.dockerjava.netty.exec.PingCmdExec.execute(PingCmdExec.java:10) at com.github.dockerjava.netty.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21) at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35) at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$null$3(DockerClientProviderStrategy.java:169) at org.rnorth.ducttape.ratelimits.RateLimiter.getWhenReady(RateLimiter.java:51) at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$ping$4(DockerClientProviderStrategy.java:167) at org.rnorth.ducttape.unreliables.Unreliables.lambda$retryUntilSuccess$0(Unreliables.java:41) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

[main] ERROR org.testcontainers.dockerclient.DockerClientProviderStrategy - Could not find a valid Docker environment. Please check configuration. Attempted configurations were: [main] ERROR org.testcontainers.dockerclient.DockerClientProviderStrategy - EnvironmentAndSystemPropertyClientProviderStrategy: failed with exception InvalidConfigurationException (ping failed) [main] ERROR org.testcontainers.dockerclient.DockerClientProviderStrategy - UnixSocketClientProviderStrategy: failed with exception InvalidConfigurationException (ping failed). Root cause ConnectException (connect(..) failed: Permission denied) [main] ERROR org.testcontainers.dockerclient.DockerClientProviderStrategy - As no valid configuration was found, execution cannot continue Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 20.668 sec <<< FAILURE! demoTest(DemoTest) Time elapsed: 0.008 sec <<< ERROR! java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$2(DockerClientProviderStrategy.java:146) at java.util.Optional.orElseThrow(Optional.java:290) at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:138) at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:99) at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:126) at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:166) at org.testcontainers.containers.BrowserWebDriverContainer.<init>(BrowserWebDriverContainer.java:61) at DemoTest.<init>(DemoTest.java:9) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217) at org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Results :

Tests in error:
demoTest(DemoTest): Could not find a valid Docker environment. Please see logs and check configuration`

endless-qa on 18 Mar 2018

Can you please run a docker info before running the tests and post the output? So basically, the environment detection fails.

kiview on 21 Mar 2018

Here is the output from the docker info command:
Containers: 7 Running: 0 Paused: 0 Stopped: 7 Images: 6 Server Version: 17.12.1-ce Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog Swarm: active NodeID: xzl6ivvyum43trwhb91g37aff Is Manager: true ClusterID: vvwfkpposfbtjkbbmain610zt Managers: 1 Nodes: 1 Orchestration: Task History Retention Limit: 5 Raft: Snapshot Interval: 10000 Number of Old Snapshots to Retain: 0 Heartbeat Tick: 1 Election Tick: 3 Dispatcher: Heartbeat Period: 5 seconds CA Configuration: Expiry Duration: 3 months Force Rotate: 0 Autolock Managers: false Root Rotation In Progress: false Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 9b55aab90508bd389d7654c4baf173a981477d55 runc version: 9f9c96235cc97674e935002fc3d78361b696a69e init version: 949e6fa Security Options: apparmor seccomp Profile: default Kernel Version: 4.13.0-37-generic Operating System: Ubuntu 16.04.4 LTS OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 3.738GiB Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

WARNING: No swap limit support`

endless-qa on 23 Mar 2018

Was this command run as part of your CI build? (that would be helpful)

kiview on 23 Mar 2018

Actually, I tried a different approach and found a workaround - I switched to the standalone Jenkins installation (not as a Docker container) and got the same error as described above.

Then I thought that maybe the jenkins user doesn't doesn't have all permissions to read the /var/run/docker.sock file, ant it causes all problems. So, as a hotfix I changed permissions for it: chmod 777 /var/run/docker.sock and now TestContainers detects Docker correctly.

I will check later with the Jenkins installation as a Docker container and let you know here. Obviously, permissions should be set for the jenkins user (maybe making it root).

endless-qa on 23 Mar 2018

👍5

Normally you should add the jenkins user to the docker group (but please also see this: Docker daemon attack surface)

Since this is nothing Testcontainers related, I'll close this issue. If you have any more questions regarding how to setup your CI to best leverage Docker and Testcontainers, simply ask in our Slack channel, we try to help as best as we can :slightly_smiling_face:

kiview on 23 Mar 2018

👍2

So, the final solution that works for me:

Add jenkins user to the docker group: sudo usermod -aG docker jenkins
Reboot a host machine. Optionally, you can open the terminal under the jenkins user and run the groups command to make sure that there is the docker group in the list.

After that Jenkins is able to deal with the docker.sock file and TestContainers can see the Docker installation.

endless-qa on 23 Mar 2018

👍10 ❤5

Sounds good, great to hear you got it working :slightly_smiling_face:

kiview on 23 Mar 2018

👍2

In my case, I solved the issue in the following way:

sudo chmod 666 /var/run/docker.sock

jabrena on 6 Nov 2018

👍4 ❤2

@jabrena This is not recommended and a considerable security issue. If possible I would recommend to change it again to 660.
Also see https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface

kiview on 6 Nov 2018

❤1 👍1

Updated the notes. Thanks @kiview

jabrena on 6 Nov 2018

For those coming here from Google, I also had to restart the jenkins service - in case that helps.
And as I was running an elasticsearch testcontainer, I also had to do this on the jenkins box:

https://github.com/docker-library/elasticsearch/issues/111#issuecomment-268989731

ndtreviv on 17 Jan 2019

So, the final solution that works for me:

Add jenkins user to the docker group: sudo usermod -aG docker jenkins

Reboot a host machine. Optionally, you can open the terminal under the jenkins user and run the groups command to make sure that there is the docker group in the list.

After that Jenkins is able to deal with the docker.sock file and TestContainers can see the Docker installation.

I did it right, but forget to reboot :)
Please, don't lose your time like me and reboot your host after adding user to docker group.

binakot on 30 Jan 2019

👍5

You can also use docker run -v /var/run/docker.sock:/var/run/docker.sock --group-add ${dockerGroupId} ... when starting the container. This will add the group to the user inside the docker container which gives it access to the .sock file.

nstaeger on 21 Feb 2019

👍6

I am following article https://www.baeldung.com/docker-test-containers for Docker Compose Module.
My test class: https://github.com/eugenp/tutorials/blob/master/testing-modules/test-containers/src/test/java/com/baeldung/testconainers/DockerComposeContainerLiveTest.java
Docker file: https://github.com/eugenp/tutorials/blob/master/testing-modules/test-containers/src/test/resources/test-compose.yml
Docker version: 19.03.13, build 4484c46d9d
JDK: openjdk version "1.8.0_242"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.242-b08, mixed mode)
OS: macOS Catalina 10.15.7 (19H2)

I am getting same exception :(
I have tried following things:

As mentioned in this discussion, changing /var/run/docker.sock permissions to 777
Adding DOCKER_HOST = unix:///var/run/docker.sock to testcontainers.properties
None of this worked for me! Any clues?

java.lang.ExceptionInInitializerError at sun.misc.Unsafe.ensureClassInitialized(Native Method) at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43) at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:156) at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088) at java.lang.reflect.Field.getFieldAccessor(Field.java:1069) at java.lang.reflect.Field.get(Field.java:393) at org.junit.runners.model.FrameworkField.get(FrameworkField.java:73) at org.junit.runners.model.TestClass.getAnnotatedFieldValues(TestClass.java:230) at org.junit.runners.ParentRunner.classRules(ParentRunner.java:255) at org.junit.runners.ParentRunner.withClassRules(ParentRunner.java:244) at org.junit.runners.ParentRunner.classBlock(ParentRunner.java:194) at org.junit.runners.ParentRunner.run(ParentRunner.java:362) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53) Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$7(DockerClientProviderStrategy.java:214) at java.util.Optional.orElseThrow(Optional.java:290) at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:206) at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:132) at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:174) at org.testcontainers.containers.DockerComposeContainer.<init>(DockerComposeContainer.java:134) at org.testcontainers.containers.DockerComposeContainer.<init>(DockerComposeContainer.java:118) at org.testcontainers.containers.DockerComposeContainer.<init>(DockerComposeContainer.java:114) at com.symantec.saep.appcontrol.rule.generator.DockerComposeContainerLiveTest.<clinit>(DockerComposeContainerLiveTest.java:16) ... 17 more