Testcontainers-java: Build testcontainers in Drone CI

[arnabkarmakar2008]

Trying to build springboot application containing test written using testcontainers(cassandra) in Drone CI. Things are all working in my local mac. But volume mapping is not allowed in my Company's Drone CI. Is there any other workaround to run testcontainers in Drone without volume mappings?

[kiview]

Hi @arnabkarmakar2008 ,
do you need volume mapping for your tests?

If the error only occurs because Testcontainer's startup checks, you can disable them via properties:
https://www.testcontainers.org/usage/properties.html

[arnabkarmakar2008]

Hi @kiview,
Thanks for your help!
I do not need volume mapping for my tests. I think I am not understanding how to configure it inside docker container properly. Getting exception related to "No Valid docker environment".

I have added testcontainers.properties file with property checks.disable = true in my classpath. And my drone task is looking like this:

pipeline:

  build:
    when:
      event: [pull_request, push]
      branch: [develop, master, PROFILE-3105]
    image: cwl.docker.test.com/java8:v1.0.0
    environment:
      - "GRADLE_USER_HOME=.gradle"
    commands:
      - ./gradlew clean -i build

With this configuration, I am getting below exception :

com.tgt.profile.dao.CardDaoTest > classMethod FAILED
53s
715
    java.lang.ExceptionInInitializerError
53s
716

53s
717
        Caused by:
53s
718
        java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
53s
719
            at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$2(DockerClientProviderStrategy.java:146)
53s
720
            at java.util.Optional.orElseThrow(Optional.java:290)
53s
721
            at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:138)
53s
722
            at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:90)
53s
723
            at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:124)
53s
724
            at com.tgt.profile.dao.CardDaoTest.<clinit>(CardDaoTest.java:56)

Could you please let me know what I am missing?

Thanks for your help again!

[bsideup]

@arnabkarmakar2008
I think you need to add

volumes:
   - /var/run/docker.sock:/var/run/docker.sock

so that TestContainers can access the Docker daemon

[arnabkarmakar2008]

@bsideup
I tried that. But as I mentioned in earlier, volumes is restricted in our org. Is there any other way to set the docker daemon? If yes, could you please give me an example.

[bsideup]

@arnabkarmakar2008
well, since TestContainers is a Docker-based testing library, obviously it must have access to Docker, either by socket or TCP.
If volumes are restricted then I assume TCP access to Docker will also be restricted, but you can try it (you can use TCP-based Docker access by setting DOCKER_HOST=tcp://1.2.3.4:2376 and DOCKER_CERT_PATH variables, where 1.2.3.4 is your Docker's host ip)

[rnorth]

We use Drone at Skyscanner, and did some work to make it possible to use Testcontainers without using 'trusted' repositories (I imagine this is the same as your situation). It was not fun to implement - Drone is by far the most awkward CI tool I've come across w.r.t. using Testcontainers.

In short; it can be done, but there are some performance downsides relating to image caching.

The brief overview of how it works:

1. We have a custom Drone plugin image which is based on docker:17.09-dind and has things like the JDK included.
2. This custom plugin image is explicitly trusted by Drone (via a little-documented Drone boot environment variable DRONE_PLUGIN_PRIVILEGED). Any plugin in this whitelist will have the necessary capabilities to run Docker-in-Docker; this is how Drone's own docker build plugin is able to run DinD.
3. The entrypoint script of the custom plugin launches a Docker-in-Docker daemon, waits for it to be ready, and finally executes a command to run the build (i.e. Gradle or Maven etc)

The significant downside of using DinD is there is no ability to cache docker image layers in between builds. This is already true for normal Drone usage (your build using the docker plugin will have to pull all layers), but is much less fun when Testcontainers might potentially be pulling multiple other images for testing. As a bit of a workaround, we implemented some pre-emptive image pulling that happens in parallel to the actual gradle/maven build command.

It's not open source, but I'll see if I can change that...

[DavidTNguyen]

@rnorth - do you have a sample yml show how one can run the testcontainers inside a drone CI job?

[rnorth]

We've released, and continued to refine, a Drone plugin that enables use of Testcontainers in Drone: https://github.com/testcontainers/dind-drone-plugin

We use this internally at Skyscanner (although we have built a further internal image on top to perform ECR registry login).

I'll keep this issue open as a reminder to update our docs, but functionally all the pieces are publicly available now.

[rambrus]

@rnorth
I'm trying to use your plugin with Drone.io 1.6.4, following your guidelines.
Added a build step to my .drone.yml like this:

- name: testcontainers_dind
  image: quay.io/testcontainers/dind-drone-plugin:latest
  build_image: openjdk:8-jdk-alpine
  settings:
    cmd: ./gradlew clean check --info
  prefetch_images:
    - "redis:4.0.6"

This step is failing:

echo "🐳 Starting docker-in-docker daemon"
/usr/local/bin/dockerd-entrypoint.sh dockerd \
  --data-root /drone/docker \
  -s ${PLUGIN_STORAGE_DRIVER:-overlay2} \
  --log-level error \
  -H tcp://0.0.0.0:2375 \
  -H unix:///var/run/docker.sock &

with the following error:

mount: permission denied (are you root?)
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: permission denied (are you root?)

Did some research and found that Drone.io supports using the official drone:dind image as a service, like this:
https://docker-runner.docs.drone.io/examples/service/docker_dind/

Unfortunately, to be able to use docker properly in a step, I still need to use docker:dind image, like this:

- name: test
  image: docker:dind
  volumes:
  - name: dockersock
    path: /var/run
  commands:
  - sleep 5 # give docker enough time to start
  - docker ps -a

Actually, I want to run some SBT commands, so I would prefer using some other images (the way you did in build_image)

Do you have a working solution on using TestContainers with Drone.io 1.6.x?

[rnorth]

Hi @rambrus
Actually we still use Drone 0.8 at $WORK, with no plans to upgrade, so I haven't yet looked into the upgrade path post-1.0.

From the description, though, it looks like the plugin is not running in a privileged mode. Have you tried (or found an equivalent for) these options?

[rnorth]

It looks like DRONE_RUNNER_PRIVILEGED_IMAGES might be the 1.0+ equivalent of DRONE_ESCALATE: https://github.com/drone/drone/blob/d240eadb6569ce4b61e684a1eb05bf175ccfc8bd/cmd/drone-agent/config/config.go#L97

and

https://docs.drone.io/runner/docker/configuration/reference/drone-runner-privileged-images/

I could be wrong, but it would be worth looking into.

If you do find a working approach, it would be great if we could work this into the README of https://github.com/testcontainers/dind-drone-plugin.

[rambrus]

@rnorth thanks for the hints, will look into it and get back to you if I can make it work.

[rambrus]

@rnorth I managed to find a way of using TestContainers in drone.io without dind and the
dind-drone-plugin.

What I did:

steps:
- name: run_tests
  image: sbtbuild:1.1.1.1
  volumes:
    - name: dockersock
      path: /var/run/docker.sock
  environment:
    DOCKER_HOST: unix:///var/run/docker.sock
  commands:
    - sbt clean test

volumes
  - name: dockersock
    host:
      path: /var/run/docker.sock

It's working fine, pretty fast, docker images can be cached, no need to fetch them everytime. The main difference is that the TestContainer images are running alongside the sbtbuild container, not inside.
What do you think about this approach? Is there any downside?

[Mickelback00]

@rambrus, can you share more details about how you got this running? For instance, when you mentioned TestContainer images running along side etc. I'm just trying to fit the pieces together in our env.

[rambrus]

@Mickelback00 you need to mount the host machine's docker socket in your step container and make it available as a unix socket, this way you can spin up containers by TestContainers "next to" your container, not inside that.

_Please update your .drone.yml following these steps:_
1./ Make the host's docker socket available as volume

volumes
  - name: dockersock
    host:
      path: /var/run/docker.sock

2./ Mount the dockersock volume in your container

steps:
- name: <your step>
  image: <your docker image>
  volumes:
    - name: dockersock
      path: /var/run/docker.sock

3./ Add the DOCKER_HOST environment variable to your container, so that TestContainers can detect the docker environment

  environment:
    DOCKER_HOST: unix:///var/run/docker.sock

The complete changes you need:

steps:
- name: <your step>
  image: <your docker image>
  volumes:
    - name: dockersock
      path: /var/run/docker.sock
  environment:
    DOCKER_HOST: unix:///var/run/docker.sock
  commands:
    - <your command>

volumes
  - name: dockersock
    host:
      path: /var/run/docker.sock

No other changes needed, no need for custom plugin, your TestContainer job running inside your container will be able to discover the host's docker environment and spin up / spin down the necessary containers.

I hope it makes sense, please let me know if you have further questions.

[Mickelback00]

@rambrus - that worked perfectly - thank you so much!!

[rambrus]

@Mickelback00 cheers

[rnorth]

Closing as this looks old and resolved.