Testcontainers-java: Unable to use static MySqlContainer in multiple test classes

[phillipjohnson]

The example here suggests that a container can be created in an abstract class and used by multiple test classes. While this is possible with the redis container, it is not with a MySqlContainer.

public abstract class AbstractIntegrationTest {
    @ClassRule
    public static MySQLContainer mysql = new MySQLContainer();
}

public class ExampleTestOne extends AbstractIntegrationTest {
    @Test
    public void someTest() {
        assertTrue(true);
    }
}

public class ExampleTestTwo extends AbstractIntegrationTest {
    @Test
    public void someTest() {
        assertTrue(true);
    }
}

One test passes, but the second fails due to "Duplicate mount point '/etc/mysql/conf.d'".

Complete stack trace:

org.testcontainers.containers.ContainerLaunchException: Container startup failed

    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:189)
    at org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:544)
    at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runners.Suite.runChild(Suite.java:128)
    at org.junit.runners.Suite.runChild(Suite.java:27)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:83)
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:182)
    ... 17 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:256)
    at org.testcontainers.containers.GenericContainer.lambda$start$0(GenericContainer.java:184)
    at org.testcontainers.containers.GenericContainer$$Lambda$35/1728579441.call(Unknown Source)
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:76)
    ... 18 more
Caused by: java.lang.reflect.UndeclaredThrowableException
    at com.sun.proxy.$Proxy13.exec(Unknown Source)
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:206)
    ... 21 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.testcontainers.dockerclient.AuditLoggingDockerClient.lambda$wrappedCommand$14(AuditLoggingDockerClient.java:98)
    at org.testcontainers.dockerclient.AuditLoggingDockerClient$$Lambda$28/1971764991.invoke(Unknown Source)
    ... 23 more
Caused by: com.github.dockerjava.api.exception.InternalServerErrorException: {"message":"Duplicate mount point '/etc/mysql/conf.d'"}

    at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:109)
    at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)
    at org.testcontainers.shaded.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at org.testcontainers.shaded.io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
    at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
    at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
    at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at org.testcontainers.shaded.io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
    at org.testcontainers.shaded.io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:608)
    at org.testcontainers.shaded.io.netty.channel.kqueue.KQueueDomainSocketChannel$KQueueDomainUnsafe.readReady(KQueueDomainSocketChannel.java:127)
    at org.testcontainers.shaded.io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:355)
    at org.testcontainers.shaded.io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:198)
    at org.testcontainers.shaded.io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:270)
    at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
    at org.testcontainers.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
    at java.lang.Thread.run(Thread.java:744)

This is on Docker Community Edition for Mac using testcontainers 1.4.1.

Version 17.06.0-ce-mac18 (18433)
Channel: stable
d9b66511e0

[kiview]

There might be some race condition which would lead to multiple MySql containers being running simultaneously, something that the current implementation of MySQLContainer does not allow.
A temporary workaround would involve omitting the @ClassRule annotation and starting/stopping the container manually in the @BeforeClass method.

[bsideup]

Hi @phillipjohnson,

Thanks for reporting! It's some weird behavior in JUnit I don't really understand.

However, you can remove @ClassRule and replace it with:

public abstract class AbstractIntegrationTest {

    public static MySQLContainer mysql = new MySQLContainer();

    static {
        mysql.start();
    }
}

by doing this, you will get a singleton-like MySQL container, and it will be destroyed at JVM shutdown :)

[phillipjohnson]

Thanks for the suggestion! it seems that using @ClassRule does try to create the container twice. I noticed when using a Tomcat container that the exposed port actually changed between initialization in AbstractIntegrationTest and the actual test I was running. Using the static initializer worked fine, though.

[rnorth]

Aha, so effectively JUnit triggers startup more than one time on the same object? (It is the same object, because it's a static member of the parent class)

This is kinda obscure, but perhaps we should have some guard code to at least detect where a container is started more than once without being stopped in between. That way we could log a more useful message!!

[kiview]

Great idea, sounds super helpful 😃 (I had the same problem with testcontainers-spock btw. and we found out about it because of MySQLContainer, but it was a bug in my code 😉, would have found it earlier with such an error message as well)

[diabluchanskyi]

Hello guys, I wonder if there any solution for this issue so far? We do use MySQL and we've been really amazed with testcontainers, but we do have the problem described above.

[bsideup]

Hi @diabluchanskyi

Please see my previous comment:
https://github.com/testcontainers/testcontainers-java/issues/417#issuecomment-318064857

[bsideup]

@diabluchanskyi you might also consider JDBC URLs:
https://www.testcontainers.org/usage/database_containers.html#jdbc-url

[diabluchanskyi]

@bsideup Thank you, it works with 1st proposed solution

[unhuman]

In addition to the suggestion that @bsideup suggested above, for Couchbase (https://github.com/differentway/testcontainers-java-module-couchbase), I also needed to add a wrapper around the Container object:

    /**
     * This class is to allow the CouchbaseContainer to be shared across tests
     * When the JVM shuts down, the container will as well
     */
    public class TestingCouchbaseContainer extends CouchbaseContainer {
        private static AtomicBoolean started = new AtomicBoolean(false);

        @Override
        public void start() {
            // only allow a single start()
            if (started.compareAndSet(false, true)) {
                super.start();
            }
        }

        @Override
        public void stop() {
            // Do nothing
        }
    }

[bsideup]

Hi @unhuman,

Did you try static {} block approach? You don't need that wrapper with it

[unhuman]

@bsideup Indeed I did. That's why I mentioned it... In case other people find they have the same problem.

[guss77]

@phillipjohnson - It is not a race condition or anything: this problem is caused by having the class rule set on a base class for the tests, instead of on the test itself. Move the MySQL container to a class rule on each test, and the problem goes away.

The confusion is because of a misunderstanding of how static fields work in OO, and I see this a lot: if you have a static field on a parent class, which you sub-class 3 times, how many copies of that object do you have? Exactly one.

When JUnit runs the class rule, it doesn't know (nor care) that the field is not stored in the test class itself, and it will run the rule again for each class. The single MySQL container isn't being run multiple times concurrently (at least not if you use the classic non-parallel JUnit runner), but it is trying to call start() on a MySQLContainer that was already stopped - and this is what causes the errors.

My solution was to simply move the container rule to each test, though a possible solution might also be to leave it on the base class and write an @AfterClass method (in the base class) that will throw away the used container and prime a new one for the next class:

@AfterClass
public void recycleDatabase() {
    mysql = new MySQLContainer(); // replace used database with a fresh one
}

I haven't actually tried that (it feels hackish to me), so YMMV - and it will for sure not work if you are using a parallel runner.

[bsideup]

@guss77 for the same use case (shared MySQL between different test classes) we use the approach I mentioned in one of my previous comments:
https://github.com/testcontainers/testcontainers-java/issues/417#issuecomment-318064857

It works like a charm and from TestContainers' perspective is absolutely fine to use, I promise (c) :D

[guss77]

@bsideup - I appreciate that your solution works for you, but it assumes that a test never needs to reinitialize the database.

There could be other use cases where it is required to use a different database instance per test class, so using MySQLContainer as a JUnit rule, like the original report does, will work better than a single database instance per VM.

[bsideup]

@guss77 sorry, I missed the part about "different database instance per test class" :) But anyway, it's still better to have the same running DB and re-initialize it by dropping & recreating it than starting a new container for every test class

[guss77]

@bsideup Better from what perspective?
Is it faster? undoubtedly.
Is it cleaner and simpler to maintain? maybe yes, maybe not - mostly depends on the custom base test code you write.
Is it safer in face of not 100% controlled code? no way - a test might create routines or other long living objects, change session or global parameters or change future behavior of the database that you may not foresee.
As always, the correct solution depends on a lot of variables that are very specific to the local domain, and it is a good idea to consider all the options instead of recommending on a single "tried and true" way.

[unhuman]

We wanted to have the same container for all of our test classes. Why? Because re-starting the container takes ~ 30s. Also, our tests are essentially independent, yet they're in the same package we're testing. Since we are aware of what's going on in our tests (leveraging different keys, etc), I think it's reasonable to have the database persist throughout the running of all tests. @bsideup's info was super-helpful for me to get (almost) the desired behavior.

I can see your point though about having tests change parameters... Being able to make the correct choice depending on use case offers tremendous flexibility.

[bearrito]

I'm seeing this with a GenericContainer. Is have the same GenericContainer across multiple test classes even supported e.g. if I do not inherit from a abstract test class? None of the fixes above work for me.

Specific error is

  java.lang.ExceptionInInitializerError
        Caused by: java.lang.IllegalStateException 

The call site where this throws is the constructor line

public static GenericContainer redisContainer = new GenericContainer("redis:3.2.11")
            .withExposedPorts(6379).waitingFor(Wait.forListeningPort());

    static {
         redisContainer.start();

    }

[guss77]

@bearrito - this does not seem like the same problem reported above. I suggest opening a different ticket where people can help you figure this one out.

[bearrito]

This was a misconfigration on my part. Our CI server was missing a docker flag. Apologies for the noise.

[gnalFF]

Hi all,

just reading, and hope this helps a bit, having the testcontainer in an abstract class as well.

Wrapping the TestContainer in an ExternalResource will startup a new instance, consistently:

```@ClassRule
public static ExternalResource resource = new ExternalResource() {
protected void before() throws Throwable {
AbstractDBComponentTest.LOGGER.info("STARTING DATABASE");
AbstractDBComponentTest.mysql = (new MySqlContainer()).withDatabaseName("db");
AbstractDBComponentTest.mysql.start();
System.setProperty("data_source.schema", AbstractDBComponentTest.mysql.getDatabaseName());
System.setProperty("data_source.url", AbstractDBComponentTest.mysql.getJdbcUrl());
System.setProperty("data_source.username", AbstractDBComponentTest.mysql.getUsername());
System.setProperty("data_source.password", AbstractDBComponentTest.mysql.getPassword());
}

    protected void after() {
        AbstractDBComponentTest.LOGGER.info("STOPPING DATABASE");
        AbstractDBComponentTest.mysql.stop();
    }
};

If your tests are not transactional, make sure to clean up the database afterward: 
```@Rule
    public TestRule cleanUpDBRule = new TestRule() {
        public Statement apply(final Statement statement, Description description) {
            return new Statement() {
                public void evaluate() throws Throwable {
                    try {
                        statement.evaluate();
                    } finally {
                        AbstractDBComponentTest.LOGGER.info("CLEANING UP DATABASE");
                        //using jooq here
                        DBCleaner.clean(AbstractDBComponentTest.this.dslContext, AbstractDBComponentTest.this.getSchema());
                    }

                }
            };
        }
    };

PS: sorry for the bad formatting

[bsideup]

@gnalFF as I mentioned before in this thread, if you need one DB for all tests in abstract class, you should not use the rules, but static {} block where you call .start() only once

[gnalFF]

right, one instance for all, true.
or put the rule into the testsuite (at least 4.9 upwards)

[kiview]

There were some different discussions in this issue, but the general problem can be solved by using the static{} block approach, so I'll close this issue for now.

[bradcupit]

@bsideup in your earlier comment you said:

by doing this, you will get a singleton-like MySQL container, and it will be destroyed at JVM shutdown

How does it get destroyed at shutdown?

I see GenericContainer registers a shutdown hook, but it only deletes temporary files.

I subclassed MySQLContainer, overriding both stop() and close(), but neither got called.

[bsideup]

Hi @bradcupit,

Shutdown hook is not only about the files :) Also, we have a sidecar container called "Ryuk" which will cleanup the containers even if you kill -9 your JVM 😎

[binakot]

Hi @phillipjohnson,

Thanks for reporting! It's some weird behavior in JUnit I don't really understand.

However, you can remove @ClassRule and replace it with:

public abstract class AbstractIntegrationTest {

    public static MySQLContainer mysql = new MySQLContainer();

    static {
        mysql.start();
    }
}

by doing this, you will get a singleton-like MySQL container, and it will be destroyed at JVM shutdown :)

Thanks to the advice, I did something like this:

public abstract class BaseIntegrationTest {

    private static final PostgreSQLContainer testPostgres = new PostgreSQLContainer("postgres");
    static {
        testPostgres.start();
    }

    @Configuration
    public static class TestDataSourceConfig {

        @Bean
        public DataSource dataSource() {            
            final PGSimpleDataSource ds = new PGSimpleDataSource();
            ds.setUrl(testPostgres.getJdbcUrl());
            ds.setUser(testPostgres.getUsername());
            ds.setPassword(testPostgres.getPassword());
            return ds;
        }
    }

    @Autowired
    private DataSource dataSource;

    @Before
    public void setUp() {
        ((PGSimpleDataSource)dataSource).setUrl(testPostgres.getJdbcUrl());
        ((PGSimpleDataSource)dataSource).setUser(testPostgres.getUsername());
        ((PGSimpleDataSource)dataSource).setPassword(testPostgres.getPassword());
        TestUtils.prepareTestSchema(dataSource); // Init test database.
    }

    @After
    public void tearDown() throws Exception {
        try (final Connection conn = dataSource.getConnection();
             final PreparedStatement stmt = conn.prepareStatement("DROP OWNED BY " + testPostgres.getUsername())) {
            stmt.execute(); // Clear test database.
        }
    }
}

[gvart]

An alternative solution to share the same container across multiple Test classes which run in same spring context, is to create a configuration class, such that:

@Configuration
public class MongoContainerConfiguration {
    private final GenericContainer MONGO_CONTAINER = new GenericContainer("mongo").withExposedPorts(27017);

    @PostConstruct
    public void start() {
        MONGO_CONTAINER.start();
    }

    @PreDestroy
    public void stop() {
        MONGO_CONTAINER.stop();
    }

    @Bean
    @Primary
    public MongoClient mongoClient() {
        return new MongoClient(MONGO_CONTAINER.getContainerIpAddress(), MONGO_CONTAINER.getMappedPort(27017));
    }
}

[benkeil]

@bearrito

This was a misconfigration on my part. Our CI server was missing a docker flag. Apologies for the noise.

What was your solution?

[VadimOza]

So is there some plans to fix @ClassRule? or we must just use static {} workaround?

[bsideup]

@VadimOza there is nothing to "fix", that's the way JUnit Rules work.

[torrespro]

@diabluchanskyi you might also consider JDBC URLs:
https://www.testcontainers.org/usage/database_containers.html#jdbc-url

Does JDBC URL approach generates one Container per Test Class or one for all the Tests?

[misaunde]

I'm having the same problem, but I'm getting a different error. I've been using the static block approach recommended above from the beginning, however, when my second class with tests runs I get this error:

CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLException: Cannot create PoolableConnectionFactory (Cannot open database "<db_name>" requested by the login. The login failed.

Why on earth would the second file not be able to login, right after the first one was able to?
If I run them separately, they both work as expected, but when I run more than one, the 2nd one always fails.
Even if I change the order, the first file always works, but the second one fails with the login error.
Even if I copy and paste one file's contents into the other, the first file works, but the second identical file fails with the login error.

Any ideas?

[misaunde]

I think I figured it out. Once I removed the @Container annotation, it started working across multiple classes:
// @Container protected static final MSSQLServerContainer sqlServer;

More info: https://stackoverflow.com/a/62443261/3806701