Testcontainers-java: BrowserWebDriverContainer with DockerComposeContainer

[Zetten]

I might be missing something really simple, but it doesn't seem like it's possible to use a BrowserWebDriverContainer to interact with the environment produced by DockerComposeContainer.

It looks like the two environments (browser+vncrecorder and docker-compose) end up with different network config. I can't see an obvious/reliable way to configure the network of either to make sure they can talk to each other.

Is there a documented example/test for this sort of configuration?

If the browser container can't be easily added to the compose context (e.g. new DockerComposeContainer(...).withAdditional(new BrowserWebDriverContainer()) would be cool), maybe it would be enough to be able to extract the ambassador container for a given service, so it can be used via withLinkToContainer. Then of course getServiceHost might not return the right hostname though - it's probably a lot more complex than I imagine.

[On a related note it would also be cool to add some API sugar to BWDC for direct/procedural manipulation. I'm using cucumber-jvm and in that context standard junit @Rules don't work. Right now I'm working around this by extending the class and making public functions which call super.starting()/super.finished() which I can run between my test scenarios. It's really the cucumber lib's problem, but they don't seem interested in fixing it. 😞]

[rnorth]

Hi @Zetten
It's not a configuration that I've used much myself, but it shouldn't be too hard. IIRC @outofcoffee and @haines might have used selenium and compose together, but I could be wrong.

Anyway, I've dug up an old, unpolished example which points a chrome browser container at a very large docker-compose stack, and put the relevant bits into a gist here. This configuration successfully allows the browser container to hit one of the compose containers, as long as it's exposed:

@ClassRule
    public static DockerComposeContainer compose =
            new DockerComposeContainer(new File("src/test/resources/docker-compose.yml"))
.withExposedService("edge-router", 80);

and:

driver.get("http://" + compose.getServiceHost("edge-router_1", 80) + ":" + compose.getServicePort("edge-router_1", 80) + "/");

Please be aware that I've not exhaustively tested this across all different Docker environment types; this works for me on Docker Machine on a Mac, and I'll give it a try on Docker for Mac shortly.

N.B. this example uses Weaveworks' awesome sockshop microservices demo if you're interested.

Re making the API easier to use in non-JUnit contexts: absolutely, yes, we have plans!

HTH
Richard

[Zetten]

Hmm, the test case is almost identical in principle to what I'm trying to do, yet in my case I can't get any response via the WebDriver.

I've checked the URL being built up and it looks good: the same URL is tested via a programmatic HTTP client in the test code, and if I pause execution I can access it with curl/a native browser. So I'm pretty sure the compose environment/ambassador container is working.

If I take a look at the container recording, it shows the same URL in the address bar, but the response is ERR_CONNECTION_REFUSED.

Because the "normal" requests from the host were working I assumed it must be something specific with the BrowserWebDriverContainer. Plus in my environment (docker.sock) getServiceHost is returning localhost - I have a feeling this might a problem...

If it helps, docker inspect <browser container> shows:

        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "3c7bb405346593b819bf31c6dcf1918ae6b7c69cb64dce85260817534138c2a1",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "4444/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "32786"
                    }
                ],
                "5900/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "32785"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/3c7bb4053465",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "c5f86319b69508d2661e4d9886402e7d4fe92d30e1b5fa4571519a84dbc12f41",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "da1e61a442b15ace9d73379cd8275054ce6853daf473989a6ae5fda10a7cda95",
                    "EndpointID": "c5f86319b69508d2661e4d9886402e7d4fe92d30e1b5fa4571519a84dbc12f41",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02"
                }
            }
        }

And for <ambassador container>:

        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "70a2c4451c8f880c8d7b7a1acd6fe194e655018023c9a3b0b0dfbe6fd1b9678f",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "80/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "32784"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/70a2c4451c8f",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "cpkf2j_default": {
                    "IPAMConfig": null,
                    "Links": [
                        "/cpkf2j_mytestenv-proxy_1:mytestenv-proxy_1"
                    ],
                    "Aliases": [
                        "1b38c76844ae"
                    ],
                    "NetworkID": "6731dff0e7946228c88a5a16df7acb313c3b9fdee03d0a6e6e682d167d0af2df",
                    "EndpointID": "fdc8f57337f88905a1b010d024f0972526656d91eeee07b37be426588f67d2a9",
                    "Gateway": "172.26.0.1",
                    "IPAddress": "172.26.0.8",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:1a:00:08"
                }
            }
        }

[Zetten]

I've created a rather contrived example demonstrating my issue here: https://github.com/Zetten/testcontainers-selenium-compose-test

(I apologise in advance for the dirty, hacky Harness class - like I mentioned before it's just supporting our use of Cucumber-Jvm and I haven't invested a lot of time in cleaning it up.)

The compose file describes a single nginx container with an index.html file mounted in the docroot.

The okhttp3 lib test passes, but the selenium test fails. It saves the screen recording to build/recording/recording-*.flv where it can be seen that it's getting a connection refused error for http://localhost:<ambassador_port>.

[Zetten]

As a workaround I've extracted the "docker-in-docker" host IP evaluation from DockerClientConfigUtils, rather than trusting environment.getServiceHost. This seems to work pretty well in the test project at least; I will test it in my main project sometime this week.

Essentially the workaround changes:

    private String getBaseUrl() {
        return "http://" +
                environment.getServiceHost("web_1", 80) + ":" +
                environment.getServicePort("web_1", 80);
    }

to:

    private String getBaseUrl() {
        String ip = StringUtils.trimToEmpty(DockerClientFactory.instance().runInsideDocker(
                cmd -> cmd.withCmd("sh", "-c", "ip route|awk '/default/ { print $3 }'"),
                (client, id) -> {
                    try {
                        return client.logContainerCmd(id)
                                .withStdOut(true)
                                .exec(new LogToStringContainerCallback())
                                .toString();
                    } catch (Exception e) {
                        LOG.warn("Can't parse the default gateway IP", e);
                        return null;
                    }
                }));

        return "http://" +
                ip + ":" +
                environment.getServicePort("web_1", 80);
    }

[arcuri82]

Hi,
I had the same issue. The workaround I am using is to start DockerComposeContainer with a specific id X, and then use withNetworkMode in BrowserWebDriverContainer to point to "X_default", ie. making this latter Docker image join the same network as the one of Docker Compose. In code (Kotlin):

companion object {

    class KDockerComposeContainer(id: String, path: File) : DockerComposeContainer<KDockerComposeContainer>(id, path)

    val composeId = "gateway"

    @ClassRule
    @JvmField
    val env = KDockerComposeContainer(composeId, File("../docker-compose.yml"))
            .withLocalCompose(true)
}

@Rule
@JvmField
var browser = KBrowserWebDriverContainer()
        .withDesiredCapabilities(DesiredCapabilities.chrome())
        .withNetworkMode("${composeId}_default")

class KBrowserWebDriverContainer : BrowserWebDriverContainer<KBrowserWebDriverContainer>()

[iirekm]

I have the same issue even without docker compose, just with BrowserWebDriverContainer: it I run a few tests, the videos of the first one are never recorded, the videos of the next ones are recorded

➜  ~ docker ps -a
CONTAINER ID        IMAGE                                    COMMAND                  CREATED              STATUS                          PORTS                                              NAMES
3e3efbbda739        richnorth/vnc-recorder:latest            "flvrec.py -o /rec..."   About a minute ago   Exited (1) About a minute ago                                                      amazing_volhard

➜  ~ docker logs 3e3efbbda739
start recording
Socket error: [Errno 111] Connection refused
stop recording
➜  ~ 

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

[stale[bot]]

This issue has been automatically closed due to inactivity. We apologise if this is still an active problem for you, and would ask you to re-open the issue if this is the case.