Testcontainers-java: Localstack integration fails to start container unless checks are disabled

[maciejwalkowiak]

I am facing an issue when using Localstack module. Container can start only once, then it gets stuck. Once I restart docker, I can run it again - only once. I tracked it down with debug to DockerClientFactory#checkMountableFile line 309, and then runInsideDocker line 361 - this is where the process gets blocked. There is no exception, process is just hanging.

The workaround I figured out for now is just to disable checks with checks.disable=true in .testcontainers.properties - I am not sure though if there won't be any surprises further down the road.

Sample repository that reproduces an issue: https://github.com/maciej-scratches/testcontainers-localstack-issue, specifically, this test

Testcontainers: 1.15.2
Java: OpenJDK 11.0.9.1
Docker: 20.10.2
OS: MacOS

[bsideup]

@maciejwalkowiak are you 100% sure that it is LocalStack specific? Perhaps you can reproduce it in isolation?

[maciejwalkowiak]

It definitely wasn't Localstack specific but it wasn't also generic enough to affect regular generic containers.

Anyways, purged docker, reinstalled docker, upgraded Java, problem is gone 🤷‍♂️ Not sure really what was it.

[jrmcdonald]

Hi, this issue is occurring for me as well, unfortunately purging/reinstalling did not solve the problem. I've also put together a reproduction repo using postgres - https://github.com/jrmcdonald/testcontainers-3790.

I'm running on macOS Mojava (10.14.6), using Docker Desktop version 3.1.0, engine version 20.10.2. Java version is AdoptOpenJdk 11.0.10 and testcontainers version is 1.15.2.

When debugging, it is waiting at exactly the same place as above. I've also noticed that after cancelling the build, I can't clean up the alpine container that is created, I have to restart docker before I can do that.

I've attached a video of me working through reproducing the issue, let me know what other information I can provide/what I can try to do to help!

Edit: a colleague just tried my reproduction repo and didn't see the issue :(

https://user-images.githubusercontent.com/920906/108364584-257c7280-71ee-11eb-908e-2035d2850577.mov

[bsideup]

@jrmcdonald this sounds like a problem with Docker, not Testcontainers (e.g. not being able to remove a container, even from CLI).

Since even Docker CLI operations aren't working, I would recommend reporting it to the Docker team.

[jrmcdonald]

Thanks @bsideup, will do!

[kristi]

I'm experiencing this, was this reported to Docker? Has there been any response from them?

OSX 10.15.6
Docker Desktop 3.2.2 (also broken on 3.1.0)
testcontainers 1.15.2
localstack 0.12.9.1

I've worked around by downgrading my Docker desktop to 2.5.0.1

[jrmcdonald]

@kristi I never got around to reporting it to Docker, however the issue appears to have resolved itself for me. The steps I tried:

1. Completely uninstalling & reinstalling docker again
2. Upgrading Docker (although I suspect this wasn't the fix given I'm running on 3.2.2, the same as you are)
3. I ticked the Use gRPC FUSE for file sharing setting in the Docker preferences console.