Che: "/fakeagent" is already in use by container

[monaka]

Run a lot of eclipse/che:nightly with difference ports at the same time.

Some coitainers can't spawn che-server. Log is like below.

Mar 08 08:35:23 che-node-00 dockerd[1056]: time="2017-03-08T08:35:23.950391448Z" level=error msg="Handler for POST /v1.23/containers/create returned error: Conflict. The container name \"/fakeagent\" is already in use by container fffd092867c7e46e801be9b50999f52543b564c55ba6cf4098f6249831b01e6f. You have to remove (or rename) that container to be able to reuse that name."

OS and version:

CoreOS 1339.0.0. But maybe OS independent.

[TylerJewell]

Hi @monaka - ok, so the startup sequence during the preflight checks uses Docker to start and stop test servers to create simulated connections between a browser and a container. This container is named fakeagent.

It seems that you must be having some sort of condition that is causing the preflight checks to be started, but our normal cleanup of those preflight checks are not being completed properly. The cleanup of the tests has us remove the fakeagent.

Or, perhaps if you are starting many containers at the same time, a second container is attempting to run preflight checks with fakeagent before the first server that is booting has finished to remove the fakeagent.

So, some options:

1. You can add --skip:preflight to skip these checks and this container will never be launched.
2. You can sequence your starting of containers so that you do not start one Che before the others have finished.
3. You can hack the CLI to randomize the container name of fakeagent (it's in the library.sh file) so that different starts will have different container names to avoid any sort of race condition.

[TylerJewell]

I have started a branch with some proposed changes. These are not well tested as I am on a plane and bandwidth is awful, so use at your own risk right now until they are more broadly covered.

https://github.com/eclipse/che/tree/che-4353

[monaka]

IMO, the best solution is "3." variant.
Users will run eclipse/che with different CHE_PORT in the multi che-server environment.
So I think this issue can be fixed by just adding suffix -$CHE_PORT to fakeagent.
(This strategy is same as che-server's container name.)

[TylerJewell]

Yeah, I agree with the usage of CHE_PORT. - committed and tested a solution for this.
https://github.com/eclipse/che/pull/4378

[ghost]

PR merged - closing the issue.

[monaka]

Oh..., the issue is still available...

docker: Error response from daemon: driver failed programming external connectivity on endpoint fakeagent-8081 (e74c50fbc19f347f255c18bf65f80a91c1326d10d4a34f042df1baa86de0f4a9): Bind for 0.0.0.0:32768 failed: port is already allocated.

So I use --skip:preflight for now as @TylerJewell suggests. Thank you for your support.

[balthild]

This issue still exists. I think it should be resolved rather than adding --skip:preflight each time.

root@sysujri:~# docker run -it -v /var/run/docker.sock:/var/run/docker.sock -v /home/che:/data eclipse/che start
WARN: Bound 'eclipse/che' to 'eclipse/che:6.2.0'
WARNING: No swap limit support
INFO: (che cli): 6.2.0 - using docker 17.12.0-ce / native
WARN: (che cli): 'CHE_HOST=[my ip address]' is != discovered IP '172.17.0.1'
INFO: (che config): Generating che configuration...
INFO: (che config): Customizing docker-compose for running in a container
INFO: (che start): Preflight checks
WARNING: No swap limit support
         mem (1.5 GiB):           [OK]
         disk (100 MB):           [OK]
         port 10006 (http):        [AVAILABLE]
docker: Error response from daemon: Conflict. The container name "/fakeagent-10006" is already in use by container "8bb4d112b47c616b254dd31a6cf2aba03e08a65410d17ae8214fb047e95db27f". You have to remove (or rename) that container to be able to reuse that name..
See 'docker run --help'.

[skabashnyuk]

@benoitf do you have any assumption what it can be?

[benoitf]

would be nice to have docker ps -a output before the startup