Swarmkit: Reuse existed containers upon scale.

[stylixboom]

With docker-compose up, usually we can reuse the existed container.

Question
How can we achieve the same thing with docker service scale ...?
So, I want to reuse some data that was modified under that existed containers.

Example

$ docker service create --replicas=4 --name=test_loop ...
$ docker ps -a
... STATUS              PORTS   NAMES
... Up 19 seconds               test_loop.1.dk4rs8eqbow3wbbjch34y287z
... Up 19 seconds               test_loop.2.0t7p2ab0wabrsmt6ljne3bh91
... Up 19 seconds               test_loop.3.57n0vhfcdfwbwbmlo0f1hghsd
... Up 19 seconds               test_loop.4.7wt2wittys6m37vehwffbb8ln

$ docker service scale test_loop=2
$ docker ps -a
... STATUS              PORTS   NAMES
... Up 3 minutes                test_loop.1.dk4rs8eqbow3wbbjch34y287z
... Up 3 minutes                test_loop.2.0t7p2ab0wabrsmt6ljne3bh91
... Exited (137) 6 seconds ago  test_loop.3.57n0vhfcdfwbwbmlo0f1hghsd
... Exited (137) 6 seconds ago  test_loop.4.7wt2wittys6m37vehwffbb8ln

$ docker service scale test_loop=4
$ docker ps -a
... STATUS              PORTS   NAMES
... Up Less than a second       test_loop.4.eplof06ud9860udsyenjckqi5  <--- Instead of creating this
... Up Less than a second       test_loop.3.4l8dvn361q1lba1dqakx74hvn  <--- Instead of creating this
... Up 4 minutes                test_loop.1.dk4rs8eqbow3wbbjch34y287z
... Up 4 minutes                test_loop.2.0t7p2ab0wabrsmt6ljne3bh91
... Exited (137) 1 minutes ago  test_loop.3.57n0vhfcdfwbwbmlo0f1hghsd  <--- I want to reuse this
... Exited (137) 1 minutes ago  test_loop.4.7wt2wittys6m37vehwffbb8ln  <--- I want to reuse this

[kunalkushwaha]

With docker-compose up previous instance [exited] is deleted and new instance is created, thats why exited containers are not visible in docker ps -a

docker run has option of --rm to auto-remove container on exit, but such option is not available in docker service command.

Also docker service tries to maintain the state and instances, that is defined during docker service [create|update] i.e. auto-recreate new instance for failing containers.

If you wish to reuse data, you should store such data outside container. May be this will be helpful https://docs.docker.com/engine/tutorials/dockervolumes/

[stylixboom]

@kunalkushwaha Thank you very much for the quick answer.

However, it seems running docker-compose actually reuses the exited containers if available.
For example,

1. I run a simple compose, then it creates container id, e.g. 6ac238e7323e
2. I create a file test.txt inside this container.
3. I stop this container. The docker ps -a shows 6ac238e7323e has exited.
4. I run the same compose up command. The docker ps -a shows 6ac238e7323e is up. And I found my test.txt file is still there.

BTW, the docker service update --replicas= ... is interesting to me, but it seemed updating the --replicas acts quite the same as docker service scale ... (not reuse the exited containers).

I'm not sure if we can have this kind of docker-compose behavior with Docker swarm 1.12 when scaling the replicas.

Also, for the volume solution, if needed, I may need to do some wrapper solution. Since the service I'm running (Hyperledger) requires a separated storage. See. https://github.com/hyperledger/fabric/issues/2316

[kunalkushwaha]

Yes, you are right.
Stopped containers can be started again using docker start

I'm not sure if we can have this kind of docker-compose behavior with
Docker swarm 1.12 _when scaling the replicas_.

Not sure about docker swarm, as scheduling logic creates new instance.
Could be added as option in scheduling logic, but not sure, about other
complexities.

On Fri, Aug 5, 2016 at 3:47 PM, Siriwat K. [email protected] wrote:

@kunalkushwaha https://github.com/kunalkushwaha Thank you very much for
the quick answer.

However, it seems running docker-compose actually reuses the exited
containers if available.
For example,

1. I run a simple compose, then it creates container id, e.g. 6ac238e7323e
2. I create a file test.txt inside this container.
3. I stop this container. The docker ps -a shows 6ac238e7323e has exited.
4. I run the same compose up command. The docker ps -a shows 6ac238e7323e
   is up. And I found my test.txt file is still there.

BTW, the docker service update --replicas= ... is interesting to me, but
it seemed updating the --replicas acts quite the same as docker service
scale ... (not reuse the exited containers).

I'm not sure if we can have this kind of docker-compose behavior with
Docker swarm 1.12 _when scaling the replicas_.

Also, for the volume solution, if needed, I may need to do some wrapper
solution. Since the service I'm running (Hyperledger) requires a separated
storage. See. hyperledger/fabric#2316
https://github.com/hyperledger/fabric/issues/2316

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/docker/swarmkit/issues/1316#issuecomment-237769813,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACoXIW1FyATfoeAjKaLvo5LJOJLH_5mDks5qctyRgaJpZM4JdUkP
.

Regards,
Kunal Kushwaha

[kunalkushwaha]

Also docker swarm mode is not yet supported by docker-compose 1.8 https://github.com/docker/compose/issues/3656

[stylixboom]

Thanks for the info @kunalkushwaha
I'm not sure if we can reuse the container with swarm right now.
If not available, I may +1 for those who need this option :)

[dperny]

Reusing containers is sort of at odds with the idea of swarmkit, which is that individual tasks are ephemeral. There are reasons why you wouldn't want to reuse containers. For instance, perhaps something about the task spec has changed between scale downs and ups? You would have to check the dead container against the task spec.

That's pretty much all there is to say on this, so I'm gonna go ahead and close it. You're definitely welcome to reopen this issue if you feel it hasn't been adequately addressed.