Docker-sync: Correct way to start/restart docker-sync?

[rogamoore]

I was wondering what is the supposed way to start/stop the sync container? Currently I just use Ctrl+C but it looks a bit strange:

^CTerminated!
    shutdown  Shutting down...
          ok  Stopping sync container condrobs-unison-sync
/usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/lib/docker-sync/execution.rb:51:in : Interrupt
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/lib/docker-sync/execution.rb:51:in block in forkexec
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/lib/docker-sync/execution.rb:50:in fork
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/lib/docker-sync/execution.rb:50:in forkexec
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/lib/docker-sync/sync_strategy/unison.rb:71:in watch
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/lib/docker-sync/watch_strategy/unison.rb:27:in run
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/lib/docker-sync/sync_process.rb:78:in run
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/lib/docker-sync/sync_manager.rb:121:in block in run
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/lib/docker-sync/sync_manager.rb:120:in each
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/lib/docker-sync/sync_manager.rb:120:in run
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/tasks/sync/sync.thor:37:in start
Traceback (most recent call last):
    from /usr/local/lib/ruby/gems/2.3.0/gems/thor-0.19.1/lib/thor/command.rb:27:in run
  File "/usr/local/bin/unison-fsmonitor", line 272, in <module>
    from /usr/local/lib/ruby/gems/2.3.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in invoke_command
    from /usr/local/lib/ruby/gems/2.3.0/gems/thor-0.19.1/lib/thor.rb:359:in dispatch
    from /usr/local/lib/ruby/gems/2.3.0/gems/thor-0.19.1/lib/thor/base.rb:440:in start
    from /usr/local/lib/ruby/gems/2.3.0/gems/docker-sync-0.1.1/bin/docker-sync:14:in <top (required)>
    from /usr/local/bin/docker-sync:23:in load
    from /usr/local/bin/docker-sync:23:in <main>
    main()
  File "/usr/local/bin/unison-fsmonitor", line 216, in main
    [cmd, args] = recvCmd();
  File "/usr/local/bin/unison-fsmonitor", line 98, in recvCmd
    line = sys.stdin.readline()
KeyboardInterrupt

When trying to restart with docker-sync start it will show a conflict:

docker: Error response from daemon: Conflict. The name "/mycontainer-unison-sync" is already in use by container d873a14ff874d6250b75d4b3b68dac1d0d71eecec70d613cc9edb2382222e45f. You have to remove (or rename) that container to be able to reuse that name..

However I can start the existing container with docker start <id> and then run docker-sync and it seems to work. But shouldn't there be an easier way?

[EugenMayer]

Usually CTRL+C should end your whole stack and all services, without removing them ( its not down ). Then, you can run docker-sync start or docker-sync-stack start and restart all of it ( without remaking it ). There is a docker-sync-stack clean if you want to remove all containers and volumes to start from scratch, but thats rather a remake, not a stop-start.

But yest, i might make sense to implement a docker-sync-stack stop to explicitly stop the stack, its not a lot of work either.

In the end, the CTRL+C way is what you commonly do with docker-compose, docker-sync itself runs in the background.

There is effort to simplify the daemon mode https://github.com/EugenMayer/docker-sync/pull/65 to make all a bit more convenient how to stop/start/clean/resume and so on.

For the daemon mode, i would love to have a task-bar-icon to control the behaviors.

[rogamoore]

Thanks for your feedback :)
Just a note: docker-sync start doesn't work anymore after stopping with Ctrl+C. I always have to manually start the existing container with docker start otherwise I'll get a conflict error message as stated above.
Do you see the same behaviour?

[EugenMayer]

No we never have this here, also when using the boilerplate here https://github.com/EugenMayer/docker-sync-boilerplate i cannot reproduce it. Can you?

Which docker / docker-compose versions are you using? Which sync strategy?

[rogamoore]

I'm using unison-unox.
docker version 1.12.1-rc1, build 7889dc7, experimental
docker-compose version 1.8.0, build f3628c7

I will try the boilerplate later!

[kulinski]

I also see this exact error message when stopping with Ctrl+C and then restarting with docker-sync start. I don't believe its related to my sync strategy. The error message looks like what you'd see if you try to docker run with an existing stopped container. Shouldn't docker-sync start check for a stopped container and (if it exists) use docker start?

[EugenMayer]

AFAICS this issue is related to timestamp issues with unison and i can slowly reproduce it, it is most probably related to https://github.com/EugenMayer/docker-sync/issues/108 and probably related to a yet not properly fixed timezone issue https://github.com/EugenMayer/docker-sync/issues/90

[EugenMayer]

I probably nailed the problem today, by this #116 this seem to address this "reusage" issue - one problem we probably created is, that we let the docker-sync user mount the volume by container name, not data-volume name - that is a mistake

[kulinski]

A couple issues with that approach:

1. We are duplicating the volume-mount configurations in docker-sync.yml and our docker-compose.yml. We shouldn't define that twice.
2. If we mount directory to the docker volume, rather then the sync containers, how do we know that the sync containers are running?

I thought this could be fixed by following the typical docker container lifecycle on startup. Do a create if the container doesn't exist. Once/if it exists, do a start, rather then relying on the one-liner run

[EugenMayer]

@kulinski i am absolutely unsure what you just commented.

1) that is not possible in any other way
2) We do not know and we do not need to know. We mount data and they syncs need to be started by docker-sync. If they are not running, you have no sync

We have a lifecycle, we on docker-sync / docker-sync-stack start we check the if the sync containers are running, if not, start them, if they do not exists, create and start them

[rogamoore]

@EugenMayer
I think you might be investigating in the wrong direction. IMHO it's not related to the app configuration. In fact I can start a sync container without any app containers or docker-compose(-dev).yml at all.

I just tried the boilerplate here https://github.com/EugenMayer/docker-sync-boilerplate/tree/master/unison and I can reproduce my issue.

I enabled verbose mode, so we get more info:

roga:...ker-sync-boilerplate/unison]$ docker-sync start                                                     (master✱)
          ok  Starting unison
          ok  appcode-unison-sync container not running
          ok  creating appcode-unison-sync container
     command  docker run -p '127.0.0.1::5000'                               -v appcode-unison-sync:/var/www                               -e UNISON_DIR=/var/www                               -e TZ=${TZ-readlink /etc/localtime | sed -e 's,/usr/share/zoneinfo/,,'}                                                                                             --name appcode-unison-sync                               -d eugenmayer/unison:unox
docker: Error response from daemon: Conflict. The name "/appcode-unison-sync" is already in use by container c2dfb39bd4a53c2c16785d276f528e471312b0c21e74a86614dc82144fd1767f. You have to remove (or rename) that container to be able to reuse that name..
See 'docker run --help'.
          ok  starting initial sync of appcode-unison-sync
     command  docker inspect --format=" {{ .NetworkSettings.Ports }} " appcode-unison-sync | /usr/bin/sed  -E "s/.*map\[5000[^ ]+ ([0-9]*)[^0-9].*/\1/"

As you can see it says appcode-unison-sync container not running and tries to create the container. However it shouldn't do this - it should just restart the existing container.

Looks like something is wrong with this line:
https://github.com/EugenMayer/docker-sync/blob/master/lib/docker-sync/sync_strategy/unison.rb#L133

When I run docker ps --filter "status=exited" --filter "name=appcode-unison-sync" it shows the existing container:

CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS                      PORTS               NAMES
dddeab8935e0        eugenmayer/unison:unox   "/entrypoint.sh super"   4 minutes ago       Exited (0) 45 seconds ago                       appcode-unison-sync

When I include the sed filter the result is empty:

docker ps --filter "status=exited" --filter "name=appcode-unison-sync" | sed -E -n 's/.*\\s(.*)$/\\1/p'

So the sed command is not working as expected. Did you know that you could can just change the output format of the docker ps command instead of using sed?
Like this:

docker ps --filter "status=exited" --filter "name=appcode-unison-sync" --format "{{.Names}}"

Result:

appcode-unison-sync

While reading the code I noticed you are using the same sed command here https://github.com/EugenMayer/docker-sync/blob/master/lib/docker-sync/sync_strategy/unison.rb#L130
Also this command is not working as expected and will always result in an empty output. I could confirm this by starting a sync container with docker-sync start, then opening another terminal and running docker-sync start again. Instead of "container still running" as expected it will always show "container not running"

A quick look at the code for the other sync strategies shows that they probably face the same bug.

So I guess we just need to change these lines and everything should work fine.

[sergiogomez]

I have the same issue, but I can workaround it with docker-sync clean before launch docker-sync start again.

[zxaos]

Same here - it's not picking up pre-existing sync containers, requiring a clean every time.

[polmiro]

Running into the same issue as well, need to always docker-sync clean and before that remove all containers that link to the docker-sync ones. Basically start everything from scratch.

[EugenMayer]

Sorry guys for the delay, i was moving into a new house. I will try to fix this this week and release 0.1.2 this week.

[EugenMayer]

Missed that @rogamoore already fixed this, thank you a lot! Can someone, having this issue, confirm that it works with the current master?

To test this:

pull this repo (master)

git clone https://github.com/EugenMayer/docker-sync
cd docker-sync
./deploy_locally.sh

Now you locally have the current master build installed, try to start / stop your stack and report back. Thanks!

[alexanderjulo]

Yep, it seems to work here. :-)

[EugenMayer]

great!

[polmiro]

awesome thanks for the speed @EugenMayer :)