Docker-sync: sync_args are no longer accepted being a yaml array since 0.3.x

[barat]

Docker Driver

d4m 17.05.0-ce-mac9 (17434)

docker-sync version

0.4.5.beta1

Sync strategy

native_osx

OS

MacOS 10.12.4

docker-sync.yml

version: "2"
options:
  verbose: false
syncs:
  gsdev-sync:
    src: './'
    host_disk_mount_mode: 'cached'
    sync_host_ip: 127.0.0.1
    sync_strategy: 'native_osx'
    sync_args:
      - "-ignore='Path .idea'"
      - "-ignore='Path .git'"
      - "-ignore='Name .DS_Store'"
      - "-ignore='BelowPath cms/app/logs'"
      - "-ignore='Path cms/app/cache/dev/*'"
      - "-ignorenot='Path cms/app/cache/dev/translations'"
      - "-ignorenot='Path cms/app/cache/dev/profiler'"
      - "-ignorenot='Path cms/app/cache/dev/appDevDebugProjectContainer.xml'"
      - "-ignorenot='Path cms/app/cache/dev/appDevUrlGenerator.php'"

docker-compose-dev.yml

version: "2"
services:
  gs-dev:
    volumes:
      - gsdev-sync:/app:nocopy
      - ~/.ssh:/root/.ssh
volumes:
  gsdev-sync:
    external: true

Result/error

$ docker-sync clean
    shutdown  Background sync has been stopped
     success  Finished cleanup. Removed stopped, removed sync container and removed their volumes
$ docker-sync start
          ok  Starting native_osx for sync gsdev-sync
Usage: unison [options]
    or unison root1 root2 [options]
    or unison profilename [options]

For a list of options, type "unison -help".
For a tutorial on basic usage, type "unison -doc tutorial".
For other documentation, type "unison -doc topics".

unison was invoked incorrectly (too many roots)
real    0m 0.05s
user    0m 0.04s
sys 0m 0.00s
     success  Sync container started
     success  Starting Docker-Sync in the background

When i docker exec into sync container I see, that files are in host_sync, but app_sync is empty.
Checked unison.log in tmp (inside container) but it's empty. From supervisord I have only this:

bash-4.3# tail supervisord.log
2017-05-11 10:23:53,518 INFO stopped: unison (exit status 1)
2017-05-11 10:23:53,522 INFO spawned: 'unison' with pid 23
2017-05-11 10:23:53,530 INFO exited: unison (exit status 1; not expected)
2017-05-11 10:23:54,535 INFO spawned: 'unison' with pid 24
2017-05-11 10:23:54,543 INFO exited: unison (exit status 1; not expected)
2017-05-11 10:23:56,559 INFO spawned: 'unison' with pid 29
2017-05-11 10:23:56,567 INFO exited: unison (exit status 1; not expected)
2017-05-11 10:23:59,583 INFO spawned: 'unison' with pid 30
2017-05-11 10:23:59,591 INFO exited: unison (exit status 1; not expected)
2017-05-11 10:24:00,594 INFO gave up: unison entered FATAL state, too many start retries too quickly

Any ideas?

[EugenMayer]

remove your manual excludes (sync args and try again) - if those are the reason, please fix them.
Also remove sync_host_ip: 127.0.0.1 (its not the issue, but not needed )

Pretty sure your sync_args causing the issue

[barat]

Ok - removing sync_args helped ... but those sync args was working fine with older docker-sync on unison strategy ... something changed with unison?
This excludes are nice for Symfony ... IMO there is no need to sync files listed there ...

[EugenMayer]

first, try removing the includenot stuff, then try to use them one by one.

Also connect to the sync container using exec bash and check the env

env | grep UNISON_SYNCARG

see if the string does not fit your expectations and then run

ruby -e '`unison #{ENV["UNISON_ARGS"]} #{ENV["UNISON_PREFER"]} #{ENV["UNISON_EXCLUDES"]} -numericids -auto -batch /host_sync /app_sync`'

ans see what happens

[barat]

Leaving even only one syncarg (eg. - "-ignore='Path .idea'") cause error ...
When I remove everything - it works.

After exec:

$ docker exec -it gsdev-sync bin/bash
bash-4.3# env | grep UNISON_SYNCARG
bash-4.3# env | grep UNISON_ARGS
UNISON_ARGS=[-ignore=Path .idea, -ignore=Path .git, -ignore=Name .DS_Store, -ignore=BelowPath cms/app/logs, -ignore=Path cms/app/cache/dev/*, -ignorenot=Path cms/app/cache/dev/translations, -ignorenot=Path cms/app/cache/dev/profiler, -ignorenot=Path cms/app/cache/dev/appDevDebugProjectContainer.xml, -ignorenot=Path cms/app/cache/dev/appDevUrlGenerator.php]
bash-4.3# env | grep UNISON_PREFER
bash-4.3# env | grep UNISON_EXCLUDES
UNISON_EXCLUDES=
bash-4.3# ruby -e '`unison #{ENV["UNISON_ARGS"]} #{ENV["UNISON_PREFER"]} #{ENV["UNISON_EXCLUDES"]} -numericids -auto -batch /host_sync /app_sync`'
Usage: unison [options]
    or unison root1 root2 [options]
    or unison profilename [options]

For a list of options, type "unison -help".
For a tutorial on basic usage, type "unison -doc tutorial".
For other documentation, type "unison -doc topics".

unison was invoked incorrectly (too many roots)

But tried one thing:

bash-4.3# ruby -e '`unison -ignore="Path .idea" #{ENV["UNISON_PREFER"]} #{ENV["UNISON_EXCLUDES"]} -numericids -auto -batch /host_sync /app_sync`'
Contacting server...
Looking for changes
Reconciling changes
Propagating updates
UNISON 2.48.4 started propagating changes at 11:41:50.29 on 11 May 2017
[BGN] Copying .DS_Store from /host_sync to /app_sync
[END] Copying .DS_Store
...
[BGN] Copying Makefile from /host_sync to /app_sync
[END] Copying Makefile
UNISON 2.48.4 finished propagating changes at 11:43:43.41 on 11 May 2017
Saving synchronizer state
Synchronization complete at 11:43:43  (22 items transferred, 0 skipped, 0 failed)

So quoting seems to be solution:

bash-4.3# ruby -e '`unison -ignore="Path .idea" -ignore="Path .git" -ignore="Name .DS_Store" -ignore="BelowPath cms/app/logs" -ignore="Path cms/app/cache/dev/*" -ignorenot="Path cms/app/cache/dev/translations" -ignorenot="Path cms/app/cache/dev/profiler" -ignorenot="Path cms/app/cache/dev/appDevDebugProjectContainer.xml" -ignorenot="Path cms/app/cache/dev/appDevUrlGenerator.php" #{ENV["UNISON_PREFER"]} #{ENV["UNISON_EXCLUDES"]} -numericids -auto -batch /host_sync /app_sync`'
Contacting server...
Looking for changes
Reconciling changes
Nothing to do: replicas have not changed since last sync.

It seems, that previous unison implementation was playing nicely with args so it imploded it, and respected that ignore vaues was single quoted (like in docker-sync.yml) ...

EDIT:

This seems to work:

version: "2"

options:
  verbose: false
syncs:
  gsdev-sync:
    src: './'
    host_disk_mount_mode: 'cached'
    sync_strategy: 'native_osx'
    sync_args: "-ignore='Path .idea' -ignore='Path .git' -ignore='Name .DS_Store' -ignore='BelowPath cms/app/logs' -ignore='Path cms/app/cache/dev/*' -ignorenot='Path cms/app/cache/dev/translations' -ignorenot='Path cms/app/cache/dev/profiler' -ignorenot='Path cms/app/cache/dev/appDevDebugProjectContainer.xml' -ignorenot='Path cms/app/cache/dev/appDevUrlGenerator.php'"

So it's a problem with docker-sync container where UNISON_ARGS aren't parsed/prepared before passing it to unison command

[EugenMayer]

Ok so we lost the ability to have sync args as arrays, right? Which did work before?

[barat]

Yes - at least with 0.2.0+ everything was fine ... it's not a problem now if "inline" works, but declaring it as array is IMO more readable :)
Now I need to play with it a little bit and then update article about docker-sync :)

[EugenMayer]

Well the point is, people will have this expectation and its fairly easy to fix. I will do so

[EugenMayer]

will be fixed with 0.4.5