Docker-sync: Docker Sync not syncing

[gravataLonga]

Hello, i have the docker sync and docker-compose working on local computador, but my docker sync aren't sync files, at least i think is not sync. Because when i edit my index.php and do <?php echo "ola"; exit; not reflect on my project.

Error/Feature Requestion/Docs

My Docker-sync isn't sync.

Docker Driver

I'm not sure, but when i run docker --version i got,
Docker version 17.06.1-ce, build 874a737

Sync strategy

native_osx

your docker-sync.yml

version: "2"

options:
  compose-file-path: 'docker-compose-dev.yml'
  verbose: false
  rsync_image: 'eugenmayer/rsync'
  unison_image: 'eugenmayer/unison'
  cli_mode: 'auto'
  max_attempt: 200
  project_root: 'pwd'
syncs:
  sumolcompal-volume-sync:
    src: './application/'
    sync_strategy: 'native_osx'
    sync_host_ip: '127.0.0.1'
    sync_host_port: 10875
    sync_excludes_type: 'docker/'
    watch_excludes: ['./.hg', './application/node_modules', './application/bower_components', './application/sass-cache', './application/.sass-cache', './application/.sass-cache', '.coffee', '.scss', '.sass', '.gitignore']

OS

MacOSx
10.12.6

Docker Sync Log

   Sync Log:  Fatal error: Warning: the archives are locked.  
   Sync Log:  If no other instance of unison is running, the locks should be removed.
   Sync Log:  The file /root/.unison/lk3d804615889d5a09d84376cb429bebe0 on host f430c2f07322 should be deleted
   Sync Log:  The file /root/.unison/lk92aafb0d5fcd5e6e76b0afe6f9dff53a on host f430c2f07322 should be deleted
   Sync Log:  Please delete lock files as appropriate and try again.
   Sync Log:  Fatal error: Warning: the archives are locked.  
   Sync Log:  If no other instance of unison is running, the locks should be removed.
   Sync Log:  The file /root/.unison/lk3d804615889d5a09d84376cb429bebe0 on host f430c2f07322 should be deleted
   Sync Log:  The file /root/.unison/lk92aafb0d5fcd5e6e76b0afe6f9dff53a on host f430c2f07322 should be deleted
   Sync Log:  Please delete lock files as appropriate and try again.

I have entered on machine, with docker exec -ti sumolcompal-volume-sync bash, and deleted the offending files, stop and restart docker-sync and docker-compose. And nothing, after 30 minutes the sync work but my echo on index.php not work.

In the logs, i got,
` Sync Log: [BGN] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_page.html.twig_ds9Cl7YqQEdHj7HW-myPsTlO5/fYb0Ir8xEPqd69DiWWxdMK2mUxYEh5E2_W7OC5Sc7J8.php from /app_sync to /host_sync Sync Log: [END] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_page.html.twig_ds9Cl7YqQEdHj7HW-myPsTlO5/fYb0Ir8xEPqd69DiWWxdMK2mUxYEh5E2_W7OC5Sc7J8.php Sync Log: [BGN] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_region.html.twig_z_MT8BjjoierWSud7LvMcaVeT/5TXOY8nKdqTSuPHykWSOr22vKE9DA_s_UW34fRYeuLU.php from /app_sync to /host_sync Sync Log: [END] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_region.html.twig_z_MT8BjjoierWSud7LvMcaVeT/5TXOY8nKdqTSuPHykWSOr22vKE9DA_s_UW34fRYeuLU.php Sync Log: [BGN] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_status-messages.html.twig_0ZavSMDYB76QOpfT7N2_d0275/DD1OcyMqlh1S0l8I9401tC0-mAJMbEWOfuh8rpU0azY.php from /app_sync to /host_sync Sync Log: [END] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_status-messages.html.twig_0ZavSMDYB76QOpfT7N2_d0275/DD1OcyMqlh1S0l8I9401tC0-mAJMbEWOfuh8rpU0azY.php Sync Log: [BGN] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_time.html.twig_Hb07jC10XDubOFBEnUFJVR-yS/lfkVFldd76P3MJ3Z_z_XahzWscF9fzIpwLPth7x-eI0.php from /app_sync to /host_sync Sync Log: [END] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_time.html.twig_Hb07jC10XDubOFBEnUFJVR-yS/lfkVFldd76P3MJ3Z_z_XahzWscF9fzIpwLPth7x-eI0.php Sync Log: [BGN] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_username.html.twig_lwouULs70LaHmpVAdzBz4aH1Y/OQYK0uq6K_t_RonAjYHm4-n6pKHseXQhHuD2izOuLqw.php from /app_sync to /host_sync Sync Log: [END] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_username.html.twig_lwouULs70LaHmpVAdzBz4aH1Y/OQYK0uq6K_t_RonAjYHm4-n6pKHseXQhHuD2izOuLqw.php Sync Log: [BGN] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_views-view--frontpage--la__UJW4lqo6WPltVU6jP-poMzTW/On1eK68fQJpXvthss4eVpC8RuwPYqYZH56dba25cfgA.php from /app_sync to /host_sync Sync Log: [END] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_views-view--frontpage--la__UJW4lqo6WPltVU6jP-poMzTW/On1eK68fQJpXvthss4eVpC8RuwPYqYZH56dba25cfgA.php Sync Log: [BGN] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_views-view--frontpage.htm_bz3R827Me_THmv10ZfkFmeWhC/y9f_sQG-Zy4eOo9AGaXFNKRL9vYNmQnYefEXSVdGI1o.php from /app_sync to /host_sync Sync Log: [END] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_views-view--frontpage.htm_bz3R827Me_THmv10ZfkFmeWhC/y9f_sQG-Zy4eOo9AGaXFNKRL9vYNmQnYefEXSVdGI1o.php Sync Log: [BGN] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_views-view-unformatted--f_Tgc5OI7qidGTOTY2Uaqwz5Sgw/PkP_xtvxj7XgocKRYiQYGQ27eSuSNqJRDMQ28dX5uHc.php from /app_sync to /host_sync Sync Log: [END] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_views-view-unformatted--f_Tgc5OI7qidGTOTY2Uaqwz5Sgw/PkP_xtvxj7XgocKRYiQYGQ27eSuSNqJRDMQ28dX5uHc.php Sync Log: [BGN] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_views-view-unformatted--f_UGw0E3x4hbknwmLSIaJlAQupC/c6tavUYqYFFzBg3aaYA-3ueCwZn8ZYvrqk4SHlqpJj4.php from /app_sync to /host_sync Sync Log: [END] Copying web/sites/development.sumolcompal.pt/files/php/twig/599edc1c065ed_views-view-unformatted--f_UGw0E3x4hbknwmLSIaJlAQupC/c6tavUYqYFFzBg3aaYA-3ueCwZn8ZYvrqk4SHlqpJj4.php Sync Log: UNISON 2.48.4 finished propagating changes at 10:44:04.30 on 25 Aug 2017 Sync Log: Synchronization complete at 10:44:04 (41 items transferred, 0 skipped, 0 failed)

Still not work.

[gravataLonga]

For the record, i'm thinking is because the php-fpm.. I'm not sure.

[EugenMayer]

Clean and rerun

[EugenMayer]

your docker-sync.yml is way to overbloated and partially wrong:

version: "2"

options:
  max_attempt: 200
syncs:
  sumolcompal-volume-sync:
    src: './application/'
    sync_excludes_type: 'docker/'

[gravataLonga]

Thanks for your tips, and thank for the great work. I will try that solution later on.

[btaylor-ihr]

@EugenMayer First off, thanks a ton for your work :) However, I honestly have to destroy my unison volume at least once a day. It was frustrating me enough to open a bug about it, but someone beat me to it :p

I'll be happily working away for hours, and then suddenly my code doesn't take effect. I generally think it's a bug in my code, but then I jump in my container and find out that it's because the changes aren't being synced.

Running docker-sync sync doesn't always work, so I have to destroy the volume and try again. Problem is that it takes around 20 minutes to maybe sync all of my files.

Can you offer any advice on this? It's very frustrating. Obviously not as frustrating as my IDE hanging every time I make a change, but it does severely hamper my workflow when it breaks.

I'm on OS X Sierra, and this is my sync file

version: "2"

options:
  compose-file-path: 'docker-compose.yml'
  compose-dev-file-path: 'docker-compose-osx.yml'
syncs:
  lapis-unison-sync:
    src: '.'
    sync_strategy: 'native_osx'
    sync_excludes: ['.git', '.github', '.idea', '.gitignore', 'node_modules', '.DS_Store']
    sync_userid: ${WWW_DATA_USER_ID}

[EugenMayer]

@btaylor-ihr seems like you running into #410 - there is a characteristic though.

What does help in your case, docker-sync clean + start or only a mac restart ( actually just a docker-engine / d4m restart needed ) ?

[btaylor-ihr]

@EugenMayer It happened again today, as normal. Restarting Docker did nothing, but restarting my Mac and editing the code a couple of times appeared to finally kick it into gear. Thanks for the tip :) Restarting my Mac is orders of magnitudes faster than recreating the volume

[EugenMayer]

Restarting docker for Mac did not help?are you sure?

[btaylor-ihr]

Maybe I didn't give it enough time. I'll update you when it happens again. I restarted my Mac because a Docker restart appeared to do nothing

[EugenMayer]

If it does actually help, i would love to have hyou commented on #410 that it does indeed do the same.

Here at our office we actually found another edge case which i will document soon, but looking at your setup, its not what you might have.

[btaylor-ihr]

Well docker-sync (or Docker) died again. This time I made sure to give Docker enough time to get its act together, and it appeared to work this time :D I just ran webpack and the files went through that time.

[EugenMayer]

ok so we have #410 again, which is a docker for mac issue we cannot somehow deal with. We will try to "detect" when it happens and ask the user to restart docker for mac - yet, i do not see a different solution.

Since we run fairly huge projects ( about 45k files in sync ) and it happens to us about every 4-5 days, i really want to know, how the metrics of your proeject are. Could you write, into #410:

• how many files you sync
• how many files you EXCLUDE ( important to know actually, it really is )
• your hardware specs ( CPU, RAM, hdd type)
• how much RAM and CPU do you give d4m?

This will help slowly classify which circumumstances are actually provoking the issue or make it at least more likely to happen. Thanks! I am closing this one i guess, its a "dupe"

[aaronash]

I continually get an error similar to this as well:

[2018-07-20 14:49:38 -0600] Sync Log:  The file /home/dockersync/.unison/lk31857890c061e1201bbc41c8dd449a45 on host 8834fffb5d03 should be deleted
[2018-07-20 14:49:38 -0600] Sync Log:  The file /home/dockersync/.unison/lka733dfdb17cf58ebabc41ef93a32dab2 on host 8834fffb5d03 should be deleted
[2018-07-20 14:49:38 -0600] Sync Log:  Please delete lock files as appropriate and try again.
[2018-07-20 14:49:39 -0600] Sync Log:  Fatal error: Warning: the archives are locked.
[2018-07-20 14:49:39 -0600] Sync Log:  If no other instance of unison is running, the locks should be removed.
[2018-07-20 14:49:39 -0600] Sync Log:  The file /home/dockersync/.unison/lk31857890c061e1201bbc41c8dd449a45 on host 8834fffb5d03 should be deleted
[2018-07-20 14:49:39 -0600] Sync Log:  The file /home/dockersync/.unison/lka733dfdb17cf58ebabc41ef93a32dab2 on host 8834fffb5d03 should be deleted
[2018-07-20 14:49:39 -0600] Sync Log:  Please delete lock files as appropriate and try again.
[2018-07-20 14:49:42 -0600] Sync Log:  Fatal error: Warning: the archives are locked.
[2018-07-20 14:49:42 -0600] Sync Log:  If no other instance of unison is running, the locks should be removed.
[2018-07-20 14:49:42 -0600] Sync Log:  The file /home/dockersync/.unison/lk31857890c061e1201bbc41c8dd449a45 on host 8834fffb5d03 should be deleted
[2018-07-20 14:49:42 -0600] Sync Log:  The file /home/dockersync/.unison/lka733dfdb17cf58ebabc41ef93a32dab2 on host 8834fffb5d03 should be deleted
[2018-07-20 14:49:42 -0600] Sync Log:  Please delete lock files as appropriate and try again.

I think I've narrowed it down to generally happening when I switch between git branches. Is this a known issue? It can happen even if there are only a couple files different between branches.

My docker-sync.yml file:

version: "2"

syncs:
  dev-root-sync:
    src: './'
    sync_excludes: ['node_modules', '.babelCache', '.git', '.docker-sync']
    sync_userid: '1000'
    sync_prefer: 'src'
    sync_strategy: 'native_osx'
    watch_excludes: ['.*/.git', '.*/node_modules', '.*/sass-cache', '.gitignore', '.docker-sync/daemon.log']

[jasonellington]

Is there an update on this issue? it still happens...