Watchman: Unable to run watchman watch <directory_name>

[piotrpalek]

When I try to run watchamn watch some_dir_name I get this error:

/Users/piotrpalek/Library/LaunchAgents/com.github.facebook.watchman.plist: Operation not permitted
1417520511: tid=2122277632 unable to talk to your watchman!

I am using OSX Yosemite and installed watchman via homebrew.

[wez]

Can you run the following commands and paste the output here?

ls -l /Users/piotrpalek/Library /Users/piotrpalek/Library/LaunchAgents /Users/piotrpalek/Library/LaunchAgents/com.github.facebook.watchman.plist
tail $TMPDIR/.watchman.piotrpalek.log

[piotrpalek]

Hey, I've since tried multiple things, fixing permissions in disk tool, also disabling the service via:
sudo launchctl unload -w ~/Library/LaunchAgents/com.github.facebook.watchman.plist but that didn't help. So I tried the only true solution to all problems, and restarted my macbook. And that fixed my problem :)

[ixtli]

I'm also getting this issue on 10.9, but restarting doesn't seem to help. I'm getting this:

[ixtli@ixtlis-Mac-Pro] ~ $ watchman watch ./Project
Could not open job overrides database at: /private/var/db/launchd.db/com.apple.launchd/overrides.plist: 13: Permission denied
launch_msg(): Socket is not connected
1420648873: tid=2052051728 unable to talk to your watchman!
[ixtli@ixtlis-Mac-Pro] ~ $

Do you have to run watchman as root? If not then the way you're loading launchd jobs is wrong: according to http://launchd.info/ there is a per-user overrides.plist and watchman seems to be trying to access one owned by root :(

[ixtli]

oh and:

-rw-r--r--   1 ixtli staff 1117 Jan  7 11:41 /Users/ixtli/Library/LaunchAgents/com.github.facebook.watchman.plist

/Users/ixtli/Library/LaunchAgents:
total 28
[...]
lrwxr-xr-x 1 ixtli staff   46 Oct 20 15:16 homebrew.mxcl.redis.plist -> /usr/local/opt/redis/homebrew.mxcl.redis.plist

[ixtli@ixtlis-Mac-Pro] ~ $ tail $TMPDIR/.watchman.ixtli.log
tail: /var/folders/23/1xzdj7mx26g_wrwlwvs0f1kc0000gn/T//.watchman.ixtli.log: No such file or directory

[wez]

Watchman doesn't contain any code to elevate privilege. You should _always_ run it as yourself unless you are doing something unusual. Did either of you run it as root? Trying to understand what is broken for you and how it got broken.

[ixtli]

I did not ever try to run it as root :( Is there an easy way I can completely delete all launchd plists and reinstall it? I installed from source originally as per the instructions on the website.

[wez]

The only thing that watchman creates is ~/Library/LaunchAgents/com.github.facebook.watchman.plist.

To remove that, I would do:

launchctl unload ~/Library/LaunchAgents/com.github.facebook.watchman.plist
rm ~/Library/LaunchAgents/com.github.facebook.watchman.plist

I'm also tracking an issue where the TMPDIR (which is random by default on OSX) gets baked in to the plist file. My recommendation until I can push a fix for that is to export TMPDIR=/tmp in your .bashrc or equivalent before you run watchman.

[piotrpalek]

The only thing I did was install it via brew and try to use it via ember-cli. Nothing more nothing less, and a restart fixed my issues.

[ixtli]

@wez Thank you for your help. I will try to completely remove my install of watchman and install the brew build and track what happens.

@piotrpalek This issue has persisted over several restarts for me :(

[ixtli]

For reference, when i try to unload that job I get

[ixtli@ixtlis-Mac-Pro] ~ $ launchctl unload ~/Library/LaunchAgents/com.github.facebook.watchman.plist
Could not open job overrides database at: /private/var/db/launchd.db/com.apple.launchd/overrides.plist: 13: Permission denied
launchctl: Error unloading: com.github.facebook.watchman

Then, as a super user:

[ixtli@ixtlis-Mac-Pro] ~ $ sudo launchctl unload ~/Library/LaunchAgents/com.github.facebook.watchman.plist
Password:
launchctl: Dubious ownership on file (skipping): /Users/ixtli/Library/LaunchAgents/com.github.facebook.watchman.plist
nothing found to unload

[ixtli]

Restart and reinstall don't work :( I can confirm that brew install watchman then watchman watch ~ works fine on the machine running 10.10 sitting next to this one.

[ixtli]

Aaand to bring an end to my comment spam here I deleted /var/db/launchd.db/com.apple.launchd because it only seemed to contain a mapping of what services i'd enabled in the "sharing" preferences pane and then restarted. It appears to have recreated the folder with the EXACT same permissions but now watchman launches just fine.

I have no idea :/

[piotrpalek]

@ixtli I would help you but I just poked around when it wasn't working, and then restarted and it did work after that. So I really don't know what caused the problem or how I (if at all) fixed it.

[vthommeret]

I'm seeing this issue. @wez Is there anything I can provide to help diagnose this issue?

Permissions for that file:

$ ls -l ~/Library/LaunchAgents/com.github.facebook.watchman.plist                                     
-rw-r--r--  1 vernon  staff  1121 Mar  2 15:27 /Users/vernon/Library/LaunchAgents/com.github.facebook.watchman.plist

When I try to watch a directory:

~/Library/LaunchAgents/com.github.facebook.watchman.plist: Operation not permitted
1425338952: tid=1974063872 unable to talk to your watchman!

When I restarted once, it started working, then the issue cropped up again. Now even after restarting, deleting the PLIST file, etc... I get that issue.

Running Mac OS X 10.10.3. Installed via Homebrew. Also tried the latest and compiling from source and ran into the same issue. There's nothing in the log file.

[wez]

@synotic what happens if you run launchctl load -f ~/Library/LaunchAgents/com.github.facebook.watchman.plist?

It might also be interesting to trace the syscall activity... can you try this in one window:

# trace syscalls for launchd (pid == 1) and its children
sudo dtruss -f -p 1 > /tmp/wat 2>&1

Then try the launchctl load -f ~/Library/LaunchAgents/com.github.facebook.watchman.plist and once it returns to your prompt, CTRL-C the dtruss process in the other window, and then put /tmp/wat in a gist or otherwise find a way to get it to me?

[vthommeret]

Running that command (I'm assuming it should be -F instead of -f?), I get the same output ("Operation not permitted").

I sent you an email with the trace output.

[wez]

http://apple.stackexchange.com/a/88670 describes what lives in /var/db/launchd.db... I don't have such a directory on my system. My guess is that the permissions or contents of yours are broken somehow. You may be able to simply remove them to fix things?

[wez]

@synotic did you get things working?

[wez]

I think this might have the same underlying cause as https://github.com/facebook/watchman/issues/90 which turned out to be a umask related problem with the .plist file we create for launchd, so I'm closing out this issue.

[stroborobo]

I can still reproduce this with version 3.3.0.

bo-mbp:bo> watchman watch-project $DIR >/dev/null
/Users/bo/Library/LaunchAgents/com.github.facebook.watchman.plist: Operation not permitted
2015-06-30T16:36:20,758: [cli] unable to talk to your watchman on /usr/local/Cellar/watchman/3.3.0/var/run/watchman/bo!
bo-mbp:bo> l -d /Users/bo/Library/LaunchAgents
drwxr-xr-x+ 11 bo  staff   374B Jun 28 14:05 /Users/bo/Library/LaunchAgents
bo-mbp:bo> l /Users/bo/Library/LaunchAgents/com.github.facebook.watchman.plist
-rw-r--r--+ 1 bo  staff   1.4K Jun 30 16:36 /Users/bo/Library/LaunchAgents/com.github.facebook.watchman.plist
bo-mbp:bo> l -d /usr/local/Cellar/watchman/3.3.0/var/run/watchman
drwxr-xr-x+ 3 bo  admin   102B Jun 28 21:42 /usr/local/Cellar/watchman/3.3.0/var/run/watchman/
bo-mbp:bo> l /usr/local/Cellar/watchman/3.3.0/var/run/watchman
bo-mbp:bo> 

Reboot or can I get some logs for you? dtrace? Or just let Disk Utility repair permissions? Rebooting fixed it for me on 3.2.0. (I just didn't have the time to do the debugging that day.)

[wez]

Try removing the .plist file?

We've seen some cases where launchd won't load a plist after a major OS upgrade or restore from another disk, even though the visible permissions look fine.

The simplest resolution has been to remove the plist file and then let watchman re-create it.

[stroborobo]

Doesn't change a thing.

[nayrmi]

Knorkebrot, are you running tmux or screen? I hit this same issue until I exited to a non-tmux session, and now I'm working ok again.

Looking around forums, it seems that launchctl and tmux aren't always good friends.

[llj098]

same issue here, but I was in a tmux session, after I quit the tmux session, it works.

[stroborobo]

@nayrmi I do, but it didn't work outside of tmux as well.

[bhamiltoncx]

Have we reported the failure to talk to launchctl from tmux to the tmux team?

I couldn't find any issues related to that (open or closed):

https://github.com/tmux/tmux/issues

[paulcavallaro]

Just ran into this myself, and I'm not sure it was ever raised with the tmux team, so I filed a report here: https://github.com/tmux/tmux/issues/600