Ember-cli: Ember Serve doesn't see file changes.

[dgavey]

Running Ember-cli 0.21 on OSX 10.10.3

For a while now (since January) Ember serve will not see file changes on one of my ember projects. The other Ember project works just fine.

The strange part is, that sometimes after a reboot it will work until the next reboot. But not always. I can't upgrade to 0.23 quite yet to try it. But if anyone has any other suggestions on how to fix it I'd love to hear them.

In the mean time I'm getting by with 'ember serve --watcher=polling'

[stefanpenner]

do either of these work?

ember serve --watcher=node
ember serve --watcher=watchman

no known issues exist on 0.2.1, maybe something if funky with watchman on your machine?

[dgavey]

I'm sure it's something funky on my machine, none of the other developers have this issue. We've tried a lot of different things to make it work. Very little luck.

Neither --watcher=node or watcher=watchman worked.

If I pull a brand new copy of the repo in a new folder, that does fix it for a short while. But within a week it's back to not seeing files.

[stefanpenner]

@dgavey can you run: watchman watch-list ?

[dgavey]

Sure here is the output

{
"version": "3.0.0",
"roots": [
*Removed all the folders listed *
]
}

The projects folder does seem to be listed.

[stefanpenner]

"roots": [
*Removed all the folders listed *
]

this is what i wanted to see. How many entries does that roots array contain?

[dgavey]

Sorry bout that, about 30 or so, I can send you a pm with it if you like. Or let me know what you're looking for.

[stefanpenner]

Sorry bout that, about 30 or so, I can send you a pm with it if you like. Or let me know what you're looking for.

just the count.

[dgavey]

Actually 56

[stefanpenner]

@dgavey hmm, lets try pruning the watchman watchers

following instructions on the gist of mine: https://gist.github.com/stefanpenner/1a215bc10257d10a7cfd

[stefanpenner]

ember-cli needs to take more advantage of watchman to reduce watcher roots, and watchman needs to age out roots

this may or may not be the issue, but i suspect it is likely.

[dgavey]

Ok. I'll let you know how it goes. Thanks!

[dgavey]

Unfortunately that didn't seem to work. The watcher list was removed from watcher, but it still does not see changes.

[stefanpenner]

Unfortunately that didn't seem to work. The watcher list was removed from watcher, but it still does not see changes.

strange, is it on a different drive ?
if you move it elsewhere does it work?

[dgavey]

No same drive. Still works fine for the other project I have (also on the same drive)

[stefanpenner]

No same drive. Still works fine for the other project I have (also on the same drive)

:( the only thing, i can think is some invalid key or pathname?

Without steps to reproduce, I am unsure how to debug this further :(

[dgavey]

Thanks for trying, I appreciate it a lot.

[stefanpenner]

Thanks for trying, I appreciate it a lot.

If you have anymore hints or a production case I'll gladly dig in. Unfortunately without a repro case this isn't really actionable.

[stefanpenner]

This appears isolated to very few users, and without a repro case I don't believe this is actionable, as such I am sadly going to close this issue.

If at some point more information comes to light that makes this actionable, or if it start affected a critical mass of users I will gladly reopen.

[blimmer]

We just ran into this on our designer's machine. Unfortunately, I don't have a solution, but here's some info:

Background

• We just upgraded his machine to 10.10.3 (Latest Yosemite)
• We're on ember-cli 0.2.3

Things we tried

• brew uninstall watchman followed by brew reinstall watchman
• fall back to node with ember serve --watcher=node
• reinstall of node (through nvm)
• kill watchman watchers via Stef's gist. we saw a bunch of watchers being killed here when the ember server wasn't running
• git clean -fdx in the repo
• reinstall everything in a clean repo (npm install && bower install)

Current workaround

• cloning the repo under a new name (reponame-new)

Sorry no more info, but hopefully this helps others that might be having the problem.

[stefanpenner]

cc @wez any debugging thoughts?

[cigoe]

The current workout worked mentioned by @blimmer works for @dgavey but then breaks randomly on a future reboot and stays broken. I have tried other OS level fixes such as repairing disk permissions but nothing seems to stick.

[blimmer]

I wonder if it's a file permission issue that happens on reboot.

I kept thinking it has something to do with Watchman, but the node watcher doesn't work either. I just can't think of what the difference could be, since I'm running the same setup as the designer on my team that's having this problem.

[stefanpenner]

I kept thinking it has something to do with Watchman, but the node watcher doesn't work either.

if both don't work, it is likely an issue with the underlying OS provided FS watching primitives.

[wez]

If you're using watchman on OS X, please make sure that you're on version 3.1 (watchman shutdown-server ; brew update ; brew reinstall watchman; watchman version).

It would be useful to see your watchman logs; if you used homebrew to install it, the log will be found in /usr/local/Cellar/watchman/3.1/var/run/watchman/$USER.log.

If you could capture that log into a gist and share it with us (or me directly if you are concerned about containing any information that might be in there), that might help reveal what is going on.

We have a general troubleshooting page here: https://facebook.github.io/watchman/docs/troubleshooting.html

The fact that you are having trouble with both the node and watchman watchers indicates that you might be having problems with the number and nature of fsevents clients on your system.

If you are a Sublime Text user, please try quitting your editor and see if that helps (it sounds crazy, but we've had numerous reports of Sublime getting fsevents into a bad state).

https://facebook.github.io/watchman/docs/troubleshooting.html#fsevents has a series of things you can try to resolve fsevents related problems.

[dgavey]

I am on watchman 3.1

Cleaned up (for privacy issues) log is here https://gist.github.com/dgavey/8cfd50458c97332de35b
Only thing that looks out of place on it for me is:

1429658296: tid=2034643712 failed to parse json from /usr/local/Cellar/watchman/3.1/var/run/watchman/derek.state: unable to open /usr/local/Cellar/watchman/3.1/var/run/watchman/derek.state: No such file or directory

That file exists, but maybe there is a permissions issue. I'm gonna look into that.

[wez]

It's expected to see that at the top of the log file; the state file is how watchman remembers the roots that it was asked to watch, and it won't be present the first time that you run it.
Your logs don't indicate any obvious problems.

I'm running low on ideas, but there's one more thing that you can try.

In one window, run this; make sure the debug.log redirection is NOT within any of the dirs that you have asked watchman to watch, and note that this file will get large rather quickly; you'll want to CTRL-C pretty quickly after the next step (so read ahead before you run this!)

watchman --server-encoding=json --persistent log-level debug > /tmp/debug.log

Then create a new file or whatever action it is that you expect ember to trigger for, wait a couple of seconds to ensure that it's had a chance to be observed by watchman and CTRL-C the command that is running above.

What this does is turn the log level up to debug (the most verbose) for the persistent session established by that command, and then logs it to the file you specify. This log will include more details about filesystem notifications and may help identify what is going on.

[dgavey]

Other than the log containing a bunch of 'assessing triggers' And I do see the the root ember project folder in there. There was no record of the file name I changed. And no other errors that I could see.

[dgavey]

Ok, I got it fixed! I found this article
( http://feedback.livereload.com/knowledgebase/articles/86239-os-x-fsevents-bug-may-prevent-monitoring-of-certai )
So I went and renamed (and then renamed back) every folder from my projects root folder to where the ember source code is. Not sure which folder did the trick, but it is now seeing events again.

Thanks for the help everyone.

[wez]

@dgavey interesting find!
Had you previously renamed any of those files/folders in such a way that you changed the case of their name(s)? Wondering if that might have been a trigger

[dgavey]

I'm not aware of it happening, but it's possible. I'll be watching to see if it breaks again, and I will try and determine if any folders had been renamed.

[wez]

@dgavey if you do see it trigger again, can you try running the find-fsevents-bugs tool referenced in the article you linked? I've also added that information to the watchman troubleshooting docs.

If that tool can detect the problem, I'd like to build that detection into watchman so that this is more discoverable. The APIs used by that tool are all deprecated so we'd need to find a more modern way to detect the issue. If it does hit you again it would be great it we can work together on that aspect.

Thanks!

[dgavey]

Ok It happened again. I ran the tools from https://github.com/andreyvit/find-fsevents-bugs

It listed quite a few problems.

find-fsevents-bugs /Users/derek/gitroot/webapp/src/Project/EmberSrc/App/app

Found (FSCopyAliasInfo): '/Users/derek/gitroot/webapp/src/Project/EmberSrc/App/app' != '/Users/derek/gitroot/webapp/src/Project/emberSrc/App/app'

• not actual pathnames

As you can see there is a number of capitals not matching in each path. I renamed (and renamed back) each offending folder and restarted ember serve. It didn't start working again till I renamed all of folders that had case differences. After re-running the find-fsevents-bugs tool there were no more problems reported.

I hope this helps you out @wez and other folks who might run into this issue.

[wez]

@dgavey Thanks! Just curious, do you know what changed the case of the folders in the first place? Was it something you did directly?

[dgavey]

I can't say I changed anything. I was actively working on the project, and it broke in between commits. As far as I can tell, none of the folders changed case or names. So it's still a mystery in that regard.

[mitchiru]

Same here.

[cstolli]

Same here, I just used the tool @dgavey suggested and it worked.

[stefanpenner]

we should likely add this to the docs.. https://github.com/andreyvit/find-fsevents-bugs

tracking issue: https://github.com/ember-cli/ember-cli.github.io/issues/54

[MacgyverMartins]

I'm having the same error here. It just stoped working in my develop branch.
So, I went back one commit, ran ember s and it was work.
Then I back to develop and ran ember s, and, I don't know why, it went back work again

[blagh]

It might have to do with OS X trickiness, the default file system is kinda weird about cases, but if there's something that isn't using the same assumptions? Just a drive by comment, hope it helps.

[camnicklaus]

Hey all... so I've hit this a time or two and this prooooobably isn't the issue here. however I thought I'd add it just in case it's a solve for anyone.
I got myself into a state where I had executed two git processes at the same time (probably one in the vs code gui and one from the command line).
a coworker suggest I change branches as a trouble shooting measure and I got this error:
Another git process seems to be running in this repository, e.g. an editor opened by 'git commit'. Please make sure all processes are terminated then try again. If it still fails, a git process may have crashed in this repository earlier: remove the file manually to continue.
I ran:
rm -f .git/index.lock to fix that and then live reload worked again :)