Karma: test files are unexpectedly removed when edited by certain editors

[fredsa]

Using:

$ testacular --version
Testacular version: 0.5.5

It appears that, when saving, some editors (e.g vim) move the original file (to preserve it as a backup) and create a new file, rather than updating the file in place.

As a result Testacular (almost always) removes the edited file:

info (watcher): Removed file "/path/to//test/unit/controllersSpec.js".

Future changes to the file are no longer detected. Once is forced to restart Testacular.

Note, Testacular behaves normally (recognizing the file has been updated) when one simply touches the file:

$ touch test/unit/controllersSpec.js

A workaround is to specify an explicit file backup policy, i.e. in the vim case by adding :set backupcopy=yes to your ~/.vimrc:

"backupcopy                                                                     
"    "yes"   make a copy of the file and overwrite the original one             
"    "no"    rename the file and write a new one                                
"    "auto"  one of the previous, what works best                               
:set backupcopy=yes

[dignifiedquire]

I believe this the same problem as mentioned in paulmillr/chokidar#18.

[vojtajina]

Are you saying it does not notice "adding" the file afterwards ? That's sounds as a bug.
Can you try --log-level debug and see whether there is debug (watcher): Add file "..." ?

I just tried to move file and then move back and it worked as expected. (file removed, file added).

[fredsa]

I tested with VIM :set backupcopy=no and find the following:

1) If I repeatedly save the source file (without leaving VIM), Testacular
often sees the file as removed (which is the issue I'm reporting), although
it does sometimes see the file as added (as you suggested), but in my
environment the file is considered removed more often than it is considered
added

2) If after a file is considered removed I leave VIM open and in a separate
window try touch name-of-removed-file, nothing happens, i.e. testacular
does NOT ever see the file as added. However, if I, in that other window,
with VIM still open, touch directory-where-removed-file-lives, Testacular
does immediately see the file as added.

3) Furthermore, I find that exiting VIM, after a file has been determined
to have been removed, also makes Testatcular see the file as added. I think
this is explained by VIM removing the .swp file, which by default lives in
the same directory, causing the directory contents to change. I can confirm
this by leaving VIM open and instead removing the .swp file manually from
another window. This indeed causes Testacular to see the file as added.

So, the issue can be reproduced as follows:

• edit ~/.vimrc to :set backupcopy=no
• edit a file being monitored by testacular
• save the file a few times (using :w), but do not exit, until testacular
  complains that the file has been removed
• wait a long time, file is not automatically found, even though you can
  verify in another window that the file exists
• save the file a few more times (again using :w); after a few saves the
  file eventually is seen as added
• save some more and the file will disappear
• exit vim (using :q) and file reliably is seen as added

On Tue, Dec 4, 2012 at 3:13 PM, Vojta Jina [email protected] wrote:

Are you saying it does not notice "adding" the file afterwards ? That's
sounds as a bug.
Can you try --log-level debug and see whether there is "debug (watcher):
Add file "..."" ?

I just tried to move file and then move back and it worked as expected.
(file removed, file added).

—
Reply to this email directly or view it on GitHubhttps://github.com/vojtajina/testacular/issues/199#issuecomment-11020238.

Fred Sauer
[email protected]

[dignifiedquire]

@fredsa Could you try using only chokidar if setting options.interval to a larger number solves the problem? As stated in paulmillr/chokidar#18

[fredsa]

Thanks. Increasing options.interval in chokidar/lib/index.js does appear to
to help, although it seems like a brittle solution. Raising it from 100 to
200 might work on one system. Perhaps 100 -> 1000 is needed on another.
Over NFS or similar might need more. I think we need a robust solution that
works with options.interval being what it is today (or even lower). We can
raise/lower the value for other reasons, but the underlying algorithm needs
to be robust.

On Sun, Dec 9, 2012 at 1:41 AM, Friedel Ziegelmayer <
[email protected]> wrote:

@fredsa https://github.com/fredsa Could you try using only chokidar if
setting options.interval to a larger number solves the problem?

—
Reply to this email directly or view it on GitHubhttps://github.com/vojtajina/testacular/issues/199#issuecomment-11169373.

Fred Sauer
[email protected]

[dignifiedquire]

Thanks for the update. We will have a closer look at this as soon as we find the time. Maybe you could open an issue with chokidar, as this algorithm should be implemented there in my opinion.

[vojtajina]

@fredsa in canary (npm install -g testacular@canary) there is already batching changes, I think it should solve this problem as well.

It triggers a run when any change happens and then wait before serving the files (250ms is default) and batches all the changes that happend within this 250ms delay. Eg. if you do a git checkout with many changes, it should trigger only one test run.

Can you try it ?

[fredsa]

I tried npm install -g testacular@canary but it does not resolve the
problem for me.

[vojtajina]

@fredsa thanks for heads up.

I can't reproduce this, even doing rm some.js; echo "xxx" > some.js triggers correctly "CHANGE" on Mac (because chokidar does stat polling).

Can you make sure the chokidar dependency is on the latest version ?
CD to the path where testacular is installed and do npm list chokidar, you should have 0.5.0.

If it triggers REMOVE and then ADD, it would be ok, but it's weird, that it only triggers the REMOVE event and that misses the adding...

[fredsa]

It sounds like this is simply an issue of a race condition.

I can simulate the failure case by making one of the files disappear
briefly, and then reappear, e.g. this works pretty well on my system:
$ cp app/js/app.js app/js/.app.js.swp && rm app/js/app.js && sleep .1 && cp
app/js/.app.js.swp app/js/app.js && rm app/js/.app.js.swp

Here's the relevant bits of my .vimrc:
"backupcopy

" "yes" make a copy of the file and overwrite the original one

" "no" rename the file and write a new one

" "auto" one of the previous, what works best

:set backupcopy=no

And here's the chokidar version information:

$ readlink -f $(which testacular)
/usr/local/google/home/fredsa/lib/node-v0.8.15-linux-x64/lib/node_modules/testacular/bin/testacular

$ cd
/usr/local/google/home/fredsa/lib/node-v0.8.15-linux-x64/lib/node_modules

$ npm list chokidar
/usr/local/google/home/fredsa/lib/node-v0.8.15-linux-x64/lib
└─┬ [email protected]
└── [email protected]

On Mon, Dec 24, 2012 at 4:43 AM, Vojta Jina [email protected]:

@fredsa https://github.com/fredsa thanks for heads up.

I can't reproduce this, even doing rm some.js; echo "xxx" > some.jstriggers correctly "CHANGE" on Mac (because chokidar does stat polling).

Can you make sure the chokidar dependency is on the latest version ?
CD to the path where testacular is installed and do npm list chokidar,
you should have 0.5.0.

If it triggers REMOVE and then ADD, it would be ok, but it's weird, that
it only triggers the REMOVE event and that misses the adding...

—
Reply to this email directly or view it on GitHubhttps://github.com/vojtajina/testacular/issues/199#issuecomment-11660175.

Fred Sauer
[email protected]

[vojtajina]

@fredsa thanks for this info!

I looked into it a bit and filed an issue with chokidar https://github.com/paulmillr/chokidar/issues/35

[maksimr]

+1 have same problem with vim. On change, Karma remove and not add file.
Need save the file several times that he again added.

Karma version: 0.10.4

Workaround with editor settings is not robust and team members have different editors

[matchew]

another +1

using PyCharm with a VIM emulator plugin I am having same issue.

---

"karma": "^0.12.1",
"grunt-karma": "^0.8.2",
"karma-jasmine": "^0.1.5",

[vojtajina]

See #974

[fresheneesz]

This problem still exists, why is this issue closed? I'm having this problem on intelliJ.

[demisx]

Same issue when deleting some files in SublimeText 3.

[szxp]

This problem still exists. I use vim and when I change a file several times Karma sometimes removes the file, sometimes reloads the file.
Karma version: 0.12.31

[pawer13]

Same happens with IntelliJ v13: Sometimes a file is detected as removed and I've never seen it being added again (actually it is actively ignored, see last line of my log) The only solution is restarting Karma.
Here is the log generated while modifying a file:

INFO [watcher]: Removed file "C:/dev/js/gtk/lib/Service.js".
DEBUG [karma]: List of files has changed, trying to execute
DEBUG [karma]: All browsers are ready, executing
DEBUG [web-server]: serving (cached): c:\dev\node_modules\karma\static/context.html
PhantomJS 1.9.8 (Windows 7) Service #getWager should request (method GET) a wager FAILED
    TypeError: 'undefined' is not a constructor (evaluating 'new Service(END_POINT,null,API_KEY)')
        at C:/dev/js/test/Service_test.js:9
    TypeError: 'undefined' is not an object (evaluating 'service.getWager')
        at C:/dev/js/test/Service_test.js:19
PhantomJS 1.9.8 (Windows 7) Service #updateWager should send a wager update (method PUT) FAILED
    TypeError: 'undefined' is not a constructor (evaluating 'new Service(END_POINT,null,API_KEY)')
        at C:/dev/js/test/Service_test.js:9
    TypeError: 'undefined' is not an object (evaluating 'service.updateWager')
        at C:/dev/js/test/Service_test.js:32
PhantomJS 1.9.8 (Windows 7): Executed 4 of 4 (2 FAILED) (0.051 secs / 0.009 secs)
WARN [web-server]: 404: /game_somewhere//apikey.json
DEBUG [watcher]: Changed file "C:/dev/js/gtk/lib/Service.js" ignored. Does not match any file in the list.

[KamilSzot]

The first version that thinks one of the watched files got deleted is 0.12.5.
I'm on windows. On 0.12.4 and earlier, 0.12.0 for example, problem does not occur.

[JamesBender]

I'm seeing this issue still as well in both WebStorm and Sublime 3. Using Windows 8.1 and Karma 0.12.31

[jagandecapri]

I'm using VIM 7.4 and having the same problem.
Using Chrome as the testing browser.
Only solution I can think of is to restart Karma.
Karma version : 0.12.31

[hoxu]

Karma 0.12.35 + vim, still having this issue. I get "Removed file" occasionally, followed by "Added file" when I save again.

[dignifiedquire]

@hoxu this is most likely an issue https://github.com/paulmillr/chokidar the filewatcher we use itself. So if you have a detailed report, please file an issue there.

[s-o-r-o-u-s-h]

same issue with karma v0.13.21

[rryter]

I just ran into this problem with karma 1.1.2
I can confirm disabling 'safe write' in Intellij _"solves"_ the problem.

[lvanderbijl]

having the same problem in webstorm 2017 using typescript spec files.

Karma 1.5.0

CORRECTION: Disabling safe write does NOT fix the issue for me

[StrikeNP]

I am also having this issue, 6 years after it's original posting. What is going on here?

[johnjbarton]

Without practical steps to reproduce this issue, only someone who experiences the issue often will be able to investigate and find a fix. The issue is likely with the file-watcher library, not karma-runner itself.