Watchdog: Modified files trigger more than one event on Python 3

[bastula]

On Python 3 (3.5), modified file events are showing up more than once.
This does not occur on Python 2.7.11 with the same exact environment.

Please see the following question on SO for an example (not my post, but same symptoms):

https://stackoverflow.com/questions/32861420/watchdog-getting-events-thrice-in-python-3

[nueverest]

Same here.

[carlcarl]

What's your OS environment? (ex: Ubuntu/Archlinux)

[bastula]

Mac OS X 10.11.4, Python 3.5.1

[nueverest]

Windows 8.1, Python 3.5

Windows Vista, Python 3.4

[valencik]

Mac OS X 10.10.5, Python 3.5.1

[g8gg]

CentOS Linux release 7.3.1611 (Core)
Python 3.6.0 :: Anaconda 4.3.1 (64-bit)
Linux 3.10.0-514.21.1.el7.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux

[andrenogueiramartins]

Win 10x64, Python 3.6.5

[alexstrzelecki-gkn]

Win 7x64, Python 3.6.4

[BoboTiG]

If someone wants to help, we will be very greatful :)

[alonme]

Does this happen in python 3.7 to anyone?
Please share more info regarding what exactly you are doing for this to happen.
What are you monitoring? what is the modification that you do

[dfitzpatrick]

I experience this in on_any_event) in a conditional block that checks if the instance is a FileModifiedEvent or a FileCreatedEvent

I open up the file I want to edit and type a word. When I hit save on the file it generates 4 events.

[choogiesaur]

Having this bug as well, I'm on Ubuntu.

I found that creating a file in the watched directory generates the following:
FileCreatedEvent followed by
DirModifiedEvent for parent directory
FileModifiedEvent for the newly created file (??)

And copying the same file to the directory while the file already exists generates anywhere from three to four FileModifiedEvents:

watcher_1      | 2020-04-15 22:34:42,797 — __main__ — INFO — Event: <FileModifiedEvent: src_path='/Eicon/input/US-RGB-8-epicard.dcm'>
watcher_1      | 2020-04-15 22:34:42,799 — __main__ — INFO — Event: <FileModifiedEvent: src_path='/Eicon/input/US-RGB-8-epicard.dcm'>
watcher_1      | 2020-04-15 22:34:42,799 — __main__ — INFO — Event: <FileModifiedEvent: src_path='/Eicon/input/US-RGB-8-epicard.dcm'>

[mmattklaus]

I experienced the exact same problem where the FileModifiedEvent executed twice.
I solve that by measuring the time between both triggers. My observation showed that both events executed in less than a 1-second difference (that is, time.time() - last_execution_time < 1).

So my walkaround was this:

import time

last_trigger = time.time()

def on_config_modified(event):
    global last_trigger
    if event.src_path.find('~') == -1 and (time.time() - last_trigger) > 1:
        config = reload_config()

.
.
.

From the code above,

• I first registered the start time of the script (in a global variable)
• Then, whenever the on_config_modified function is triggered (according to my watchdog implementation), I do two things:

1. I check that the modified src_file name doesn't contain a tilt (~) sign (...event.src_path.find('\~') == -1). This was the case for my YAML config file which I watched. It is because my config.yaml file first gets a temporal file (with the tilt sign), before it's finally modified.

2. The second condition ((time.time() - last_trigger) > 1) is where the solution happens. I compare the difference in time since the function was last executed. If that difference in time is greater than 1, then I reload my configuration file (by calling the reload_config function).

I hope someone finds this useful.
Best regards.

[pmundt]

I had the same issue, and worked around it in two different ways:

• Using a timer-based debouncing of general modifications (similar as above)
• Stashing the path at creation time and checking to see if the modification is part of a create/modify pair notification, which inotify seems to raise for newly created files.

I'm not sure what other people are using it for, but I found the creation event triggering too fast for my needs, it was triggering at the point where the filesystem allocated a dentry, but well before the file contents had been written out, which was a bit counterintuitive.

    def on_created(self, event):
        self.plugins_pending.append(event.src_path)

    def on_modified(self, event):
       module_name = os.path.splitext(os.path.basename(event.src_path))[0]

        # Look for a completion to an earlier creation event
        for pending in self.plugins_pending:
            if pending == event.src_path:
                self.load_plugin(module_name, event.src_path)
                self.plugins_pending.remove(pending)
                return

The rest of on_modified implements the debouncing logic for the general modification case.

Hope this ends up saving someone some time!