Watchdog: Uncatchable OSError: Permission Denied

[hgrecco]

Hi,

I use watchdog to monitor for new files in the filesystem and trigger automated processing of them. What happens is that the thread does not have access to certain files and crashes. Is there a way to provide to watchdog an error handler? Or that it continue monitoring for other files?

Below is an example of a traceback.

Hernan

  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 530, in **bootstrap_inner
    self.run()
  File "/Users/X/Documents/code/envs/Y/lib/python2.7/site-packages/watchdog/observers/api.py", line 192, in run
    self.queue_events(self.timeout)
  File "/Users/X/Documents/code/envs/Y/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 703, in queue_events
    new_snapshot)
  File "/Users/X/Documents/code/envs/Y/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 548, in _queue_dirs_modified
    self.queue_event(DirCreatedEvent(directory_created))
  File "/Users/X/Documents/code/envs/Y/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 520, in queue_event
    self._register_kevent(event.src_path, event.is_directory)
  File "/Users/X/Documents/code/envs/Y/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 470, in _register_kevent
    self._descriptors.add(path, is_directory)
  File "/Users/X/Documents/code/envs/Y/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 257, in add
    self._add_descriptor(KeventDescriptor(path, is_directory))
  File "/Users/X/Documents/code/envs/Y/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 342, in __init**
    self._fd = os.open(path, WATCHDOG_OS_OPEN_FLAGS)
OSError: [Errno 13] Permission denied: '/Users/X/Public/Y/FOLDER'

[parente]

This impacts inotify too when it first tries to setup a watch over a directory which happens to contain subdirectories that are not readable / listable.

[rhunwicks]

I'm also getting this error using inotify to watch a directory that includes a deeply nested sub-directory that I don't have permissions to read:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/djsupervisor/management/commands/supervisor.py", line 163, in run_from_argv
    return super(Command,self).run_from_argv(argv)
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/djsupervisor/management/commands/supervisor.py", line 186, in handle
    return method(cfg_file,*args[1:],**options)
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/djsupervisor/management/commands/supervisor.py", line 264, in _handle_autoreload
    observer.start()
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/watchdog/observers/api.py", line 255, in start
    emitter.start()
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/watchdog/utils/__init__.py", line 111, in start
    self.on_thread_start()
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/watchdog/observers/inotify.py", line 121, in on_thread_start
    self._inotify = InotifyBuffer(path, self.watch.is_recursive)
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/watchdog/observers/inotify_buffer.py", line 35, in __init__
    self._inotify = Inotify(path, recursive)
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/watchdog/observers/inotify_c.py", line 187, in __init__
    self._add_dir_watch(path, recursive, event_mask)
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/watchdog/observers/inotify_c.py", line 371, in _add_dir_watch
    self._add_watch(full_path, mask)
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/watchdog/observers/inotify_c.py", line 385, in _add_watch
    Inotify._raise_error()
  File "/home/roger/.virtualenvs/fdw/local/lib/python2.7/site-packages/watchdog/observers/inotify_c.py", line 406, in _raise_error
    raise OSError(os.strerror(err))
OSError: Permission denied

[jacksonofalltrades]

So, uh, this issue has been opened since 2011. Is the owner of this repo no longer maintaining Watchdog? I've just run into this, too.

Please comment, @gorakhargosh

[danilobellini]

Whoa, this issue is quite old!

[...] Is there a way to provide to watchdog an error handler? Or that it continue monitoring for other files?

For now the best you can do is to schedule a non-recursive watcher.

That's something quite scattered in the code, and it's both a watcher creation time handling issue and a emitter handling issue, since the forbidden access might happen at any time (with chmod, chown or SELinux tools). Do you have any idea on how that could be properly fixed in the API (besides a dirty hack)?

Perhaps errno.EACCES (which reminds me of #459) should be handled by default issuing some warning, so that it doesn't forbid the watching of the remaining directories. But that doesn't seem enough for solving this issue.

[petersilva]

This makes basically any use case where there are multiple users unusable. Another event that doesn't work... a user has a directory that is 700 to them, they copy the directory into a watched tree. It won't crash, it just gives an error. chmod 755 so now the watchdog process can read all the files in the tree. it doesn't.

[petersilva]

perhaps just make some sort of os-error an additional event? user can just decide what to do with it.
that would probably be cleanest.