DOCSEARCHbyAlgolia
Submit Docs

watchdog

repository·master·Indexed 27 days ago

https://github.com/gorakhargosh/watchdog

A Python library and shell utility suite for monitoring file system events. It provides a cross-platform API to observe file creations, modifications, and deletions using native APIs (inotify, FSEvents, kqueue, ReadDirectoryChangesW) or a polling fallback. Includes the watchmedo CLI utility for logging events and executing shell commands based on file patterns.

Tokens
4.9K
Snippets
13
Records
35
Agent score
91%

What's inside watchdog

  1. Watchdog platform support and behavior

    master

    Watchdog uses native APIs where possible and falls back to periodic disk polling when native APIs are unavailable.

    Supported Platforms

    • Linux 2.6+: Uses inotify.
    • macOS: Uses FSEvents (preferred) or kqueue.
    • BSD: Uses kqueue.
    • Windows Vista and later: Uses ReadDirectoryChangesW.
    • Fallback: OS-independent polling via directory tree snapshots.

  2. Install Watchdog with watchmedo utility

    master

    To use the watchmedo shell utility, install Watchdog with the [watchmedo] extra.

    From PyPI:

    python -m pip install -U 'watchdog[watchmedo]'

    From source:

    python -m pip install -e '.[watchmedo]'
    python -m pip install -U 'watchdog[watchmedo]'

  5. Install watchdog from the code repository

    master

    Clone the repository recursively and install in editable mode. To include the watchmedo utility, use the [watchmedo] extra.

    $ git clone --recursive git://github.com/gorakhargosh/watchdog.git
$ cd watchdog
$ python -m pip install -e .

# or to install the watchmedo utility:
$ python -m pip install -e '.[watchmedo]'

  6. Implement a basic file system monitor with Observer and FileSystemEventHandler

    master

    To monitor file system changes, follow these steps:

    1. Create a subclass of watchdog.events.FileSystemEventHandler and override event methods (like on_any_event).
    2. Instantiate watchdog.observers.Observer.
    3. Use observer.schedule(event_handler, path, recursive=True) to attach your handler to a specific path. Setting recursive=True ensures sub-directories are also monitored.
    4. Start the observer using observer.start() or by using the observer as a context manager.
    import time

from watchdog.events import FileSystemEvent, FileSystemEventHandler
from watchdog.observers import Observer


class MyEventHandler(FileSystemEventHandler):
    def on_any_event(self, event: FileSystemEvent) -> None:
        print(event)


event_handler = MyEventHandler()
observer = Observer()
observer.schedule(event_handler, ".", recursive=True)
observer.start()
try:
    while True:
        time.sleep(1)
finally:
    observer.stop()
    observer.join()

  7. Set up a development environment for Watchdog

    master

    To contribute to Watchdog, you need to set up a local development environment using a virtual environment and install the package in editable mode.

    Prerequisites

    Ensure your system has the following installed:

    1. Python
    2. git
    3. XCode (on macOS)

    Setup Steps

    1. Fork the repository to your GitHub account.
    2. Clone your fork and create a virtual environment:
    $ git clone https://github.com/gorakhargosh/watchdog.git
$ cd watchdog
$ python -m venv venv
    1. Install the package in editable mode:

    On Linux:

    $ . venv/bin/activate
(venv)$ python -m pip install -e '.'

    On Windows:

    > venv\Scripts\activate
(venv)> python -m pip install -e '.'
    git clone https://github.com/gorakhargosh/watchdog.git
cd watchdog
python -m venv venv

  8. Monitor CIFS shares using PollingObserver

    master

    When monitoring changes on CIFS (Common Internet File System) shares, standard OS-level notifications may not work. You must explicitly use PollingObserver instead of the default observer.

    from watchdog.observers.polling import PollingObserver as Observer
    from watchdog.observers.polling import PollingObserver as Observer

  9. Use the watchmedo CLI utility

    master

    The watchmedo command is a shell utility provided by Watchdog to monitor file system events and trigger actions. It supports several subcommands for logging, executing shell commands, and managing long-running processes.

    Commonly used subcommands:

    • log: Logs file system events to the console.
    • shell-command: Executes a shell command in response to matching events.
    • auto-restart: Starts a long-running subprocess and restarts it when matched events occur.
    • tricks: Executes custom "tricks" defined in a YAML configuration file.
    • generate-tricks-yaml: Generates YAML configuration for custom tricks.

  10. Configure Watchdog tricks via tricks.yaml

    master

    The watchmedo utility can execute 'tricks' defined in a tricks.yaml file. Tricks are event handlers that subclass watchdog.tricks.Trick. The directory containing tricks.yaml is automatically monitored. Each trick is initialized with the keys provided in the YAML file.

    tricks:
- watchdog.tricks.LoggerTrick:
    patterns: ["**/*.py", "**/*.js"]
- watchmedo_webtricks.GoogleClosureTrick:
    patterns: ['**/*.js']
    hash_names: true
    mappings_format: json                  # json|yaml|python
    mappings_module: app/javascript_mappings
    suffix: .min.js
    compilation_level: advanced            # simple|advanced
    source_directory: app/static/js/
    destination_directory: app/public/js/
    files:
      index-page:
      - app/static/js/vendor/jquery*.js
      - app/static/js/base.js
      - app/static/js/index-page.js
      about-page:
      - app/static/js/vendor/jquery*.js
      - app/static/js/base.js
      - app/static/js/about-page/**/*.js

  11. Configure file descriptor limits for kqueue (macOS/BSD)

    master

    When using kqueue on macOS or FreeBSD, you must increase the number of allowed file descriptors to be greater than the number of files being monitored. You can do this by editing your ~/.profile and adding:

    ulimit -n 1024

    or

    ulimit -n unlimited
    ulimit -n 1024