watchfiles

repository·main·Indexed 25 days ago

https://github.com/samuelcolvin/watchfiles

A high-performance file watching and code reloading library for Python that utilizes the Rust-based Notify library for efficient filesystem notifications. It provides synchronous and asynchronous generators (watch, awatch), process execution utilities (run_process, arun_process), and a CLI for executing shell commands or Python functions upon file changes. Features include debouncing, custom filtering via watch_filter, and a low-level RustNotify backend.

Tokens
6.9K
Snippets
19
Records
46
Agent score
81%

What's inside watchfiles

  1. How to use the watch_filter argument

    main

    The watch_filter argument in the watch function (and similar functions) allows you to include or ignore specific file changes.

    It accepts a callable that takes two arguments:

    1. change: An instance of the Change type.
    2. path: The file path as a str.

    The callable must return True to include the change or False to ignore it. You can provide either a custom callable function or an instance of a BaseFilter subclass.

  2. How watchfiles works

    main

    Core Engine

    watchfiles uses the Notify Rust library to handle underlying file system notifications and polling fallbacks.

    Key Features

    • Debouncing: Change grouping (batching changes together rather than firing for every single file) is managed in Rust.
    • Threading Model:
      • Synchronous methods (watch, run_process): The Rust code creates a new thread to watch for changes, so no manual threading logic is required in Python.
      • Asynchronous methods (awatch, arun_process): Uses anyio.to_thread.run_sync to wait for changes in a separate thread, allowing the async loop to remain unblocked.
  3. Access file changes via the WATCHFILES_CHANGES environment variable

    main
    When using watchfiles.run_process (which the CLI uses internally), the most recent file changes are made available to the running process via the WATCHFILES_CHANGES environment variable. This variable contains a JSON-encoded representation of the changes.
  4. Run and restart a Python function with the watchfiles CLI

    main

    You can use the watchfiles CLI to monitor file changes and automatically restart a specific Python function. To do this, provide the module path and function name (e.g., module.function). The CLI can be invoked using the watchfiles command or via python -m watchfiles.

    watchfiles foobar.main
  5. Use the watchfiles CLI

    main

    The watchfiles CLI allows you to watch one or more directories and execute either a shell command or a Python function whenever a file change is detected.

    By default, it watches the current directory. You can specify specific paths to watch as positional arguments. The target is the command or dotted Python function path you want to execute.

  6. Migrate from watchgod to watchfiles

    main

    The library was renamed from watchgod to watchfiles. While the old watchgod package remains available on PyPI for compatibility, it is recommended to migrate to watchfiles to benefit from the new architecture which uses OS file system notifications (via the notify Rust library) instead of file scanning/polling.

    Key API Changes

    When migrating your code to watchfiles, note the following changes to the main methods (watch, awatch, run_process, and arun_process):

    1. Filtering: The watcher_cls argument has been removed. It is replaced by watch_filter, which must be a simple callable.
    2. Multiple Paths: All methods now support watching multiple paths simultaneously.
    3. run_process / arun_process Arguments: Because multiple paths are supported, the target argument is now a keyword-only argument.
    4. Argument Cleanup: Other optional keyword arguments have been cleaned up or renamed. Check the updated documentation for the specific signatures of your target methods.
  7. Run and restart a shell command with the watchfiles CLI

    main

    The watchfiles CLI can monitor file changes and re-run any shell-like command. By default, it watches the current directory and all subdirectories. To run a command, pass the command string as an argument.

    watchfiles 'pytest --lf'
  8. Configure watched directories and file filters in the CLI

    main

    You can customize which directories are watched and which file types trigger a reload using the --filter flag. To watch specific directories, list them as arguments after the command.

    Example: To watch only the src and tests directories and only react to changes in .py files, use --filter python followed by the command and the target directories.

    watchfiles --filter python 'pytest --lf' src tests
  9. Install watchfiles

    main

    You can install watchfiles via PyPI, conda-forge, or from source.

    From PyPI

    Use pip to install the package. Binaries are available for various architectures on Linux, MacOS, and Windows.

    From conda-forge

    Use conda or mamba to install from the conda-forge channel.

    From source

    Installing from source requires Rust stable to be installed on your system.

    Requirements:

    • Python 3.10 to Python 3.15
    # From PyPI
    pip install watchfiles
    
    # From conda-forge
    mamba install -c conda-forge watchfiles
  10. Configure polling and filesystem notifications

    main

    Watchfiles uses filesystem notifications by default but can be configured to use polling via the force_polling and poll_delay_ms arguments.

    Force Polling

    If force_polling=True, the library will use file polling instead of native notifications.

    • Automatic Polling: If force_polling is unset, polling is automatically enabled if the library detects it is running on WSL (Windows Subsystem for Linux).
    • Environment Variable: WATCHFILES_FORCE_POLLING can control this. If it exists and is not false, disable, or disabled, polling is enabled.

    Polling Delay

    If polling is enabled, you can control the delay between iterations using poll_delay_ms (in milliseconds).

    • Environment Variable: WATCHFILES_POLL_DELAY_MS can override the argument value if it is a numeric value.
  11. Create custom filters by inheriting from BaseFilter

    main

    To create a custom filtering logic for file changes, inherit from BaseFilter. BaseFilter provides three mechanisms to ignore files or directories:

    1. ignore_dirs: A sequence of directory names to ignore (e.g., ['.git', '__pycache__']). If any part of the file path matches one of these names, the file is ignored.
    2. ignore_entity_patterns: A sequence of regex patterns applied to the 'entity' name (the last component of the path, e.g., the filename). For example, [r'\.py[cod]$'] ignores Python bytecode files.
    3. ignore_paths: A sequence of full paths (as strings or Path objects) to ignore. If a changed path starts with any of these, it is ignored.

    Subclasses of BaseFilter are used as callables. The __call__(change, path) method receives the Change type and the raw path string, returning True to include the change or False to ignore it.

  12. Configure watchfiles filters

    main

    Filters determine which file changes trigger the target execution. You can specify filters using the --filter flag:

    • default: Uses the DefaultFilter. Supports --ignore-paths to skip specific directories.
    • python: Uses the PythonFilter (optimized for Python projects).
    • all: Uses no filter (watches everything). Note that --ignore-paths is ignored when using all.
    • Custom Filters: You can provide a dotted path to a Python class or function that implements the filter logic. The custom filter must be a subclass of BaseFilter or a callable that accepts (Change, str) and returns a boolean.