DOCSEARCHbyAlgolia
Submit Docs

changedetection.io

repository·master·Indexed 12 days ago

https://github.com/dgtlmoon/changedetection.io

An automated website change detection tool that monitors web pages for updates and sends notifications via Discord, Email, Slack, and Webhooks. Features include AI-powered summaries, Playwright-based interactive browser steps, and visual change detection using OpenCV and pixelmatch. Supports specialized post-processors for e-commerce restock tracking and a flexible Watch/Tag API schema.

Tokens
14.8K
Snippets
38
Records
67
Agent score
96%

What's inside changedetection.io

  1. Use Browser Steps for interactive web monitoring

    master

    If a website requires interaction before content can be detected (such as logging in, clicking buttons, accepting cookies, or filling out search forms), you can use the Browser Steps configuration.

    Browser Steps allow you to define a sequence of actions to perform before the change detection occurs. This feature requires a Playwright content fetcher to be enabled. Once steps are configured, you can use the Visual Selector tab to refine the specific elements you wish to monitor.

  3. How UI Stats Tab Plugins work

    master

    UI Stats Tab Plugins allow you to inject custom statistics or visualizations into the 'Stats' tab of a watch's Edit page.

    To create one, you must implement the ui_edit_stats_extras hook using the global_hookimpl decorator. The hook receives a watch object as an argument. Your implementation should calculate the desired data and return a string containing HTML content. This HTML will be rendered directly within the Stats tab UI.

    import pluggy

global_hookimpl = pluggy.HookimplMarker("changedetectionio")

@global_hookimpl
def ui_edit_stats_extras(watch):
    # ... logic to calculate stats ...
    return "<div>Your HTML Content</div>"

  4. Configure web page filters (XPath, JSONPath, jq, CSS)

    master

    changedetection.io supports several selector and filtering languages to target specific parts of a webpage or parse data:

    • XPath (1.0): Supports LXML re:test, re:match, and re:replace.
    • CSS Selectors: Standard CSS pathing.
    • JSONPath: For navigating JSON structures.
    • jq: Recommended for complex JSON parsing, filtering, and logic (e.g., comparing values).

    Monitoring JSON APIs and Embedded JSON

    You can monitor JSON data by using json: or jq: prefixes in your filters. This allows you to:

    • Parse and restructure JSON data.
    • Extract JSON embedded within <script> tags in HTML (e.g., application/ld+json).

    Example: If a page contains embedded JSON, using json:$..price or jq:..price will extract the price value directly.

    json:$..price
# or
jq:..price

  5. CJK italic policy for translators

    master

    In Chinese, Japanese, and Korean (CJK) locales (ja, zh, zh_Hant_TW), standard <i> tags can reduce legibility due to mechanical slanting. Follow these substitution rules:

    • General emphasis: Replace <i> with <strong or drop it if emphasis is self-evident.
    • Nested tags: Collapse <strong><i...></i></strong> to <strong...></strong>.
    • UI terms: Wrap UI terms (e.g., "check unique lines") in locale-conventional quotation marks:
      • ja / zh_Hant_TW: Use 「」.
      • zh: Use "".

    To prevent the dennis linter from flagging these as HTML tag mismatches (W303), use a TRANSLATORS comment with dennis-ignore: W303 in your source code or templates.

    {# TRANSLATORS: CJK fonts lack native italics; allow substitution with conventional local styling. dennis-ignore: W303 #}
<p>{{ _('These settings are <strong><i>added</i></strong> to any existing watch configurations.')|safe }}</p>

  6. Interpret Screenshot Change Percentages

    master

    When a change is detected, the processor calculates the percentage of changed pixels. Use this guide to understand the magnitude of the detected change:

    • 0%: Identical images (or below minimum change threshold).
    • 0.1-1%: Minor differences (anti-aliasing, slight rendering differences).
    • 1-5%: Noticeable changes (text updates, small content changes).
    • 5-20%: Significant changes (layout shifts, content additions).
    • >20%: Major differences (page redesign, large content changes).

  8. Implement the Pydantic-as-validator pattern

    master

    To improve security (preventing mass-assignment vulnerabilities like CWE-915) and enforce schemas, use Pydantic models as validators at the boundary rather than replacing the entire domain model with Pydantic objects.

    In this pattern, the underlying storage remains a plain dictionary, allowing existing code to continue using watch['x'] style access. The Pydantic model is used only to validate input at the API/Form boundary and to ensure data integrity before writing back to storage.

    Key Rules:

    • Use model_config = ConfigDict(extra='forbid') to reject unknown keys and prevent attackers from smuggling extra fields into storage.
    • Match WTForms field names to the Pydantic/storage field names directly to avoid using Field(alias=...). Using aliases introduces complexity and merge bugs during model_dump(by_alias=True) operations.
    • Avoid model_copy(update=...) for merges as it bypasses type coercion and extra='forbid' checks. Instead, use model_validate({**old.model_dump(), **updates}).
    # The recommended boundary pattern

# 1. Read and validate existing data
settings = LLMSettings.model_validate(
    datastore.data['settings']['application'].get('llm') or {}
)

# 2. Prepare form input and strip protected/system-managed fields
form_input = dict(form.data.get('llm') or {})
for protected in LLMSettings.PROTECTED_FIELDS:
    form_input.pop(protected, None)

# 3. Merge and re-validate
merged = LLMSettings.model_validate({**settings.model_dump(), **form_input})

# 4. Write back as a plain dict
datastore.data['settings']['application']['llm'] = merged.model_dump()

  9. Understand Change Detection Post-Processors

    master

    Post-processors allow the system to switch between different domain-specific logic for handling detected changes. Instead of a generic diff, a post-processor can interpret the change based on the context of the website being monitored.

    Currently supported logic types include:

    • text_json_diff: The standard handler for comparing text or JSON structures.
    • restock_diff: A specialized handler for e-commerce. It specifically looks for text indicating a product is 'out of stock'; if that text is absent, it assumes the product is in stock.

  10. How plugins are loaded in changedetection.io

    master

    The plugin system in changedetection.io supports two primary loading methods:

    1. Built-in plugin directories: Directories defined within the application's codebase.
    2. External packages: Plugins can be distributed as external packages using setuptools entry points.

    To register or add a new directory for plugin scanning, you must modify the plugin_dirs dictionary located in pluggy_interface.py.

  11. Use AI for smart change detection and summaries

    master

    You can connect an LLM (OpenAI, Gemini, Anthropic, Ollama, etc.) to filter noise and generate human-readable summaries of changes.

    AI Change Detection Rules

    Instead of receiving an alert for every minor change, you can write plain-English intent rules. The AI evaluates the detected diff against your intent and suppresses irrelevant changes. Example intents:

    • "notify me only when the price drops below $50"
    • "alert me when the item comes back in stock"
    • "ignore navigation and footer changes"

    AI Change Summaries

    Instead of raw diffs, notifications can include plain-language summaries like "Price dropped from $89.99 to $67.00" or "3 new products added to the listing".

    Supported Providers

    • Cloud Providers: OpenAI, Gemini, Anthropic, etc.
    • Local/Self-hosted: Ollama, vLLM, LM Studio, or any OpenAI-compatible endpoint. To use a self-hosted server, select the OpenAI-compatible (vLLM, LM Studio, llama.cpp) option in the provider dropdown and point it to your server's /v1 URL.

    Note: This feature is available in the subscription/hosted service as of June 2026.

  12. How real-time updates and async workers work together

    master

    The real-time system architecture bridges asynchronous watch processing with the web interface using the following components:

    1. Async Workers: Run in a separate asyncio event loop thread to process web watches.
    2. Queue: An AsyncSignalPriorityQueue is used to distribute jobs to workers.
    3. Communication: Workers communicate with the Socket.IO server using Blinker signals. When a worker completes a task or a watch state changes, a signal is emitted.
    4. Socket.IO Server: Listens for these signals and uses direct emit() calls to push updates to connected web clients.

    This separation ensures that heavy browser automation (like Playwright) does not block the real-time update stream.