DOCSEARCHbyAlgolia
Submit Docs

LoggiFly Documentation

repository·main·Indexed 23 days ago

https://github.com/clemcer/loggifly

A lightweight monitoring tool that watches Docker, Docker Swarm, or Podman container logs for plain text or regex patterns. LoggiFly can trigger automated container lifecycle actions (stop, restart, start), send notifications via Ntfy or Apprise, and execute predefined commands in OliveTin via its API.

Tokens
41.8K
Snippets
97
Records
165
Agent score
82%

What's inside LoggiFly

  1. Overview of LoggiFly

    main

    LoggiFly is a lightweight tool designed to monitor Docker container logs for predefined keywords or regular expression (regex) patterns. When a match is detected, it can send notifications through various channels and perform automated actions on the monitored containers.

    Key Capabilities:

    • Pattern Detection: Supports plain text, regex, and multi-line log detection.
    • Automated Actions: Can trigger a container restart or stop a container entirely upon detecting critical keywords.
    • Notification Channels: Integrates with Ntfy or Apprise (which supports 100+ services like Slack, Discord, and Telegram).
    • Contextual Alerts: Can automatically attach log files to notifications for better debugging context.
    • Dynamic Configuration: Automatically reloads the config.yaml file when changes are detected.
    • Environment Support: Works with Docker, Docker Swarm, and Podman, and can connect to multiple remote Docker hosts.

  2. What is LoggiFly and its core capabilities

    main

    LoggiFly is an open-source tool designed to watch Docker container logs and events. It monitors for specific keywords or container events and responds by sending notifications or triggering automated actions.

    Key capabilities include:

    • Automated Container Management: Automatically restart, start, or stop containers based on detected events.
    • Notification Customization: Send notifications with custom templates, extract relevant information from logs, and attach log files to notifications.
    • Action Triggers: Trigger OliveTin actions or other external workflows.
    • Monitoring Use Cases: Catch security breaches (e.g., failed logins), debug crashes with log context, and monitor custom application behaviors.

  3. What is LoggiFly

    main
    LoggiFly is an open-source tool designed to watch Docker container logs and events. It monitors for specific keywords or container events and responds by sending notifications or triggering automated actions. It is intended as a lightweight alternative to a full logging stack for monitoring containers, catching security breaches, debugging crashes, or automating container lifecycle management (start/stop/restart) based on log patterns.

  4. Automate actions and notifications in LoggiFly

    main

    LoggiFly can perform several actions when a trigger (keyword, regex, or event) is matched:

    • Notifications: Send alerts with customizable templates.
    • Container Management: Automatically restart, start, or stop containers.
    • OliveTin Integration: Trigger actions in OliveTin.
    • Log Context: Attach relevant log files to your notifications.
    • Data Extraction: Format messages by extracting only the specific relevant information from the log lines.

  5. Configure LoggiFly settings

    main

    LoggiFly can be configured using three different methods, allowing for varying levels of granularity:

    1. Environment Variables: For global configuration.
    2. config.yaml file: A highly flexible configuration file that supports global settings, per-container settings, and per-trigger settings (e.g., specific rules for a keyword, regex, or container event).
    3. Docker Labels: Configure monitoring directly on the containers you want to watch using Docker labels.

  6. How settings priority and inheritance work in Loggifly

    main

    Loggifly applies settings at three different levels. When a setting is defined at multiple levels, the most specific level takes precedence according to the following priority:

    trigger > container > global

    Levels of application:

    1. Global: Defined under settings or notifications.
    2. Per Container: Defined under containers.
    3. Per Trigger: Defined under keywords, regex, or container_event within a container.

    Note on Lists: Settings that are lists (such as excluded_keywords) are merged rather than overwritten when applying priority.

    Example configuration showing inheritance:

    containers:
  container1:
    notification_cooldown: 15
    title_template: "{container}: {keywords}"
    keywords:
      - keyword: "error"
        notification_cooldown: 0
        title_template: "Error in {container}"

settings:
  notification_cooldown: 5
  title_template: "{keywords} found in {container}"

  7. Use Rule Groups to share settings across hosts

    main

    In v2, the old hosts: block is replaced by containers.groups:. This allows you to apply a shared scope (like a specific host) to a set of rules without repeating the host configuration for every rule.

    containers:
  groups:
    - scope:
        hosts: ["my-remote-host"]
      rules:
        - container_name: my-app
          keywords:
            - keyword: error
        - container_name: my-other-app
          keywords:
            - keyword: warning

  8. What the LoggiFly v1 to v2 migration script does

    main

    The migration script performs the following transformations to convert a v1.8.0 configuration to the v2 format:

    1. Validation: Validates input against the v1.8.0 model (including legacy migrations up to v1.8).
    2. Field Renaming:
      • action $\rightarrow$ container_action
      • hide_regex_in_title $\rightarrow$ hide_full_regex
      • excluded_keywords $\rightarrow$ ignore_keywords
    3. Structural Changes: Moves modular settings from the settings: section to a new defaults: section.
    4. Rule Conversion:
      • Converts containers and swarm_services entries to the v2 rules format.
      • Converts hosts entries to scoped rules.
    5. Final Validation: Validates the resulting configuration against the v2 model.

    Important: The original input file is never modified. Always review the output file and check the conversion logs for warnings regarding fields that could not be migrated automatically.

  9. Configure hierarchical settings for containers and keywords

    main

    Loggifly allows a hierarchical configuration where settings (like apprise_url, ntfy_topic, title_template, etc.) can be defined at the container level and overridden at the keyword/regex level.

    Precedence Rule: When multiple keywords with the same setting (e.g., title_template) are found in a log line, the one listed first in the YAML takes precedence.

    containers:
  container5:
    apprise_url: "discord://webhook-url"  
    ntfy_tags: closed_lock_with_key   
    ntfy_priority: 3
    ntfy_topic: container3
    attachment_lines: 50
    title_template: '{keywords} found in {container}'
    notification_cooldown: 2  
    attach_logfile: true
    action_cooldown: 60 
  
    keywords:
      - critical

      - regex: 'download.*failed' 
        ntfy_tags: partying_face   
        ntfy_priority: 5
        ntfy_topic: error
        attachment_lines: 10

      - keyword: timeout
        apprise_url: "discord://webhook-url" 
        title_template: '{container} restarted because these keywords were found: {keywords}'
        notification_cooldown: 10
        attach_logfile: true

  10. Migrate template syntax to Jinja2

    main

    In v2, title_template and message_template fields have switched from simple brace interpolation to Jinja2 templating syntax.

    Example Change:

    • v1: Email: '{2}'
    • v2: Email: '{{ email }}' (using named capture groups from the regex)

    Ensure your regexes use named capture groups (e.g., (?P<email>...)) to make templates more readable and compatible with the new syntax.

  11. Extract additional fields from logs

    main

    You can extend the available template fields by extracting data directly from your logs:

    1. JSON logs: Every key in a JSON log object becomes an available template field.
    2. Plain text logs: Use regex named capturing groups to define custom fields.

    Field Precedence: If a field name exists in multiple sources, the priority is: Fields from JSON logs > Fields from regex > Default fields.

  12. Understand the top-level structure of config.yaml

    main

    A standard config.yaml file for LoggiFly uses the following top-level keys to organize settings, sources, and notifications:

    • version: The configuration schema version.
    • global: Global keywords and default settings applied across all sources.
    • containers: Configuration for monitoring Docker containers.
    • swarm: Configuration for monitoring Docker Swarm services.
    • notifications: Settings for how alerts and notifications are dispatched.
    • settings: General application-level settings.
    version: 2
global:
containers:
swarm:
notifications:
settings: