Robusta Documentation

repository·master·Indexed 25 days ago

https://github.com/robusta-dev/robusta

Robusta Classic is a rule-based alert enrichment engine for Kubernetes that integrates with Prometheus and Alertmanager-compatible sources. It provides smart grouping, alert enrichment with pod logs, self-healing auto-remediation, and advanced routing to notification sinks such as Slack, MS Teams, PagerDuty, and Jira. It also supports AI-powered root cause analysis via HolmesGPT and cost management through KRR integration.

Tokens
153.3K
Snippets
446
Records
826
Agent score
84%

What's inside Robusta

  1. Overview of Robusta AI-powered SRE Agent

    master

    Robusta is an AI-powered SRE agent designed to automatically investigate alerts and identify root causes. It is built on HolmesGPT and provides the following capabilities:

    • Automatic investigation: Every alert undergoes AI-powered root cause analysis.
    • Data source integration: Connects to existing monitoring, ITSM, cloud tools, and MCP servers to gather evidence.
    • On-demand investigation: Allows users to chat with the agent by tagging HolmesGPT in Slack or Teams.
    • Centralized management: Use the Robusta Platform to manage SRE agents, triage alerts, and review investigation timelines.

    Robusta is available in three deployment models: SaaS, self-hosted, or open source.

  2. Overview of Robusta Classic

    master

    Robusta Classic is a rule-based alert enrichment engine designed for Kubernetes. It integrates with Prometheus via webhooks to enhance alerts with additional context.

    Key capabilities include:

    • Smart Grouping: Reduces notification spam by using Slack threads.
    • Alert Enrichment: Automatically gathers pod logs and other relevant data alongside alerts.
    • Self-Healing: Allows defining auto-remediation rules for faster incident response.
    • Advanced Routing: Routes notifications based on criteria like team or namespace.
    • Problem Detection: Generates Kubernetes-native alerts (e.g., for OOMKills or failing Jobs) without requiring PromQL.
    • Change-Tracking: Correlates alerts with Kubernetes resource changes and rollouts.
    • Auto-Resolve: Automatically updates external systems like Jira when alerts are resolved.
    • AI Investigation: Optional integration with HolmesGPT for AI-powered root cause analysis.
  3. Playbook Basics

    master

    Playbooks are deterministic rules used to respond to alerts and unhealthy conditions in a Kubernetes cluster. Each playbook consists of two main parts:

    1. Trigger: A condition that defines when the automation runs (e.g., a specific Kubernetes event or Prometheus alert).
    2. Action: Defines what the automation does (e.g., fetching logs, running a command, or creating a notification).

    Playbooks operate like pipelines: Events enter Robusta $\rightarrow$ Triggers are checked $\rightarrow$ Matching triggers fire $\rightarrow$ Playbook actions execute in order $\rightarrow$ Findings (notifications) are sent to configured sinks.

  4. Note on Mattermost sink vs HolmesGPT

    master

    The Mattermost sink is considered a legacy sink in Robusta classic. It is deterministic, meaning it forwards every notification unchanged to the destination.

    For new setups, it is recommended to use HolmesGPT instead. HolmesGPT uses AI to investigate alerts, surface root causes, and escalate only actionable items, reducing notification noise.

  5. Understand Robusta Sinks vs. HolmesGPT

    master

    Robusta Sinks are a legacy feature used to forward notifications to fixed destinations. Sinks are deterministic: they send every notification, unchanged, to the destination, requiring you to manually read and prioritize each one.

    For new setups, it is recommended to use HolmesGPT instead. Unlike Sinks, HolmesGPT uses AI to investigate alerts, surface likely root causes, and escalate only what needs attention, resulting in fewer and more actionable notifications.

    Setup Options:

  6. Use Robusta Classic for rule-based enrichment

    master

    Robusta Classic is the original open-source alert engine that provides deterministic, rule-based enrichment. It uses configurable playbooks to automatically attach pod logs, resource state, and related events to alerts before routing them to notification channels like Slack, Teams, or PagerDuty.

    Robusta Classic can be installed as part of the Agent and runs alongside HolmesGPT.

  7. Understand the Robusta Architecture

    master

    Robusta operates using a distributed architecture consisting of an in-cluster Agent and a centralized Robusta Platform (SaaS or self-hosted). This separation ensures security by keeping data access local to your environment.

    Components

    • Agent (In-Cluster): Runs inside your Kubernetes cluster. It includes HolmesGPT and is responsible for fetching data from your local data sources. It can also track Kubernetes deploys, changes, and events.
    • Robusta Platform: A centralized control plane (SaaS or self-hosted) used to manage SRE agents, triage alerts, view AI-powered root cause analysis, and interact with the HolmesGPT Slack/Teams bot.
    • Data Sources: External environment data used by HolmesGPT during investigations.

    Security Model

    • Local Data Access: The Agent fetches data from sources directly within your cluster. The Robusta Platform never connects to your data sources.
    • Outbound Connectivity: For SaaS users, connectivity is outbound-only; no inbound access to your cluster is required.
    • RBAC: The Agent runs with configurable Kubernetes RBAC permissions.
  8. Track Kubernetes resource changes

    master

    By default, Robusta tracks all changes to the following resources and sends them to the Robusta UI for correlation with Prometheus alerts and errors:

    • Deployments
    • DaemonSets
    • StatefulSets

    Note: These changes are not sent to other sinks (like Slack) by default to prevent spam. To selectively track specific changes in Slack, you must configure [Notification Routing].

  9. Understand the relationship between Triggers and Events

    master

    In Robusta, triggers are the mechanism that detects changes (e.g., a pod update) and generates event objects. These event objects are then passed to your playbook action functions.

    Lifecycle of a Robusta Event:

    1. A resource (like a pod) changes.
    2. The Kubernetes API Server notifies Robusta.
    3. Robusta checks if any configured triggers (e.g., on_pod_update) are activated.
    4. If activated, the trigger converts the API Server data into a concrete event object (e.g., PodEvent).
    5. The event object is passed to all registered playbook actions.
  10. Understand Robusta Notifications and Routing

    master

    Robusta allows you to send notifications to various destinations and route them intelligently based on alert attributes like type, namespace, severity, and more.

    Key concepts include:

    • Sinks: The destinations where notifications are sent (e.g., Slack, Teams, Email).
    • Routing: Rules that determine which alerts are sent to which sinks.
    • Grouping: A mechanism to thread related alerts together to reduce noise, particularly in Slack.
    • Silencing: The ability to temporarily disable specific notifications.
  11. Available Robusta Playbook Triggers

    master

    Robusta playbooks can be executed automatically based on several event types. The supported trigger categories are:

    • Kubernetes (API Server): Triggers based on Kubernetes events and changes.
    • Prometheus and AlertManager: Triggers based on alerts from Prometheus or AlertManager.
    • Helm Releases Monitoring: Triggers based on changes in Helm releases.
    • Scheduled: Triggers based on a defined schedule (cron-like).
    • Webhooks: Triggers via incoming HTTP webhooks.
    • Manual Triggers: Triggers initiated manually by a user.
    • ElasticSearch: Triggers based on ElasticSearch monitors.