DOCSEARCHbyAlgolia
Submit Docs

superlog

repository·main·Indexed 21 days ago

https://github.com/superloglabs/superlog

An open-source agentic telemetry system and observability workspace for OpenTelemetry data. Superlog ingests traces, logs, and metrics, using agentic workflows to group signals into incidents for easier debugging. The community edition includes a web app, API, OTLP ingest proxy, worker processes for incident grouping and background jobs, and a ClickHouse-backed telemetry query system.

Tokens
66.1K
Snippets
197
Records
273
Agent score
76%

What's inside superlog

  1. What is Superlog?

    main

    Superlog is an open-source agentic telemetry system and observability workspace designed for OpenTelemetry data. It ingests traces, logs, and metrics, groups noisy signals into incidents, and provides a local-first product surface for debugging production systems.

    The community edition includes:

    • Web app and API
    • OTLP ingest proxy
    • Worker processes for incident grouping and background jobs
    • Postgres schema and ClickHouse-backed telemetry queries
    • Agent runner interfaces for pluggable investigation runtimes
    • A default community agent runner that records local incident summaries.

  2. Classify issues as Noise vs. Not Noise

    main

    The Superlog agent classifies issues to determine the remediation path:

    Issue is Noise

    An issue is noise if the system is behaving normally or users are not meaningfully impacted (e.g., a user hitting a non-existent landing page).

    • Intended behaviour / no impact: The agent sets the issue to silenced and the incident to resolved.
    • 'One-off' errors: The agent sets the issue to under observation with an escalation trigger.

    Issue is Not Noise

    If end users are impacted, performance is degraded, or a system is inoperative, the agent attempts resolution via:

    • Source code modification: Opening Pull Requests (PRs).
    • Approval prompts: Requesting permission to change infrastructure or databases.
    • Investigation: If no tools are available, the agent submits its findings.

  3. Manage investigation memory via Comments and Project Memory

    main

    Superlog provides two ways to store information to improve investigation quality and personalize the experience for clients:

    1. Comments: These are associated with specific Issues. Both humans and agents can add comments, which are visible to the Agent Run investigating the Incident.
    2. Project Memory: These are dated, free-form text entries associated with a Project. All Project memories are visible to the Agent Run. Agents can add new Memories during investigations in response to PR comments, Slack messages, or user interactions.

  5. Superlog Repository Layout

    main

    The repository is organized into the following main packages:

    • apps/web: Vite/React frontend
    • apps/api: HTTP API
    • apps/proxy: OTLP intake proxy
    • apps/worker: Background workers and agent orchestration
    • packages/db: Drizzle schema and migrations
    • packages/fingerprint: Telemetry fingerprinting helpers

  6. Prioritize integration-first onboarding

    main
    The Superlog product principle for onboarding is 'integration-first'. When designing onboarding flows, favor connected, no-code integrations over manual SDK wiring whenever possible. Implementation details for this model can be found in the apps/web/src/onboarding/ directory under the 'connect-choice' model.

  7. Manage incident responses with silence, observation, and resolution

    main

    The Superlog agent has expanded its response capabilities beyond just opening Pull Requests (PRs). When an issue is detected, the agent can now apply different states to individual issues independently:

    • Silence: Used for clear false positives (e.g., 404 errors on a landing page) to prevent them from triggering noise.
    • Under Observation: Used for issues that log errors but do not appear to break the application (e.g., transient 401 Unauthorized errors from deleted users). The agent will swallow short bursts but will investigate if the errors cross a specific reoccurrence threshold.
    • Resolved: Used when the impact of an error has stopped (e.g., the error was fixed elsewhere). Any new occurrence of a resolved issue will trigger a fresh investigation.
    • Open PR: The traditional response to fix the underlying code issue.

  8. How background jobs work in Superlog

    main

    Background jobs in Superlog are auto-discovered at worker boot via loadJobs() and scheduled using pg-boss.

    Key characteristics:

    • Isolation: Jobs run outside the worker tick loop, ensuring long-running tasks do not block telemetry ingestion, alerts, or agent runs.
    • Concurrency Control: Each job is assigned an exclusive queue, meaning at most one instance of a specific job can be queued or active at a time.
    • Scheduling: Jobs use a 5-field cron syntax (minute precision). Note that pg-boss checks schedules every 30 seconds.
    • Extensibility: The job directory acts as a build seam; you can overlay additional job files into this directory at image-build time without modifying the core repository.
    import type { JobDefinition } from "../jobs.js";

export const job: JobDefinition = {
  name: "my-thing.sync",
  schedule: "0 */6 * * *",
  expireInSeconds: 30 * 60,
  create: ({ db, clickhouse }) => async () => {
    await doTheWork(db, clickhouse);
  },
};

  9. Manage Error Issue states: silenced, under observation, and resolved

    main

    Error Issues (logs and spans) can exist in several states to manage noise and escalation. Note that Alert episodes cannot be silenced or put under observation; they are only open or resolved.

    • open: The default state. Triggers a new Incident.
    • silenced: The Issue is recognized as noise (e.g., a 404 error that is intended behavior). New occurrences will not trigger new Incidents.
    • under observation: Used for non-critical, one-off events. The Issue requires an escalation trigger (such as a specific rate of errors per minute or an absolute count). If the trigger trips, the Issue becomes open again and triggers a new Incident.
    • resolved: The issue has been addressed. New occurrences of the same error will trigger a new Incident, but the agent can consult previous findings from the prior investigation.

  10. How the Web design system and /design sheet work together

    main

    The web UI follows a strict design system consisting of a written contract (DESIGN.md) and a living reference (the /design route).

    • The /design sheet: Located at the /design route (served by src/design/DesignLanguage.tsx), this is a component catalog that renders every canonical primitive and token. It is used to catch 'drift'—if a change to a component or token isn't visible in the /design sheet, the change is considered incomplete.
    • The Written Contract: DESIGN.md defines the rules for components and tokens.

    Rule of thumb: If you change a primitive or a token, you must update both the component in src/design/ui.tsx (or its specific file) AND its corresponding panel in the /design sheet.

  11. How Sentry webhooks are processed

    main

    The Superlog Sentry webhook endpoint handles incoming events from Sentry with the following security and performance characteristics:

    1. Verification: The endpoint verifies the Sentry-Hook-Signature against the raw request body before processing any event.
    2. Idempotency: Issue events are persisted idempotently to prevent duplicate processing.
    3. Asynchronous Handling: Events are handled asynchronously, allowing the endpoint to acknowledge Sentry's request quickly to avoid timeouts.

  12. Avoid using monospace fonts with uppercase text

    main

    To maintain readability and avoid a 'terminal chrome' aesthetic, never combine a monospace font with uppercased text for labels, headings, or UI chrome (e.g., avoid using font-mono + uppercase + tracking-[…]).

    Instead, use a capitalized label in the sans-serif font (Inter). For small labels, use the following pattern:

    text-[13px] font-medium text-muted

    Monospace fonts should be reserved exclusively for genuine code, IDs, or tabular values in their natural case.

    // Recommended pattern for small labels:
// text-[13px] font-medium text-muted