DOCSEARCHbyAlgolia
Submit Docs

HyperDX Documentation

repository·main·Indexed 27 days ago

https://github.com/hyperdxio/hyperdx

An observability platform for searching and visualizing logs, traces, metrics, and session replays on ClickHouse. Includes documentation for the @hyperdx/cli for source map uploads and terminal charting, the @hyperdx/api OpAMP HTTP server for OpenTelemetry collector management, and the @hyperdx/hdx-eval framework for benchmarking AI agents using MCP servers.

Tokens
39.5K
Snippets
81
Records
201
Agent score
95%

What's inside HyperDX

  1. Overview of Agent Documentation Directory

    main

    The agent_docs/ directory contains task-specific documentation designed for AI coding agents. It follows a progressive disclosure pattern, meaning documentation is segmented into specialized files to maintain a focused context window and improve instruction following. Instead of a single monolithic file, agents are expected to read only the relevant documentation for their current task.

    Key documentation areas include:

    • Architecture & Tech Stack: System models, service relationships, and library usage.
    • Development & Style: Workflows, testing strategies, and code patterns.
    • UI & Design: Page layouts, data visualization colors, and theme systems (HyperDX vs ClickStack).
    • Evaluations: The MCP eval framework for benchmarking and A/B comparisons.

  2. Overview of the MCP Eval Framework

    main

    The MCP Eval Framework is used to compare different MCP (Model Context Protocol) servers by spawning Claude Code as an SRE agent. The agent attempts to solve observability scenarios (e.g., finding root causes or building dashboards) against synthetic telemetry in ClickHouse.

    Results are graded using a combination of programmatic regex checks (40%) and an LLM judge (60%), with penalties applied for tool-call errors. The framework is MCP-agnostic, allowing side-by-side comparison of any two or more MCP servers.

  4. Overview of HyperDX OTel Collector

    main
    The HyperDX OTel Collector is a custom-built OpenTelemetry Collector compiled using the OpenTelemetry Collector Builder (OCB). It is designed to replace the standard otel/opentelemetry-collector-contrib image with a specialized binary that includes all components required by HyperDX, common core/contrib components, and custom HyperDX receivers and processors.

  5. Backend Tech Stack and Architecture

    main

    The HyperDX backend (packages/api) is a Node.js 22+ environment using TypeScript. It follows this architectural stack:

    • Framework: Express.js
    • Databases:
      • Telemetry Data: ClickHouse
      • Metadata: MongoDB (managed via Mongoose)
    • Authentication: Passport.js (using local strategy)
    • Validation: Zod schemas
    • Telemetry: The service is self-instrumented using @hyperdx/node-opentelemetry.

  6. Common Utilities Overview

    main

    The @hyperdx/common-utils package provides shared logic used across the repository, specifically focusing on:

    • Querying: TypeScript utilities for query parsing and ClickHouse operations.
    • Validation: Shared Zod schemas for data validation.
    • SQL: Helpers for SQL formatting and query building.

  7. Frontend Tech Stack and UI Development Guidelines

    main

    The HyperDX frontend (packages/app) is built using Next.js 14 and TypeScript. When developing or extending the UI, follow these technology choices and component priorities:

    Core Technologies

    • Framework: Next.js 14
    • UI Library: Mantine UI (@mantine/core, @mantine/dates, @mantine/hooks)
    • State Management:
      • Global Client State: Jotai
      • Server State: TanStack Query
      • Filters: URL parameters
    • Visualization: Recharts and uPlot
    • Code Editor: CodeMirror (used for SQL and JSON editing)
    • Icons: @tabler/icons-react
    • Styling: Mantine's built-in system or SCSS modules

    UI Component Priority

    To maintain consistency, follow this hierarchy when building components:

    1. Use Mantine components first.
    2. Build Custom components on top of Mantine primitives.
    3. Use Custom SCSS modules only as a last resort.

  9. Understand HyperDX Core Architecture

    main

    HyperDX is composed of several core services that manage the lifecycle of telemetry data:

    • HyperDX UI (packages/app): A Next.js frontend for user interaction.
    • HyperDX API (packages/api): A Node.js/Express backend for business logic and queries.
    • OpenTelemetry Collector: Receives and processes incoming telemetry.
    • ClickHouse: The primary data store for logs, metrics, and traces.
    • MongoDB: Stores metadata such as users, dashboards, alerts, and saved searches.

    Data Flow:

    1. Applications send telemetry via OpenTelemetry to the OTel Collector.
    2. The OTel Collector processes and forwards data to ClickHouse.
    3. Users interact with the UI, which queries ClickHouse via the API.
    4. Configuration and metadata are persisted in MongoDB.

  10. Use termchart renderers for ANSI terminal charts

    main
    The termchart module provides pure ANSI terminal chart renderers. Each renderer is a pure function that takes shaped data and target dimensions as input and returns a string containing ANSI escape codes. These functions have no dependencies on Ink, React, or the terminal process, making them suitable for both interactive TUIs and non-interactive stdout commands. The only runtime dependency is chalk.

  11. Understand the HyperDX theming system

    main

    HyperDX uses two independent theming concepts that work together via CSS variables:

    1. Brand Theme: Controls branding elements like logos, favicons, accent colors, and fonts. It is determined at deployment via the NEXT_PUBLIC_THEME environment variable and applied as a class to the <html> element (e.g., theme-hyperdx or theme-clickstack).
    2. Color Mode: Controls light/dark appearance (backgrounds, text, borders). It is controlled by the end user and persisted in localStorage under the key hdx-user-preferences using the colorMode field (values: light, dark, or system).

    Critical Rules for Developers:

    • Never hard-code hex or named colors. Always use semantic CSS variables (e.g., var(--color-bg-surface)).
    • Never branch on themeName to choose colors. The CSS variables automatically resolve to the correct color based on the active brand and mode.
    • Do not confuse IS_CLICKHOUSE_BUILD with the theme. This flag controls features, not branding.
    • For chart-specific colors, refer to the data_viz_colors.md guide.

  12. Overview of Data Visualization Color Systems

    main

    HyperDX uses three distinct color systems for data visualization. When adding or changing colors in charts, sparklines, heatmaps, legends, or status pills, you must use the appropriate system to ensure theme switching (light/dark mode) works correctly.

    Color Systems Summary

    SystemUse CaseSource of TruthConsumption Method
    Categorical (10 hues)Multi-series line, bar, area, or pie chartsCATEGORICAL_HEX_BY_TOKEN in utils.tsgetColorProps(index, label) (positional) or getColorFromCSSToken('chart-{hue}') (by name)
    SemanticStatus indicators, log levels, deltasCSS vars --color-chart-{success,warning,error,info}getChartColor{Success,Warning,Error,Info}()
    Heatmap continuousDBHeatmapChart density gradientsdarkPalette/lightPalette arraysImported directly from DBHeatmapChart.tsx

    Critical Rules

    • Do not pass hex colors directly to chart series. Always use the provided helpers to support theme switching.
    • Do not map log levels to raw Mantine colors (e.g., red.5, yellow.6). Instead, use logLevelColor() or getColorProps() to ensure the theme-correct semantic chart color is used.
    • Do not mix palettes: The categorical palette and the heatmap palette are distinct. Do not use --color-chart-{hue} for heatmap density, and do not use heatmap arrays for series colors.