DOCSEARCHbyAlgolia
Submit Docs

Omicron Documentation

repository·main·Indexed 19 days ago

https://github.com/oxidecomputer/omicron

Omicron is the Oxide Rack control plane, a large-scale Rust workspace managing distributed hardware resources. It includes services for orchestration (Nexus), resource management (Sled Agent), telemetry (Oximeter), and networking (DNS/Gateway). The documentation covers the Oximeter metric collection system, the Reconfigurator facility for dynamic software and hardware lifecycle management, and various developer utilities such as crdb-seed and clickhouse-schema-updater.

Tokens
166.8K
Snippets
590
Records
870
Agent score
66%

What's inside Omicron

  1. Overview of the Oximeter metric collection system

    main

    Oximeter is the metric collection system for the Oxide Rack, used to describe, generate, and store telemetry data. It consists of four primary components:

    • oximeter crate: The core library used for describing and generating metric data samples.
    • collector binary: A program run by the control plane that pulls metrics from various sources.
    • producer library: A library used by applications to produce metric data. It allows applications to register with the control plane and provides an HTTP endpoint for the oximeter collector to pull data.
    • db library: A library for interacting with the ClickHouse telemetry database.

  2. Overview of Wicket

    main

    Wicket is a Terminal User Interface (TUI) designed for technicians to perform critical rack operations at the technician port. It is built using crossterm and tui-rs (or ratatui).

    Core Responsibilities:

    • Rack Initialization
    • Boundary service setup
    • Disaster Recovery
    • Minimal rack updates and emergency updates

  3. Overview of Reconfigurator

    main

    Reconfigurator is a control plane facility designed for the dynamic reconfiguration of software running on Oxide racks. It enables automated and manual management of the rack's composition and software state, specifically for:

    • Hardware Lifecycle: Adding, removing, or replacing sleds (supporting both graceful and non-graceful operations).
    • Software Lifecycle: Upgrading Oxide-delivered software.
    • Service Scaling: Scaling control plane services up or down.
    • Fault Tolerance: Providing automated responses to hardware failures.

  4. Overview of illumos hardware APIs

    main

    The sled-hardware directory provides APIs for accessing information about the underlying hardware of an illumos system.

    Because some integrations require linking against native illumos libraries, the crate is architected with two distinct targets:

    • illumos: Contains the actual hardware integration logic.
    • non_illumos: Contains stub implementations. These stubs allow the rest of the Sled Agent to compile and provide code completion in non-illumos development environments (like macOS or Linux) without requiring the native libraries.

  5. Overview of illumos-utils commands

    main

    The illumos-utils package provides Rust wrappers around various commands that are specific to the illumos operating system.

    Currently, these wrappers function as interfaces for CLI-based invocations. Because they wrap CLI calls rather than implementing OS-specific syscalls directly in Rust, they can be compiled on non-illumos operating systems (though executing the wrapped commands will likely fail on those platforms). This design choice is intended to maintain compatibility with tools like rust-analyzer across different development environments.

  6. Overview of the Sled Agent

    main
    The Sled Agent is a per-sled component responsible for managing local hardware and instances. It provides both a "real" implementation for physical hardware and a "simulated" implementation for low-overhead testing of state machine management. Decision-making logic is shared between these two implementations where possible.

  8. Use the zone-bundle tool to manage debugging data

    main

    The zone-bundle tool (part of the omicron-sled-agent crate) manages tarballs of debugging data collected from Oxide zones. It packages system log files (current, rotated, and archived) and the output of various debugging commands (like netstat -an) into a compressed archive. These bundles can be created, listed, fetched, or removed via the sled-agent HTTP API.

    # No single runnable example for the whole tool, but it is a CLI utility.

  10. What is the Management Gateway Service (MGS)?

    main

    The Management Gateway Service (MGS) acts as the communication bridge between the control plane and entities located on the management network.

    Its primary responsibilities include:

    • Message Passing: Routing messages from the control plane to Service Processors (SPs) within sleds, switches, and power shelf controllers.
    • Alert Routing: Routing alert messages from management network entities back to the control plane.
    • Inventory Inspection: Facilitating the inspection of racks and their subcomponents.
    • Power Management: Managing power via Service Processors (SPs) and the lower-level Ignition subsystem.
    • Component Updates: Serving as the mechanism for updating hardware components.
    • Error Routing: Routing errors generated by SPs and the Ignition subsystem.

    For a deep dive into the architecture of this service, the management network, and Service Processors, refer to RFD 210.

  11. What is the Bootstore and how does it work?

    main

    The bootstore crate implements the Oxide trust quorum protocol. Its primary purpose is to provide the key share storage necessary to unlock a RackSecret via a trust quorum protocol, which in turn allows the rack to boot by providing input key material to the key-manager for disk decryption.

    Core Concepts

    • Schemes: The bootstore uses a Scheme abstraction to implement different versions of the trust quorum protocol. Each scheme is represented by a zero-sized struct (e.g., V0Scheme).
    • RackSecret: A secret used as input key material for the key-manager. It is split into key shares and distributed across sleds.
    • Key Shares: Individual pieces of the RackSecret stored on M.2 devices. To reconstruct the secret, K shares must be combined from K independent sleds (where K = N/2 + 1 of the initial members).
    • Node: A type used to replicate and store early networking configuration required to bring up the control plane, sitting at a layer above the LRTQ FSM.

  12. How Omicron's test infrastructure works

    main

    Omicron uses a consolidated repository to simplify integration testing. The test suite automates the setup and teardown of the service under test and all its dependencies.

    Key features include:

    • Isolation: Every Nexus integration test receives its own isolated copy of the entire stack (including databases) to prevent interference between tests.
    • Dynamic Port Allocation: Server sockets in the test suite bind to port 0, allowing the OS to pick free ports and preventing collisions.
    • Real Dependencies: The ControlPlaneTestContext in the Nexus test suite spins up real instances of CockroachDB, Clickhouse, Nexus, and DNS, rather than relying on manual setup or mocks for core services.