Pyroscope Rust SDK

repository·main·Indexed 42 days ago

https://github.com/grafana/pyroscope-rs

A profiling tool for Rust applications to monitor performance and identify bottlenecks. Version 2.1.1 provides a Pyroscope Agent for collecting profiling data, support for the pprof-rs and jemalloc backends, and tools for managing stack traces, symbols, and profiling reports.

Tokens
6.6K
Snippets
25
Records
30
Agent score
52%

What's inside Pyroscope

  1. Profile Rust applications with Pyroscope Profiler

    main
    Pyroscope Profiler is a tool designed to profile Rust applications. It allows developers to collect profiling data from their Rust code to identify performance bottlenecks and understand resource usage.
  2. Structure profiling data with Report and ReportBatch

    main

    Profiling data is organized into Reports and ReportBatches.

    Report

    A Report contains a collection of StackTraces and their associated hit counts (HashMap<StackTrace, usize>), along with shared Metadata.

    ReportBatch

    A ReportBatch groups reports by a specific profile_type (e.g., "process_cpu" or "memory"). The data field in a batch can be one of two types via the ReportData enum:

    • Reports(Vec<Report>): Structured stack-trace reports that require pprof encoding before transmission.
    • RawPprof(Vec<u8>): Pre-encoded pprof bytes (e.g., from a backend like jemalloc) that may already be gzipped.
  3. Manage backend lifecycle using BackendImpl

    main

    The BackendImpl<S> struct manages the lifecycle and thread-safe access to a Backend implementation using a state-machine pattern enforced by the Rust type system. This ensures that operations like report() can only be called when the backend is in the BackendReady state.

    Lifecycle Workflow

    1. Creation: Create a new instance using BackendImpl::new(Box::new(your_backend)). This returns a BackendImpl<BackendUninitialized>.
    2. Initialization: Call .initialize() on the uninitialized instance. This transitions the state to BackendImpl<BackendReady>.
    3. Usage:
      • Use .add_tag(tag) or .remove_tag(tag) on any BackendAccessible state (Uninitialized or Ready).
      • Use .report() only on the BackendReady state.
    4. Shutdown: Call .shutdown() on the ready instance to clean up and consume the backend.
    // Example lifecycle flow
    let backend_box = Box::new(MyCustomBackend::new());
    let uninitialized = BackendImpl::new(backend_box);
    
    // Transition to Ready
    let mut ready = uninitialized.initialize()?;
    
    // Perform operations
    ready.add_tag(my_tag)?;
    let report = ready.report()?;
    
    // Shutdown
    ready.shutdown()?;
  4. Understand the Frames and UnresolvedFrames data structures

    main

    The profiling data is structured into two main types:

    1. UnresolvedFrames: Represents a raw sample. It contains a SmallVec of frames (up to MAX_DEPTH), the thread_id, a sample_timestamp, and the thread_name stored as a fixed-size byte array.

    2. Frames: Represents a resolved sample. It contains a Vec<Vec<Symbol>> where each inner vector represents the symbols associated with a specific stack frame, a String for the thread_name, the thread_id, and the sample_timestamp.

  5. Manage profiling metadata with Tag and Metadata

    main

    Metadata in Pyroscope is used to attach searchable information to StackTrace or Report objects. Metadata is composed of a set of Tag objects.

    • Tag: A simple key-value pair (key=value).
    • Metadata: A collection of Tags. You can retrieve a unique identifier for metadata using get_id(), which returns a hash of the metadata content.

    To add information to a report or stack trace, create a Tag and use Metadata::add_tag().

    let mut metadata = Metadata::default();
    metadata.add_tag(Tag::new("service".to_string(), "api-gateway".to_string()));
    let id = metadata.get_id();
  6. Represent stack traces with StackTrace and StackFrame

    main

    A StackTrace represents a single execution path and consists of a sequence of StackFrames.

    StackFrame

    Each frame contains details about the code location:

    • module: The module name.
    • name: The function name.
    • filename: The filename.
    • relative_path / absolute_path: File paths.
    • line: The line number.

    StackTrace

    A StackTrace includes:

    • pid: Optional Process ID.
    • thread_id: Optional Thread ID.
    • thread_name: Optional Thread Name.
    • frames: A vector of StackFrames.
    • metadata: Metadata attached to the trace.

    When creating a StackTrace via StackTrace::new(), the inclusion of pid, thread_id, or thread_name in the metadata depends on the provided BackendConfig settings (report_pid, report_thread_id, and report_thread_name).

    let frame = StackFrame::new(
        Some("my_module".to_string()),
        Some("my_function".to_string()),
        Some("main.rs".to_string()),
        None,
        None,
        Some(42),
    );
    
    let stack_trace = StackTrace::new(
        &config, // BackendConfig
        Some(1234), // pid
        None, // thread_id
        Some("worker-thread".to_string()), // thread_name
        vec![frame],
    );
  7. How the PyroscopeAgent lifecycle works

    main

    The PyroscopeAgent uses a type-state pattern to ensure methods are only called when the agent is in a valid state. The lifecycle transitions are:

    1. PyroscopeAgentBare: The initial state before construction.
    2. PyroscopeAgentReady: Created after calling .build() on the builder. The agent is initialized but not yet profiling.
    3. PyroscopeAgentRunning: Created after calling .start() on a Ready agent. The agent is actively profiling and uploading data.

    Transitions:

    • Builder $\rightarrow$ .build() $\rightarrow$ Ready $\rightarrow$ .start() $\rightarrow$ Running $\rightarrow$ .stop() $\rightarrow$ Ready.
  8. Initialize and start a PyroscopeAgent

    main

    To use the profiler, use the PyroscopeAgentBuilder to construct an agent with a specific backend, then call .build() to initialize it, and .start() to begin profiling. The agent runs in a background thread and periodically uploads data based on the configured upload_interval.

    use pyroscope::pyroscope::PyroscopeAgentBuilder;
    use pyroscope::backend::{pprof_backend, PprofConfig, BackendConfig};
    
    fn main() -> Result<(), Box<dyn std::error::Error>> {
        let agent = PyroscopeAgentBuilder::new(
            "http://localhost:8080", 
            "my-app", 
            100, 
            "pyroscope-rs", 
            "0.1.0",
            pprof_backend(PprofConfig::default(), BackendConfig::default()),
        )
        .build()?;
    
        let agent_running = agent.start()?;
        
        // Agent is now profiling in the background...
        
        Ok(())
    }
  9. Configure and build a Pyroscope Agent

    main

    Use PyroscopeAgentBuilder to configure a new agent. You must provide the Pyroscope server URL, the application name, a sample rate, the agent name, the application version, and a backend configuration (such as pprof_backend).

    use pyroscope::pyroscope::PyroscopeAgentBuilder;
    use pyroscope::backend::{pprof_backend, PprofConfig, BackendConfig};
    
    // Example configuration
    let agent = PyroscopeAgentBuilder::new(
        "http://localhost:4040",
        "myapp",
        100, // sample rate
        "pyroscope-rs",
        env!("CARGO_PKG_VERSION"),
        pprof_backend(PprofConfig::default(), BackendConfig::default()),
    )
    .build()?;
  10. Add the Pyroscope Library and pprof-rs backend to Cargo.toml

    main

    To use the Pyroscope Rust integration with the pprof-rs backend, add the following dependency to your Cargo.toml file. Ensure you enable the backend-pprof-rs feature.

    [dependencies]
    pyroscope = { version = "2.0.0", features = ["backend-pprof-rs"] }
  11. Start, stop, and shutdown the Pyroscope Agent

    main

    Once an agent is built, you can control the profiling lifecycle using the following methods:

    1. Start profiling: Call .start()? on the agent to begin collecting data and sending it to the server. This returns a running agent handle.
    2. Stop profiling: Call .stop()? on the running agent handle to pause profiling. This returns a handle representing the stopped state.
    3. Shutdown: Call .shutdown() on the stopped agent handle to clean up resources. It is important to call this before the variable is dropped.
    // Start profiling
    let agent_running = agent.start()?;
    
    // Stop profiling
    let agent_ready = agent_running.stop()?;
    
    // Shutdown the agent
    agent_ready.shutdown();
  12. Configure Backend reporting options

    main

    The BackendConfig struct allows you to specify which metadata should be included in the profiling reports. Use these boolean flags to enable or disable specific identifiers:

    • report_thread_id: Whether to include the thread ID.
    • report_thread_name: Whether to include the thread name.
    • report_pid: Whether to include the process ID (PID).
    pub struct BackendConfig {
        pub report_thread_id: bool,
        pub report_thread_name: bool,
        pub report_pid: bool,
    }