DOCSEARCHbyAlgolia
Submit Docs

CodSpeed Documentation

website·Indexed 35 days ago

https://docs.codspeed.io

Continuous performance profiling and benchmarking platform. Features include support for Rust, Go, Java, Python, and Node.js (Vitest, Tinybench, Benchmark.js), flame graphs, and performance status checks in CI. Includes AI-powered tools such as the Model Context Protocol (MCP) server, Agent Skills, and the @codspeedbot Wizard for autonomous benchmark setup and optimization.

Tokens
60.9K
Snippets
280
Records
446
Agent score
56%

What's inside CodSpeed

  1. Overview of CodSpeed Repository Provider Integrations

    CodSpeed integrates with Git forges to embed performance data directly into the development workflow. This integration enables three primary capabilities:

    1. Performance Reports in Pull Requests: Automatically posts detailed benchmark results and comparisons with the base branch as PR comments.
    2. Automated Status Checks: Integrates with branch protection rules to block merges if performance regressions exceed defined tolerance thresholds.
    3. Centralized Performance Tracking: Links all performance data to specific commits, branches, and pull requests to create a historical audit trail of performance changes.

  2. Overview of the CodSpeed Divan compatibility layer

    codspeed-divan-compat is a drop-in replacement for the Divan benchmarking framework. It allows developers to maintain existing Divan benchmark code while enabling integration with CodSpeed performance measurement. Divan is the recommended benchmarking framework for Rust when using CodSpeed.

  3. Overview of CodSpeed Macro Runners on CircleCI

    Macro runners are dedicated bare-metal machines managed by CodSpeed, used for walltime measurements with low variance. Unlike standard CircleCI executors that share hosts, macro runners are dedicated to a single job for its entire duration. On CircleCI, these are implemented as self-hosted runner resource classes created and managed by CodSpeed within the user's CircleCI organization.

  4. Overview of pytest-codspeed

    pytest-codspeed is a pytest plugin used for measuring and tracking the performance of Python code. It supports multiple measurement instruments, including wall-clock time, CPU simulation, and memory heap allocations.

  5. Overview of @codspeed/playwright-plugin

    The @codspeed/playwright-plugin allows developers to drive an application through Playwright and report measured user flow timings to CodSpeed. It supports any flow that Playwright can drive. Currently, it only supports the walltime instrument; CPU Simulation is not available. As of the current version, it supports Electron apps as targets, with browser-based targets planned for the future.

  6. Overview of Rust benchmarking with CodSpeed

    CodSpeed integrates with Rust through a combination of a dedicated cargo subcommand (cargo-codspeed) and compatibility layers for popular benchmarking crates. It supports divan (recommended), criterion.rs, and libtest (bencher). The process involves both a build phase and a runtime phase to ensure accurate performance measurements.

  7. Overview of CodSpeed continuous performance testing

    CodSpeed is a continuous performance testing platform designed for software engineering teams to anticipate the performance impact of new deliveries before they reach production. Unlike traditional statistical benchmarking which can be unreliable in CI environments, CodSpeed uses specialized instruments to isolate 'noisy neighbors' and side effects, providing accurate, reproducible performance reports and detailed execution profiles directly within Pull Request comments and Merge checks.

  8. Overview of CodSpeed Java Integration

    CodSpeed integrates with Java projects via a fork of the Java Microbenchmark Harness (JMH). Developers write standard JMH benchmarks but use the CodSpeed JMH fork as a dependency to enable automatic collection and reporting of results in CI. Currently, only the walltime instrument is supported.

  9. Overview of CodSpeed Agent Skills

    Agent Skills are instruction sets that teach AI coding assistants how to perform specific tasks with CodSpeed, such as setting up benchmarks and optimizing performance. They are designed to work in conjunction with the CodSpeed MCP server; while the MCP server provides access to performance data, the skills provide the operational knowledge to act on that data. The CodSpeed MCP server must be configured before using Agent Skills.

  10. Overview of CodSpeed MCP Server

    The CodSpeed MCP server implements the Model Context Protocol (MCP) to provide AI-powered tools with direct access to performance data, including benchmark runs, comparisons, and flamegraphs. This enables AI assistants to investigate regressions and review performance changes within the editor or chat interface. The server is hosted at https://mcp.codspeed.io/mcp.

  11. Overview of CodSpeed Macro Runners

    CodSpeed Macro Runners are dedicated bare-metal machines managed by CodSpeed. They provide a stable, isolated environment that eliminates virtualization noise and shared resource interference, enabling precise walltime measurements with low variance.

    Macro runners should be used instead of CPU Simulation when measuring:

    • System calls and I/O operations
    • End-to-end performance (network, disk, system interactions)
    • Walltime benchmarks where actual execution time is critical
    • Integration tests requiring a complete system environment.

  12. Overview of CodSpeed Bencher compatibility layer

    codspeed-bencher-compat is a drop-in replacement for the bencher crate. It allows developers to maintain existing benchmark code written for the unstable libtest benchmarking framework while enabling integration with CodSpeed performance measurement. Note: For new projects, using Divan is recommended over Bencher due to better ergonomics and features.