DOCSEARCHbyAlgolia
Submit Docs

cargo-llvm-cov

repository·main·Indexed 23 days ago

https://github.com/taiki-e/cargo-llvm-cov

A Cargo subcommand that simplifies using LLVM source-based code coverage for Rust projects. It provides precise line, region, and branch coverage data and supports various test runners including cargo test, nextest, and UI tests. It features a CLI compatible with standard Cargo commands and can generate reports in HTML, text, JSON, LCOV, and Cobertura formats, with additional support for including C/C++ FFI code coverage.

Tokens
6.3K
Snippets
9
Records
36
Agent score
73%

What's inside cargo-llvm-cov

  1. What is cargo-llvm-cov

    main

    cargo-llvm-cov is a Cargo subcommand that provides a wrapper around rustc -C instrument-coverage to facilitate easy use of LLVM source-based code coverage.

    Key features include:

    • Precise Data: Generates line, region, and (optionally via nightly) branch coverage.
    • Broad Support: Works with cargo test, cargo run, cargo nextest, proc-macros (including UI tests like trybuild and ui_test), and doc tests (requires nightly).
    • Performance: Fast execution by only instrumenting necessary crates without adding extra layers between rustc, cargo, and llvm-tools.
    • Compatibility: Provides a CLI compatible with standard Cargo commands.

  2. Get coverage for external tests and fuzzers

    main

    For binaries built outside of the standard cargo test flow (e.g., custom build scripts, make, or AFL.rs fuzzers), use cargo llvm-cov show-env to prepare the environment.

    For arbitrary binaries:

    1. source <(cargo llvm-cov show-env --sh) (or the PowerShell equivalent)
    2. cargo llvm-cov clean --workspace
    3. Build your binary (e.g., cargo build)
    4. Run your binary/tests.
    5. Generate report: cargo llvm-cov report --lcov

    For AFL.rs fuzzers:

    1. Set environment and clean: source <(cargo llvm-cov show-env --sh) and cargo llvm-cov clean --workspace.
    2. Build: cargo afl build.
    3. Run fuzzer: AFL_FUZZER_LOOPCOUNT=20 cargo afl fuzz -c - -i in -o out target/debug/fuzz-target.
    4. Report: cargo llvm-cov report --lcov (use --release if you built with --release).
    # External tests workflow
source <(cargo llvm-cov show-env --sh)
cargo llvm-cov clean --workspace
cargo build
cargo llvm-cov report --lcov

# AFL fuzzer workflow
source <(cargo llvm-cov show-env --sh)
cargo llvm-cov clean --workspace
cargo afl build
AFL_FUZZER_LOOPCOUNT=20 cargo afl fuzz -c - -i in -o out target/debug/fuzz-target
cargo llvm-cov report --lcov

  3. Exclude code and files from coverage

    main

    Exclude files via Regex

    Use --ignore-filename-regex <PATTERN> to skip files matching a regular expression. By default, cargo-llvm-cov ignores vendored sources, tests, examples, benches, and certain internal paths.

    Exclude code via Attributes

    You can use the unstable #[coverage(off)] attribute to exclude specific functions or modules. It is recommended to use this with cfg(coverage) or cfg(coverage_nightly) which are automatically set by cargo-llvm-cov.

    Example (Module level):

    #![cfg_attr(coverage_nightly, feature(coverage_attribute))]

#[cfg(test)]
#[cfg_attr(coverage_nightly, coverage(off))]
mod tests {
    // This module will be excluded from coverage
}

    To avoid warnings in Rust 1.80+, add the following to your Cargo.toml:

    [lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(coverage,coverage_nightly)'] }
    #![cfg_attr(coverage_nightly, feature(coverage_attribute))]

#[cfg_attr(coverage_nightly, coverage(off))]
fn exclude_fn_from_coverage() {
    // ...
}

#[cfg_attr(coverage_nightly, coverage(off))]
mod exclude_mod_from_coverage {
    // ...
}

  4. Install cargo-llvm-cov in GitHub Actions

    main

    Use the taiki-e/install-action to install prebuilt binaries on Linux, macOS, and Windows. This is faster than compiling from source.

    If you are using nextest, install both actions:

    # Standard installation
- uses: taiki-e/install-action@cargo-llvm-cov

# Installation with nextest
- uses: taiki-e/install-action@cargo-llvm-cov
- uses: taiki-e/install-action@nextest

  5. Install cargo-llvm-cov via package managers

    main

    Various package managers support cargo-llvm-cov:

    Homebrew (macOS/Linux):

    brew install cargo-llvm-cov
# Or from the maintained tap:
brew install taiki-e/tap/cargo-llvm-cov

    Scoop (Windows):

    scoop bucket add taiki-e https://github.com/taiki-e/scoop-bucket
scoop install cargo-llvm-cov

    Arch Linux:

    pacman -S cargo-llvm-cov

    FreeBSD:

    pkg install cargo-llvm-cov

    cargo-binstall:

    cargo binstall cargo-llvm-cov

  6. Merge coverages from different test conditions

    main

    You can combine coverage data from multiple runs (e.g., different features) by using --no-report during the test phase and then running the report subcommand.

    To ensure accuracy, it is recommended to run cargo llvm-cov clean --workspace before starting a new merge sequence to remove old artifacts.

    Workflow:

    1. Clean artifacts: cargo llvm-cov clean --workspace
    2. Run tests for condition A: cargo llvm-cov --no-report --features a
    3. Run tests for condition B: cargo llvm-cov --no-report --features b
    4. Generate final report: cargo llvm-cov report --lcov

    Note: To include doctests in the merged report, pass --doctests to the report command.

    cargo llvm-cov clean --workspace
cargo llvm-cov --no-report --features a
cargo llvm-cov --no-report --features b
cargo llvm-cov report --lcov

  7. Install cargo-llvm-cov from source

    main

    You can install cargo-llvm-cov directly using cargo install.

    Requirements:

    • rustc 1.87 or newer is required for installation.
    • Cargo 1.60 or newer is required to run the tool.

    Note that you can often run cargo-llvm-cov with a Cargo version older than the one required to perform the installation (e.g., cargo +1.60 llvm-cov).

    cargo +stable install cargo-llvm-cov --locked

  9. Basic usage of cargo-llvm-cov

    main

    By default, cargo llvm-cov runs tests via cargo test and prints a coverage summary to stdout.

    Common tasks:

    • Run binaries/examples: Use the run subcommand: cargo llvm-cov run.
    • HTML reports: Generate an HTML report in target/llvm-cov/html using --html or automatically open it in a browser with --open.
    • Text reports: Generate a plain text report with --text. If --output-path is not specified, it prints to stdout.
    • Export data: Use --json, --lcov, or --cobertura to export coverage data. If --output-path is not specified, it prints to stdout.

  10. Install cargo-llvm-cov via prebuilt binaries

    main

    Download prebuilt binaries from the GitHub Releases page. Supported platforms include Linux (various architectures), macOS, Windows, and FreeBSD.

    To install manually without verification, you can use a script to detect your host target and extract the binary to your $CARGO_HOME/bin (defaults to $HOME/.cargo/bin).

    # Get host target.
host=$(rustc -vV | grep '^host:' | cut -d' ' -f2)
# Download binary and install to $CARGO_HOME/bin (or $HOME/.cargo/bin if CARGO_HOME is unset).
curl --proto '=https' --tlsv1.2 -fsSL "https://github.com/taiki-e/cargo-llvm-cov/releases/latest/download/cargo-llvm-cov-$host.tar.gz" \
  | tar xzf - -C "${CARGO_HOME:-"$HOME/.cargo"}"/bin

  11. Install cargo-llvm-cov with verification

    main

    For security, you can verify the downloaded binary using GitHub release attestations and artifact attestations. This ensures the binary matches the expected SHA-256 digest and was produced by the official workflow.

    Steps:

    1. Download the binary.
    2. Use gh release verify-asset to check the release integrity.
    3. Use gh attestation verify to check the build provenance.
    # Get host target.
host=$(rustc -vV | grep '^host:' | cut -d' ' -f2)
# Download binary.
curl --proto '=https' --tlsv1.2 -fsSL -o cargo-llvm-cov.tar.gz "https://github.com/taiki-e/cargo-llvm-cov/releases/latest/download/cargo-llvm-cov-${host}.tar.gz"
# Verify release attestations.
gh release -R https://github.com/taiki-e/cargo-llvm-cov verify-asset cargo-llvm-cov.tar.gz
# Verify artifact attestations.
gh attestation verify --repo taiki-e/cargo-llvm-cov --signer-workflow taiki-e/github-actions/.github/workflows/rust-release.yml cargo-llvm-cov.tar.gz
# Install to $CARGO_HOME/bin (or $HOME/.cargo/bin if CARGO_HOME is unset).
tar xf cargo-llvm-cov.tar.gz -C "${CARGO_HOME:-"$HOME/.cargo"}"/bin
# Remove archive.
rm cargo-llvm-cov.tar.gz

  12. Get coverage of C/C++ code linked to Rust

    main

    To include coverage for FFI (C/C++) code, use the --include-ffi flag. You must set the following environment variables to Clang/LLVM tools compatible with the LLVM version used by your rustc:

    • CC
    • CXX
    • LLVM_COV
    • LLVM_PROFDATA

    Example:

    CC=<clang-path> \
CXX=<clang++-path> \
LLVM_COV=<llvm-cov-path> \
LLVM_PROFDATA=<llvm-profdata-path> \
  cargo llvm-cov --lcov --include-ffi