cargo-tarpaulin

Tarpaulin is a code coverage reporting tool designed for the Cargo build system. It provides line coverage for Rust projects.

Key Characteristics:

• Tracing Backends: On Linux, the default is Ptrace (works on x86_64). On macOS and Windows, the default is the collection method used by those platforms. You can switch to LLVM coverage instrumentation using the --engine llvm flag.
• Docker Support: Tarpaulin can be run in Docker, which is useful for running coverage locally on non-Linux systems.
• Capabilities: Supports line coverage, HTML report generation, uploading to Coveralls or Codecov, and coverage for tests, doctests, benchmarks, and examples.

If tarpaulin fails with an error stating it cannot open libssl.so (e.g., libssl.so.1.1: cannot open shared object file), you can resolve this by installing tarpaulin with the vendored-openssl feature enabled.

cargo install --features vendored-openssl cargo-tarpaulin

To integrate Tarpaulin coverage into GitLab CI pipelines, add a regex to the coverage field in your job definition and configure artifacts to export a cobertura.xml report.

Regex for coverage detection:

coverage: '/^\d+.\d+% coverage/'

Artifact configuration for merge request diffs:

artifacts:
  reports:
    coverage_report:
      coverage_format: cobertura
      path: cobertura.xml

Note: You must generate the cobertura.xml using the appropriate output flag in Tarpaulin.

You can use the Python library pycobertura to diff or implement Cobertura reports generated by Tarpaulin.

1. Generate the report: cargo tarpaulin --out xml
2. Install pycobertura via pip: pip install pycobertura
3. Run pycobertura commands against the generated cobertura.xml.

To run Tarpaulin inside a Docker container without installing it locally, use the official image. You must use --security-opt seccomp=unconfined to allow the necessary syscalls for coverage collection.

Run latest version:

docker run --rm --security-opt seccomp=unconfined -v "${PWD}:/volume" xd009642/tarpaulin

Run latest development (nightly) version:

docker run --rm --security-opt seccomp=unconfined -v "${PWD}:/volume" xd009642/tarpaulin:develop-nightly

You can install Tarpaulin using cargo install. It is recommended to use the --locked flag to ensure dependency stability. Alternatively, you can use the Nix package manager or cargo-binstall.

Using Cargo:

cargo install --locked cargo-tarpaulin

Using cargo-binstall:

cargo binstall cargo-tarpaulin

Using Nix: Use the nixpkgs.cargo-tarpaulin package.

Tarpaulin is used as a Cargo subcommand. You can pass arguments directly to the underlying test executables after a -- separator to filter or skip specific tests.

Basic Usage:

cargo tarpaulin [OPTIONS] [-- <ARGS>...]

Tarpaulin can use a .tarpaulin.toml or tarpaulin.toml file to define multiple coverage configurations. The config file can be specified via the --config argument or the CARGO_TARPAULIN_CONFIG_FILE environment variable.

Configuration structure:

• Named sections (e.g., [feature_a_coverage]) define specific test runs. You can specify features, release, and args (arguments passed to the test binary after --).
• The [report] section is a reserved name used to configure output formats like Html or Xml, and integration with services like coveralls.
• Use run-types to specify which targets to include (e.g., --lib, --tests, --bins).

Example tarpaulin.toml:

[feature_a_coverage]
features = "feature_a"

[feature_a_and_b_coverage]
features = "feature_a feature_b"
release = true

[report]
coveralls = "coveralls_key"
out = ["Html", "Xml"]

Tarpaulin can be configured using a TOML file. It searches for configuration files in the following order:

1. The file specified by the CARGO_TARPAULIN_CONFIG_FILE environment variable.
2. tarpaulin.toml in the project root.
3. .tarpaulin.toml in the project root.

Multiple configuration tables can be defined in a single file, and they will be merged with command-line arguments.

The default engine on Linux is ptrace. If your tests use Unix signals or spawn many processes, you may encounter issues:

• Signal stealing: If tarpaulin steals signals and causes failures, use the --forward-signals flag.
• Process spawns: Use --follow-exec if you use many process spawns.
• General Ptrace issues: If signal handling or ptrace complexity causes issues, switch to the LLVM engine using --engine llvm.
• EPERM (Operation not permitted): If the personality syscall is blocked (common in Docker), either use --engine llvm or configure your container to allow the syscall (e.g., --seccomp=unconfined or setting personality to SCMP_ACT_ALLOW).