undercover

repository·master·Indexed 21 days ago

https://github.com/grodowski/undercover

A tool that warns developers about methods, classes, and blocks changed in a git diff that lack test coverage. It integrates git diff analysis with SimpleCov or LCOV coverage reports to provide actionable feedback during development and CI/CD. It features a CLI for flagging untested changes, support for JSON output for machine-readability, and a configuration file (.undercover) to manage options like comparison refs and file filters.

Tokens
6.3K
Snippets
25
Records
33
Agent score
70%

What's inside undercover

  1. How to ignore/skip coverage for specific code blocks

    master

    If you want to acknowledge an untested change without fixing it immediately, or if you want to skip certain blocks of code entirely, use the :nocov: syntax supported by SimpleCov.

    Wrap the code you wish to ignore with # :nocov: and # :nocov::

    # :nocov:
    def skip_this_method
        never_reached
    end
    # :nocov:
  2. Set up coverage reporting with SimpleCov

    master

    To make your tests compatible with undercover, you must use the undercover/simplecov_formatter. This allows undercover to analyze your code changes against your test coverage.

    1. Add undercover to your :test group in the Gemfile.
    2. Configure spec_helper.rb (or your test helper) to use SimpleCov::Formatter::Undercover.
    3. Important: Run your test suite at least once to generate the initial coverage/coverage.json file before running undercover.

    To enable branch-level warnings, ensure you call enable_coverage(:branch) in your SimpleCov configuration.

    # Gemfile
    group :test do
      gem 'undercover'
    end
    
    # the very top of spec_helper.rb
    require 'simplecov'
    require 'undercover/simplecov_formatter'
    
    # SimpleCov::Formatter::Undercover.output_filename = 'my_project_coverage.json'
    SimpleCov.formatter = SimpleCov::Formatter::Undercover
    
    SimpleCov.start do
      add_filter reality_check_regex # e.g., /^\/spec\//
      enable_coverage(:branch)
    end
  3. Install the undercover gem

    master

    You can install undercover by adding it to your application's Gemfile and running bundle, or by installing it directly via the command line.

    To add to a Gemfile:

    gem 'undercover'

    Then run:

    $ bundle

    Or install it globally:

    $ gem install undercover
  4. Use the undercover CLI

    master

    The undercover CLI flags untested methods, classes, and blocks that were changed in the current git diff without corresponding tests.

    • Run on current diff: Run undercover with no arguments to check changes in your current working directory.
    • Run in CI/CD (Recommended): Use the --compare flag to specify a git ref (branch, commit, or tag) to compare against. This ensures undercover exits with code 1 if untested changes are found, which is ideal for build pipelines.

    Example for CI:

    undercover --compare origin/master
  5. How Undercover filters files

    master

    Undercover uses a FilterSet to determine which files should be included in coverage analysis. The filtering logic follows a specific hierarchy:

    1. SimpleCov Filters: If a file is ignored by your existing SimpleCov configuration, Undercover will ignore it automatically. SimpleCov filters are supported in three formats:
      • :string: A substring match against the normalized filepath.
      • :regex: A regular expression match against the normalized filepath.
      • :file: An exact match against the filepath.
    2. Undercover Allow Filters: The file must match at least one pattern in the allow_filters list.
    3. Undercover Reject Filters: The file must NOT match any pattern in the reject_filters list.

    For a file to be included, it must pass the SimpleCov check AND match an allow filter AND not match any reject filter.

  6. Configure undercover via a .undercover file

    master

    You can create a .undercover configuration file at the root of your project to store CLI options. This avoids repeating flags in your CI configuration.

    Example .undercover file:

    -l path/to/different.lcov
    -c origin/master

    Note: Options in this file can be overridden by passing arguments directly to the undercover command.

  7. Get machine-readable JSON output

    master

    To use undercover with custom tooling or CI integrations, use the --format json flag. The output contains a warnings array and a summary object.

    Each warning includes details about the node (method/class), file location, coverage percentage, and specific uncovered_lines or uncovered_branches.

    undercover --compare origin/main --format json
  8. Configure the output filename for Undercover coverage reports

    master

    You can customize the name of the generated coverage JSON file by setting SimpleCov::Formatter::Undercover.output_filename. If this is not set, the gem defaults to coverage.json. The file will be saved within your configured SimpleCov.coverage_path.

    SimpleCov::Formatter::Undercover.output_filename = 'my_project_coverage.json'
  9. Reference: undercover CLI options

    master

    The following options are available when running the undercover command from the terminal:

    FlagLong FlagDescription
    -s--simplecov pathSimpleCov JSON report file
    -l--lcov pathLCOV report file path (to be deprecated)
    -p--path pathProject directory
    -g--git-dir dirOverride .git with a custom directory
    -c--compare refGenerate coverage warnings for all changes after ref
    -r--ruby-syntax verRuby syntax version (e.g., ruby31, ruby32)
    -w--max-warnings limitMax warnings to generate before stopping (performance optimization)
    -f--include-files globsComma-separated glob patterns to include (default: '*.rb,*.rake,*.ru,Rakefile')
    -x--exclude-files globsComma-separated glob patterns to skip
    --format FORMATOutput format: text (default) or json
    -h--helpPrints help
    --versionShow version
    Usage: undercover [options]
        -s, --simplecov path             SimpleCov JSON report file
        -l, --lcov path                  LCOV report file path (to be deprecated)
        -p, --path path                  Project directory
        -g, --git-dir dir                Override `.git` with a custom directory
        -c, --compare ref                Generate coverage warnings for all changes after `ref`
        -r, --ruby-syntax ver            Ruby syntax version, one of: current, ruby18, ruby19, ruby20, ruby21, ruby22, ruby23, ruby24, ruby25, ruby26, ruby30, ruby31, ruby32, ruby33
        -w, --max-warnings limit         Maximum number of warnings to generate before stopping analysis. Useful as a performance improvement for large diffs.
        -f, --include-files globs        Include files matching specified glob patterns (comma separated). Defaults to '*.rb,*.rake,*.ru,Rakefile'
        -x, --exclude-files globs        Skip files matching specified glob patterns (comma separated). Empty by default.
        --format FORMAT              Output format: text, json (default: text)
        -h, --help                       Prints this help
        --version                        Show version
  10. Determine the exit code from JSON formatting

    master

    When using the Undercover::JsonFormatter, the exit_code method determines the process exit status based on the presence of validation errors and results:

    • Returns 0 if a @validation_error is present.
    • Returns 0 if there are no @results (no warnings found).
    • Returns 1 if there are results and no validation error (indicating warnings were found that need attention).
  11. Use UndercoverSimplecovFormatter with SimpleCov

    master

    To generate coverage reports compatible with Undercover, add SimpleCov::Formatter::Undercover to your SimpleCov.formatters configuration. This formatter extends the standard JSON formatter to include metadata such as the project root and ignored files (based on your SimpleCov filters).

    require 'undercover'
    
    SimpleCov.formatters = [
      SimpleCov::Formatter::Undercover
    ]