diff-cover

repository·main·Indexed 21 days ago

https://github.com/bachmann1234/diff_cover

A tool for running coverage and linting reports on git diffs to identify lines requiring test coverage or containing quality violations. It includes the diff-cover CLI for analyzing coverage reports (XML or lcov.info) and the diff-quality CLI for analyzing code quality using drivers like pylint, flake8, ruff, and eslint. Version 10.4.1.

Tokens
10.6K
Snippets
37
Records
46
Agent score
74%

What's inside diff_cover

  1. Handle multi-line statements in coverage analysis

    main

    By default, diff-cover compares diff reports with coverage reports. Since coverage reports often list code statements rather than individual lines, changes within a multi-line statement might be missed.

    To ensure all changed lines are analyzed, use the --expand-coverage-report argument. This adds lines not present in the coverage report to the report itself, assigning them the same hit count as the previously reported line.

    Note: This argument is only available for XML coverage reports and assumes the coverage tool reports untested statements with 0 hits based on the opening line.

    diff-cover coverage.xml --expand-coverage-report
  2. Use diff-quality for quality coverage

    main

    The diff-quality tool allows you to see violations (from tools like pycodestyle, flake8, pylint, etc.) specifically on the lines changed in your diff.

    Supported tools include: pycodestyle, pyflakes, flake8, pylint, checkstyle, checkstylexml, ruff.check, and clang.

    diff-quality --violations=<tool>
  3. Compare a specific branch or diff file

    main

    By default, diff-cover compares the current branch to origin/main. You can change this behavior using:

    • --compare-branch=<branch>: To specify a different target branch.
    • --diff-file=<path>: To provide a file containing the output of git diff instead of using a branch name.
    # Compare to a specific branch
    diff-cover coverage.xml --compare-branch=origin/release
    
    # Use a git diff file
    git diff main..feature > diff.txt
    diff-cover coverage.xml --diff-file=diff.txt
  4. Get started with diff-cover

    main

    To use diff-cover, you must be in a git repository and have a coverage report in Cobertura, Clover, JaCoCo, or LCov XML format.

    1. Generate a coverage report (e.g., using pytest-cov with --cov-report=xml).
    2. Run diff-cover pointing to the XML file. By default, it compares your current branch to origin/main.
    # 1. Generate coverage report
    pytest --cov --cov-report=xml
    
    # 2. Run diff-cover
    diff-cover coverage.xml
  5. Set up development environment with poetry

    main

    This project uses poetry for dependency management and packaging. To set up a development environment:

    1. Install poetry: pip install poetry
    2. Install project dependencies: poetry install
    3. (Optional) Configure git to ignore large formatting commits in blame: git config blame.ignoreRevsFile .git-blame-revs
    pip install poetry
    poetry install
    git config blame.ignoreRevsFile .git-blame-revs
  6. Combine multiple XML coverage reports

    main

    If you have multiple XML reports from different test suites, you can combine them. A line is considered covered if it is covered in any of the provided XML reports.

    diff-cover coverage1.xml coverage2.xml
  7. Show covered and uncovered lines in diff-cover

    main

    By default, the console report only shows the percentage. Use these flags to see more detail:

    • --show-uncovered: Lists individual lines that lack coverage in the console output.
    • --html-report=<path> --show-covered: When generating an HTML report, this highlights covered diff lines in green alongside the existing red highlighting for missing lines.
    diff-cover coverage.xml --show-uncovered
    diff-cover coverage.xml --html-report report.html --show-covered
  8. Install diff-cover

    main

    You can install the latest release of diff-cover using pip. For development, you can clone the repository and use poetry to install dependencies.

    # Install latest release
    pip install diff-cover
    
    # Install development version
    git clone https://github.com/Bachmann1234/diff-cover.git
    cd diff-cover
    poetry install
    poetry shell
  9. Pass pre-generated quality reports to diff-quality

    main

    To improve efficiency, you can run a quality tool manually and pass the resulting report file to diff-quality instead of letting it re-run the tool.

    # For pylint < 1.0
    pylint -f parseable > pylint_report.txt
    
    # For pylint >= 1.0
    pylint --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" > pylint_report.txt
    
    # Use the report with diff-quality
    diff-quality --violations=pylint pylint_report.txt
    
    # For pycodestyle
    pycodestyle > pycodestyle_report.txt
    diff-quality --violations=pycodestyle pycodestyle_report.txt
  10. Add support for a new quality checker in diff-quality

    main

    You can extend diff-quality by creating a plugin using the pluggy package.

    1. Define the Entry Point

    In your plugin's setup.py, define a diff_cover entry point. The key must be diff_cover and the value must follow the format TOOL_NAME = YOUR_PACKAGE.PLUGIN_MODULE.

    setup(
        ...
        entry_points={
            'diff_cover': [
                'sqlfluff = sqlfluff.diff_quality_plugin'
            ],
        },
        ...
    )

    2. Implement the Plugin

    Your module must contain a function named diff_cover_report_quality decorated with @diff_cover_hookimpl. This function must return an object (typically a subclass of BaseViolationReporter) that implements:

    • supported_extensions: A list of file extensions (e.g., ['sql']).
    • violations(src_path): Returns a list of Violation objects.
    • measured_lines(src_path): (Optional) Returns line information.
    • installed(): A static method returning True.

    3. Usage

    Once installed, you can run the tool using the name defined in your entry point:

    diff-quality --violations sqlfluff
    from diff_cover.hook import hookimpl as diff_cover_hookimpl
    from diff_cover.violationsreporters.base import BaseViolationReporter, Violation
    
    class SQLFluffViolationReporter(BaseViolationReporter):
        supported_extensions = ['sql']
    
        def __init__(self):
            super(SQLFluffViolationReporter, self).__init__('sqlfluff')
    
        def violations(self, src_path):
            return [
                Violation(violation.line_number, violation.description)
                for violation in get_linter().get_violations(src_path)
            ]
    
        def measured_lines(self, src_path):
            return None
    
        @staticmethod
        def installed():
            return True
    
    
    @diff_cover_hookimpl
    def diff_cover_report_quality():
        return SQLFluffViolationReporter()
  11. Filter diff results with include and exclude patterns

    main

    Both BaseDiffReporter and GitDiffReporter support filtering which files are processed using include and exclude patterns.

    • include: If provided, a path is only considered if it matches one of the glob patterns. The matching is performed by expanding the patterns using glob.glob(pattern, recursive=True) and checking if the path exists in the resulting list.
    • exclude: If provided, a path is excluded if its basename matches any of the patterns (via fnmatch) OR if its absolute path matches any of the patterns.

    Note: If include patterns are provided, they act as a whitelist. If a path does not match any include pattern, it is automatically excluded before exclude patterns are even checked.

  12. Configure diff-cover and diff-quality with TOML

    main

    Both diff-cover and diff-quality support configuration via TOML files using the --config-file or -c flag.

    Requirements:

    • You must install diff-cover with the toml extra: pip install diff-cover[toml].
    • Files must end in .toml.
    • Only non-mandatory options are supported.
    • If an option is provided in both the config file and the command line, the command line value takes precedence.

    Formatting Rules:

    • Option names must use underscores (_) instead of dashes (-).
    • For options that can be specified multiple times, use a list in the TOML file.
    • Use the [tool.diff_cover] or [tool.diff_quality] table headers.
    diff-cover coverage.xml --config-file myconfig.toml
    diff-quality --violations=pycodestyle --config-file myconfig.toml
    [tool.diff_cover]
    compare_branch = "origin/feature"
    quiet = true
    
    [tool.diff_quality]
    compare_branch = "origin/feature"
    ignore_staged = true