DOCSEARCHbyAlgolia
Submit Docs

pytest-cov Documentation

repository·master·Indexed 24 days ago

https://github.com/pytest-dev/pytest-cov

A pytest plugin for measuring code coverage, version 7.1.0. It provides CLI options for source filtering, branch coverage, and failure thresholds, and supports multiple report formats including HTML, XML, JSON, and LCOV. The plugin integrates with pytest-xdist for distributed testing and offers features like test-specific coverage contexts via --cov-context and the ability to disable coverage for specific tests using the no_cover marker or fixture.

Tokens
5.2K
Snippets
15
Records
43
Agent score
84%

What's inside pytest-cov

  1. Important behaviors and overrides in pytest-cov

    master

    pytest-cov overrides certain standard coverage.py settings. Users should be aware of the following interactions:

    1. Parallelism: pytest-cov overrides the parallel option of coverage. Setting these in .coveragerc is generally ineffective unless running coverage without pytest-cov.
    2. Source Filtering: Using --cov=something overrides the source option in your coverage configuration file.
    3. Branch Coverage: Using the --cov-branch CLI flag overrides the branch option in your coverage configuration file.

  2. Requirements for distributed testing with pytest-xdist

    master
    When using distributed testing (e.g., via pytest-xdist), all worker nodes must have the pytest-cov package installed. This is required because the plugin must be registered through setuptools so that pytest can automatically start the plugin on each worker.

  3. Configure terminal report variations

    master

    You can customize the terminal output using specific flags or modifiers:

    • Default terminal report: pytest --cov=myproj tests/
    • Terminal report with missing line numbers: Use term-missing.
    • Terminal report skipping fully covered files: Use term:skip-covered.
    • Combined terminal report: You can combine modifiers using a colon, such as term-missing:skip-covered.
    pytest --cov-report=term-missing --cov=myproj tests/

pytest --cov-report term:skip-covered --cov=myproj tests/

pytest --cov-report term-missing:skip-covered --cov=myproj tests/

  4. Migrate subprocess support to coverage configuration

    master

    Subprocess support was removed from pytest-cov in version 7.0. To maintain subprocess coverage, you must now configure coverage directly using the patch setting.

    When you enable the subprocess patch, parallel = true is automatically enabled by coverage.

    ### For .ini files:

```ini
[run]
patch = subprocess

    For pyproject.toml:

    [tool.coverage.run]
patch = ["subprocess"]

  5. Configure pytest-cov with tox for parallel runs

    master

    For parallel execution in tox, you must use --cov-append to ensure data from different environments is merged. You should also define a dedicated report environment that depends on your test environments to generate the final coverage summaries (e.g., terminal and HTML reports).

    Example tox.ini structure:

    [tox]
envlist = clean,py27,py36,report

[testenv]
commands = pytest --cov --cov-append --cov-report=term-missing
deps =
    pytest
    pytest-cov
depends =
    {py27,py36}: clean
    report: py27,py36

[testenv:report]
deps = coverage
skip_install = true
commands =
    coverage report
    coverage html

[testenv:clean]
deps = coverage
skip_install = true
commands = coverage erase

  6. Release process for pytest-cov

    master

    To release a new version of pytest-cov, follow these steps in order:

    1. Verify Documentation: Run tox -e docs to ensure documentation builds and renders correctly. If you encounter spelling errors, add the words to spelling_wordlist.txt.
    2. Update Metadata: Update CHANGELOG.rst and AUTHORS.rst.
    3. Bump Version: Use bumpversion [ major | minor | patch ] to increment the version and create a git tag.
    4. Push Changes: Push your commits and tags to the remote repository:
      git push
git push --tags
    5. Verify CI: Wait for GitHub Actions to complete successfully.
    6. Verify Docs: Ensure the documentation on ReadTheDocs is built.
    7. Clean Environment: Verify you have a clean checkout with git status. Manually remove temporary files to force distutils/setuptools to rebuild metadata:
      rm -rf dist build src/*.egg-info
    8. Build Distributions: Create the source and wheel distributions:
      python -m build
    9. Verify Archives: Check the contents of the dist/ directory.
    10. Upload to PyPI: Use twine to upload the artifacts:
      twine upload dist/*

  8. Configure pytest-cov with tox for sequential runs

    master

    When running multiple environments in tox, pytest-cov will erase previous coverage data by default, leading to incomplete results. To support sequential runs, use the --cov-append flag in your pytest commands and include a clean environment to erase old data before starting.

    Example tox.ini structure:

    [tox]
envlist = clean,py27,py36,...

[testenv]
commands = pytest --cov --cov-append --cov-report=term-missing ...
deps =
    pytest
    pytest-cov

[testenv:clean]
deps = coverage
skip_install = true
commands = coverage erase

  9. Use "load" mode for distributed testing with xdist

    master

    When using pytest-xdist with the --dist load mode, pytest-cov reports on the combined coverage of all workers. This mode is suitable when workers are spread across different hosts or different file system locations. Each worker's subprocesses are measured and aggregated into a single terminal report.

    pytest --cov=myproj -n 2 tests/

  10. Configure pytest-cov with tox.ini for different project layouts

    master

    When using tox to run tests, you can configure pytest-cov via tox.ini to handle different project structures.

    There are two primary layouts to consider:

    1. src layout: This is the recommended and less complicated configuration. It keeps source code in a src/ directory, reducing the risk of mixing installed code with local source code.
    2. adhoc layout: This is an older layout where source code and tests might live in the root. It is more prone to errors where installed code is confused with local source code, but pytest-cov can still be configured to work correctly with it.

    Common configuration goals when using tox include:

    • Aggregating coverage data from multiple interpreters.
    • Supporting tox parallel mode.
    • Ensuring tests run against the installed version of the code rather than the local source.

  11. Use "each" mode for distributed testing with xdist

    master
    When using pytest-xdist with the --dist each mode, each worker runs all tests. pytest-cov reports on the combined coverage of all workers, which allows for generating a combined coverage report across multiple different environments (e.g., different Python versions or platforms) in a single view.

  12. Disable coverage using the --no-cov flag

    master

    If you are using an IDE debugger (like PyCharm) and coverage is interfering with your ability to hit breakpoints, you can disable coverage completely by adding the --no-cov flag to your pytest command.

    Note that if coverage is configured in your pytest.ini file, pytest does not currently support removeopts to automatically strip this flag, so it must be passed manually during the command invocation. When successful, you will see the following message in your output:

    Coverage disabled via --no-cov switch!