ExCoveralls Documentation

repository·master·Indexed 21 days ago

https://github.com/parroty/excoveralls

An Elixir library that reports test coverage statistics using Erlang's `cover` module. It provides tools to display coverage summaries locally, generate reports in HTML, JSON, XML, Cobertura, and LCOV formats, and post results to the Coveralls.io service via its JSON API. It includes dedicated integrations for Travis CI and GitHub Actions, as well as support for Espec and umbrella projects.

Tokens
7.6K
Snippets
34
Records
40
Agent score
71%

What's inside ExCoveralls

  1. Configure ExCoveralls in mix.exs

    master

    To use ExCoveralls, you must add it to your project's dependencies and configure the test_coverage and preferred_cli_env keys in your mix.exs file.

    Dependency Setup

    Add :excoveralls to your deps function, typically restricted to the :test environment:

    defp deps do
      [
        {:excoveralls, "~> 0.18", only: :test}
      ]
    end

    Configuration Options

    In your project function, you can use the following keys:

    • test_coverage: [tool: ExCoveralls]: Enables ExCoveralls for coverage reporting.
    • test_coverage: [tool: ExCoveralls, export: "cov"]: Enables reporting and exports data to cover/cov.coverdata.
    • test_coverage: [tool: ExCoveralls, test_task: "espec"]: Use this if you are using Espec instead of the default ExUnit.
    • preferred_cli_env: [coveralls: :test]: (Optional) Allows you to skip specifying MIX_ENV=test when running mix coveralls tasks by setting the default environment.
    • Application.put_env(:excoveralls, :base_path, "/path/to/root"): (Optional) Explicitly sets the application root path. By default, this is the directory containing mix.exs.
    def project do
      [
        app: :excoveralls,
        version: "1.0.0",
        elixir: "~> 1.0.0",
        deps: deps(),
        test_coverage: [tool: ExCoveralls],
        preferred_cli_env: [
          coveralls: :test,
          "coveralls.detail": :test,
          "coveralls.post": :test,
          "coveralls.html": :test,
          "coveralls.cobertura": :test
        ]
      ]
    end
    
    defp deps do
      [
        {:excoveralls, "~> 0.18", only: :test}
      ]
    end
  2. Post coverage from Travis CI

    master

    To submit coverage to Coveralls from Travis CI, use mix coveralls.travis.

    In your .travis.yml, set mix coveralls.travis as the build script and ensure MIX_ENV is set to test.

    If using Travis Pro for private projects, use mix coveralls.travis --pro and ensure your Coveralls repo token is available via the COVERALLS_REPO_TOKEN environment variable.

    language: elixir
    
    elixir:
      - 1.2.0
    
    otp_release:
      - 18.0
    
    env:
      - MIX_ENV=test
    
    script: mix coveralls.travis
  3. Merge coverage results from multiple test runs

    master

    You can combine coverage data from different test suites (e.g., unit tests and integration tests) using the --import-cover flag in mix coveralls.

    1. Generate coverage data for a specific suite using mix test --cover and specify an output name with --export-coverage.
    2. Report combined coverage using mix coveralls --import-cover <path_to_directory_containing_coverdata>.

    This is useful for partitioned tests or integration tests running in separate processes.

    # 1. Run integration tests and export coverage data
    $ mix test --only integration --cover --export-coverage integration-coverage
    
    # 2. Run coveralls and import the exported data from the 'cover' directory
    $ mix coveralls --exclude integration --import-cover cover
  4. Ignore specific lines in coverage calculation

    master

    You can manually exclude specific lines or blocks of code from coverage reports using comments in your Elixir source files.

    Block Ignore

    Wrap code in coveralls-ignore-start and coveralls-ignore-stop.

    Single Line Ignore

    Use coveralls-ignore-next-line immediately above the line you wish to ignore.

    # Block ignore
    # coveralls-ignore-start
    def ignored do
    end
    # coveralls-ignore-stop
    
    # Single line ignore
    def covered do
      # coveralls-ignore-next-line
      "ignored"
      "covered"
    end
  5. Configure ExCoveralls for Espec

    master

    If your project uses Espec instead of the default ExUnit for testing, you must specify the test_task in your mix.exs configuration so ExCoveralls knows which task to run to generate coverage data.

    Add test_task: "espec" to your test_coverage list.

    test_coverage: [tool: ExCoveralls, test_task: "espec"]
  6. Post coverage from GitHub Actions

    master

    To submit coverage from GitHub Actions, use mix coveralls.github.

    In your GitHub workflow YML file:

    1. Set MIX_ENV: test.
    2. Add GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} to the env section. This is required for Coveralls to check the action and add statuses.

    Example workflow step:

    - run: mix coveralls.github
    on: push
    
    jobs:
      test:
        runs-on: ubuntu-latest
        env:
          MIX_ENV: test
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        steps:
          - uses: actions/checkout@v1.0.0
          - uses: erlef/setup-beam@v1
          - run: mix deps.get
          - run: mix coveralls.github
  7. Configure Coverage Options in coveralls.json

    master

    The coverage_options object in coveralls.json allows fine-tuning of the coverage engine:

    • treat_no_relevant_lines_as_covered (boolean): If true, files with no relevant lines show 100% coverage instead of 0%.
    • output_dir (string): The directory for HTML reports. Defaults to cover/.
    • template_path (string): Custom path for HTML report templates.
    • minimum_coverage (number 0-100): If the coverage falls below this threshold, mix coveralls and mix coveralls.html will exit with status code 1 (useful for CI).
    • html_filter_full_covered (boolean): If true, files with 100% coverage are hidden from the HTML report.
    • floor_coverage (boolean): If false, coverage values are ceiled instead of floored.
    • xml_base_dir (string): Custom path for XML reports.
    {
      "coverage_options": {
        "treat_no_relevant_lines_as_covered": true,
        "output_dir": "cover/",
        "template_path": "custom/path/to/template/",
        "minimum_coverage": 90,
        "xml_base_dir": "custom/path/for/xml/reports/",
        "html_filter_full_covered": true
      }
    }
  8. Understand ExCoveralls.Stats.Source and Line structs

    master

    The ExCoveralls.Stats module uses two primary structs to represent coverage data:

    ExCoveralls.Stats.Source

    Represents the coverage state of an entire file.

    • filename: String path to the file.
    • coverage: Float representing the percentage of coverage.
    • sloc: Integer count of source lines of code.
    • hits: Integer count of lines hit.
    • misses: Integer count of lines missed.
    • source: A list of %ExCoveralls.Stats.Line{} structs.

    ExCoveralls.Stats.Line

    Represents the coverage state of a single line.

    • coverage: The coverage value for that line (e.g., hit count or nil).
    • source: The actual string content of the line.
  9. Configure ExCoveralls via coveralls.json

    master

    ExCoveralls uses a coveralls.json file for configuration. You can place a custom coveralls.json in your Mix project root to override the default settings located in deps/excoveralls/lib/conf. If a key is missing in your custom file, the default value is used.

    Stop Words

    Use default_stop_words or custom_stop_words to define regular expressions for words that should be excluded from coverage calculations (e.g., certain Elixir kernel macros).

    Exclude Files

    Use the skip_files key to provide an array of file paths (as Elixir regex strings) to ignore from coverage.

    Note for Umbrella Projects: skip_files in the umbrella root does not work for individual apps. To exclude files within a specific app, create a coveralls.json at the root of that specific app's folder. Paths should be relative to that app's folder.

    Terminal Report Output

    Customize how the report appears in your terminal using terminal_options:

    • file_column_width: Sets the column width for file names.
    • print_files: Set to false to show only the total coverage summary without the per-file table.
    {
      "skip_files": [
        "folder_to_skip",
        "folder/file_to_skip.ex"
      ],
      "terminal_options": {
        "file_column_width": 40,
        "print_files": false
      }
    }
  10. Configure HTTP options for ExCoveralls

    master

    ExCoveralls uses :httpc to post results. You can customize these options in your application configuration (e.g., config/config.exs) using the :http_options key under :excoveralls.

    config :excoveralls,
      http_options: [
        timeout: 10_000,
        ssl: [
          verify: :verify_peer,
          depth: 2,
          customize_hostname_check: [
            match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
          ],
          cacertfile: to_charlist(System.fetch_env!("TEST_COVERAGE_CACERTFILE"))
        ]
      ]
  11. Configure options for mix coveralls tasks

    master

    The following options are common across most coveralls mix tasks:

    • -o, --output-dir: Write coverage information to a specific directory.
    • -u, --umbrella: Show overall coverage for an umbrella project.
    • -v, --verbose: Show the JSON string used for posting.
    • --subdir: Add a prefix to file paths. Use this if your source files reside in a subfolder of the git repo (e.g., if your file is test.ex but its relative path in the repo is src/lib/test.ex, use src/lib).
    • --rootdir: Strip this path from file paths to resolve relative paths. This must be the exact path to your git repo's root on the CI environment.
    • --flagname: Set the job flag name shown in the Coveralls UI.
    • --import-cover: Directory from which to import .coverdata files to add to the report.

    To see the full list of options, run mix coveralls --help.

    Usage: mix coveralls <Options>
      Used to display coverage
    
      <Options>
        -h (--help)         Show helps for excoveralls mix tasks
    
        Common options across coveralls mix tasks
    
        -o (--output-dir)   Write coverage information to output dir.
        -u (--umbrella)     Show overall coverage for umbrella project.
        -v (--verbose)      Show json string for posting.
        --subdir            Git repo sub directory...
        --rootdir           This will be stripped from the file path...
        --flagname          Job flag name which will be shown in the Coveralls UI
        --import-cover      Directory from where '.coverdata' files should be imported...