DOCSEARCHbyAlgolia
Submit Docs

Test Results Reporter

website·Indexed 29 days ago

https://test-results-reporter.github.io/

A library for publishing automation test results to communication platforms such as Slack, Microsoft Teams, and Google Chat. It supports various testing frameworks like Cucumber and Mocha, integrates with CI/CD tools including GitHub Actions and Azure Pipelines, and provides extensions for ReportPortal, Percy, and QuickChart.

Tokens
22.1K
Snippets
114
Records
144
Agent score
69%

What's inside Test Results Reporter

  1. Overview of Test Results Reporter

    Test Results Reporter is a library used as an alerting tool to forward automation test results from CI/CD environments to communication platforms. It provides immediate visibility into test success or failure to facilitate quicker issue resolution.

  2. Integrate ReportPortal analysis into test reports

    The report-portal-analysis extension allows you to include ReportPortal failure analysis (associating failures with product bugs, automation issues, or system issues) in notifications sent to Microsoft Teams or Slack. By default, this extension triggers on the end hook when the test condition is fail.

  3. Add a radial gauge test summary using QuickChart

    The quick-chart-test-summary extension generates a radial gauge image representing the total pass percentage of test results. It is designed for use with Teams and Slack targets. Note that this extension is not supported for the Google Chat target.

    Default Behavior:

    • Hook: post-main
    • Condition: passOrFail

  4. Integrate QuickChart test summary radial gauge into reports

    The quick-chart-test-summary extension generates a radial gauge image representing the total pass percentage of tests and attaches it to the report.

    Defaults:

    • Hook: post-main
    • Condition: passOrFail

    Limitation: This extension is not supported for the Google Chat target.

  5. Use Mocha with the JSON reporter for test results

    To use the default Mocha JSON reporter, run tests with the --reporter json flag. In the tool's configuration file, specify the result type as mocha and provide the path to the generated JSON result file.
    mocha tests --reporter json
    {
  "targets": [
    {
      "name": "slack",
      "inputs": {
        "url": "<incoming-webhook-url>"
      }
    }
  ],
  "results": [
    {
      "type": "mocha",
      "files": ["path/to/result.json"]
    }
  ]
}

  6. Publish Mocha test results using the JSON reporter

    To use the default Mocha JSON reporter, run your tests with the --reporter json flag. In the tool's configuration file, set the result type to mocha and provide the path to the generated JSON result file.
    mocha tests --reporter json
    {
  "targets": [
    {
      "name": "slack",
      "inputs": {
        "url": "<incoming-webhook-url>"
      }
    }
  ],
  "results": [
    {
      "type": "mocha",
      "files": ["path/to/result.json"]
    }
  ]
}

  7. Set up a TestBeats account and API key

    To begin using TestBeats, sign up with a work email at app.testbeats.com. After signing up, create a new organization to generate the API key required for reporting. Note that other individuals within the same organization sharing the same email domain will automatically have access to the test results.

  8. Integrate Percy Analysis with a Teams report target

    This example demonstrates how to add the percy-analysis extension to a Teams notification target. This configuration will send Percy snapshot status (approved/pending) along with the test results from a TestNG XML file to a specified Microsoft Teams webhook.
    {
  "targets": [
    {
      "name": "teams",
      "inputs": {
        "url": "<teams-incoming-webhook-url>"
      },
      "extensions": [
        {
          "name": "percy-analysis",
          "inputs": {
            "token": "<api-token>",
            "project_name": "<project-name>"
          }
        }
      ]
    }
  ],
  "results": [
    {
      "type": "testng",
      "files": ["path/to/testng-results.xml"]
    }
  ]
}

  9. Implement a custom target using a separate JavaScript file

    To send test results to a custom target using an external file, set the target name to custom and provide the path to the JavaScript file in the load input. The external file must export an asynchronous run function that receives the target and result objects.
    // custom.js
async function run({ target, result }) {
  // custom code to handle results
}

module.exports = {
  run
};

  10. Choose a Slack report type

    Depending on the level of detail needed, you can specify one of three report types using the publish option:

    • test-summary: Includes test title, total duration, and success percentage. If multiple suites exist, it displays these details per suite.
    • test-summary-slim: Includes only the test title, total duration, and success percentage, ignoring individual test suites. Recommended for keeping primary channels clean.
    • failure-details: Lists the names of failed test cases along with their specific failure reasons.

  11. Implement a custom extension using an inline function

    Custom extensions can be defined directly within a JavaScript configuration file by passing an asynchronous function to the load input of the custom extension.
    module.exports = {
  targets: [
    {
      name: "teams",
      inputs: {
        url: "<some-url>",
      },
      extensions: [
        {
          name: "custom",
          inputs: {
            load: async ({ result }) => {
              // custom logic here
            },
          },
        },
      ],
    },
  ],
  results: [
    {
      type: "testng",
      files: ["path/to/testng-results.xml"],
    },
  ],
};

  12. Use Mocha with the jUnit reporter for test results

    To use the jUnit reporter with Mocha, install the mocha-junit-reporter package and run tests using the mocha-junit-reporter flag. In the tool's configuration file, specify the result type as junit and provide the path to the generated XML result file.
    npm install mocha-junit-reporter --save-dev
mocha test --reporter mocha-junit-reporter
    {
  "targets": [
    {
      "name": "chat",
      "inputs": {
        "url": "<incoming-webhook-url>"
      }
    }
  ],
  "results": [
    {
      "type": "junit",
      "files": ["path/to/results.xml"]
    }
  ]
}