Cluecumber Reporting Engine

repository·main·Indexed 43 days ago

https://github.com/trivago/cluecumber

A reporting engine that transforms Cucumber-compatible JSON test results from frameworks like Cucumber BDD and Karate into aggregated, navigable HTML reports. It provides a core JVM library (cluecumber-core 4.0.0) and a Maven plugin (cluecumber-maven 4.0.0) for report generation, featuring customizable branding, custom CSS, navigation links, and advanced configuration for scenario run grouping and auto-expansion of attachments and error messages.

Tokens
6.6K
Snippets
26
Records
35
Agent score
53%

What's inside Cluecumber

  1. Overview of Cluecumber reporting

    main

    Cluecumber is a reporting tool that generates aggregated test reports from Cucumber-compatible JSON files. It supports JSON formats generated by Cucumber BDD, Karate, and other compatible frameworks.

    There are two ways to use Cluecumber:

    • Cluecumber Core: Used to generate reports directly from Java code (e.g., within Karate runners).
    • Cluecumber Maven: Used to generate reports via Maven.

    Reports can be customized with a custom logo, favicon, CSS, custom parameters, custom links, and custom embedded views.

  2. Overview of Cluecumber Engine

    main

    Cluecumber Engine is the primary component responsible for generating reports. It is designed to be an internal component used by higher-level tools like Cluecumber Core and Cluecumber Maven.

    Important: Do not use Cluecumber Engine as a standalone dependency. If you need to generate reports programmatically from your own code, you should use Cluecumber Core instead.

  3. What pages are generated in a Cluecumber report

    main

    Cluecumber produces a comprehensive set of report pages to help analyze test results:

    • All Scenarios: Scenarios grouped by status (passed, failed, or skipped).
    • Rerun Scenarios: Scenarios that had previous runs (if the option is enabled).
    • Scenario Sequence: Scenarios in their running order with individual status information.
    • Scenario Details: Detailed view of a single scenario including steps, hooks, stack traces, and attachments.
    • All Features: An overview of all features.
    • All Tags: All tags used in scenarios, features, and example tables, including status information.
    • All Exceptions: A list of all exception types that occurred.
    • All Steps: All steps used in the suite with status information.
    • Scenarios by Tag: Scenarios filtered by a specific tag.
    • Scenarios by Feature: Scenarios belonging to a specific feature.
    • Scenario by Step: Scenarios that include a specific step.
    • Scenario by Exception: Scenarios that failed with a specific exception type.
    • Tree View: A hierarchical tree view of features and scenarios for an overview of the test suite.

    Users can also specify custom links or embedded views.

  4. Generate Cluecumber reports

    main

    To generate a report, use the CluecumberCore.Builder to instantiate the reporter and call generateReports(jsonDirectory, reportDirectory).

    Two parameters are mandatory:

    • jsonDirectory: The source folder containing your Cucumber JSON result files.
    • reportDirectory: The root directory where the Cluecumber HTML report will be generated.

    Typically, both directories should be located within your Maven target folder.

    String jsonDirectory = "examples/core-example/json";
    String reportDirectory = "examples/core-example/target/cluecumber_report";
    
    new CluecumberCore.Builder()
            .build().generateReports(jsonDirectory, reportDirectory);
  5. Add custom information to the report

    main

    You can add custom metadata to the report start page using two methods:

    Using Properties

    Define key-value pairs within the <customParameters> block.

    • Underscores in keys are converted to spaces in the report.
    • URLs starting with http, https, or ftp become clickable links.
    • If a key starts with an underscore (_), only the value is displayed (the key is hidden).
    <customParameters>
        <Custom_Parameter>This is a test</Custom_Parameter>
        <Custom_URL>http://www.google.com</Custom_URL>
        <_Text>This is a long text displayed without the key.</_Text>
    </customParameters>

    Using a File

    Specify a path to a .properties file using the customParametersFile property. The file should follow the standard properties format:

    Custom_Parameter=This is a test
    Custom_URL=http://www.google.com
    _Text=This is a long text without the key.

    Display Mode

    Use customParametersDisplayMode to control where these parameters appear:

    • SCENARIO_PAGES (default): Only on scenario and scenario sequence pages.
    • ALL_PAGES: On every page in the report.
    <customParameters>
        <Custom_Parameter>This is a test</Custom_Parameter>
        <Custom_URL>http://www.google.com</Custom_URL>
        <_Text>This is a long text that is displayed without the key. This can be used to display longer texts in the
            report!
        </_Text>
    </customParameters>
  6. Install Cluecumber Core via Maven

    main

    Add the cluecumber-core dependency to your pom.xml to use Cluecumber for JVM reporting of Cucumber BDD JSON formats.

    <dependency>
      <groupId>com.trivago.rta</groupId>
      <artifactId>cluecumber-core</artifactId>
      <version>4.0.0</version>
    </dependency>
  7. Prepare Cucumber JSON output

    main

    Before generating reports, you must ensure your Cucumber runner is configured to output JSON files. In your Cucumber runner configuration, add the json plugin pointing to a target directory.

    @CucumberOptions(
        plugin = {"json:target/cucumber-report/cucumber.json"}
    )
  8. Pass custom parameters to the Cluecumber report

    main

    To include custom data in your report via the command line, follow these two steps:

    1. In your pom.xml: Define an empty property in the <properties> block and map it inside the <customParameters> section of the Cluecumber configuration.
    2. Via Command Line: Pass the property using the -D flag.

    If the property is not passed during execution, Cluecumber will ignore it and it will not appear in the report.

    <!-- 1. Setup in pom.xml -->
    <properties>
        <someProperty/>
    </properties>
    
    <configuration>
        <customParameters>
            <My_Parameter_Name>${someProperty}</My_Parameter_Name>
        </customParameters>
    </configuration>
    # 2. Pass the value via CLI
    mvn cluecumber:reporting -DsomeProperty="this is cool"
  9. Install Cluecumber Maven plugin

    main

    To use Cluecumber for Maven reporting, add the cluecumber-maven plugin to your pom.xml. It is recommended to bind the reporting goal to the post-integration-test phase to ensure reports are generated after tests complete.

    <plugin>
        <groupId>com.trivago.rta</groupId>
        <artifactId>cluecumber-maven</artifactId>
        <version>4.0.0</version>
        <executions>
            <execution>
                <id>report</id>
                <phase>post-integration-test</phase>
                <goals>
                    <goal>reporting</goal>
                </goals>
            </execution>
        </executions>
        <configuration>
            <sourceJsonReportDirectory>${project.build.directory}/cucumber-report</sourceJsonReportDirectory>
            <generatedHtmlReportDirectory>${project.build.directory}/generated-report</generatedHtmlReportDirectory>
        </configuration>
    </plugin>
    <plugin>
        <groupId>com.trivago.rta</groupId>
        <artifactId>cluecumber-maven</artifactId>
        <version>4.0.0</version>
        <executions>
            <execution>
                <id>report</id>
                <phase>post-integration-test</phase>
                <goals>
                    <goal>reporting</goal>
                </goals>
            </execution>
        </executions>
        <configuration>
            <sourceJsonReportDirectory>${project.build.directory}/cucumber-report</sourceJsonReportDirectory>
            <generatedHtmlReportDirectory>${project.build.directory}/generated-report</generatedHtmlReportDirectory>
        </configuration>
    </plugin>
  10. Prerequisites for Cluecumber Maven

    main

    Cluecumber requires Cucumber JSON result files as input. You must configure your Cucumber runner to generate these files by adding the json plugin to your @CucumberOptions:

    @CucumberOptions(
            plugin = {"json:target/cucumber-report/cucumber.json"}
    )
    @CucumberOptions(
            plugin = {"json:target/cucumber-report/cucumber.json"}
    )
  11. Requirements for building and running Cluecumber

    main

    Building from source

    To build Cluecumber from source, you need:

    • Java >= 17
    • Maven >= 3.9.9

    Runtime requirements

    To use cluecumber-core at runtime, you must have:

    • Java >= 17 (via cluecumber-engine)