Testplane Documentation

repository·master·Indexed 21 days ago

https://github.com/gemini-testing/testplane

A scalable testing framework for web applications based on mocha and wdio (formerly Hermione). Testplane supports visual testing with assertView, cross-browser/platform execution, and component/unit testing in Node.js and browser environments. It includes a GUI for managing visual regression failures, integration with Storybook for automatic screenshot testing, and support for Android applications via Appium. Built on WebdriverIO v8, it provides additional browser and element commands and supports both Devtools and Webdriver protocols.

Tokens
69.2K
Snippets
239
Records
311
Agent score
73%

What's inside testplane

  1. How the TestCollection API works

    master

    Available Methods:

    • getBrowsers(): Returns list of browsers with tests.
    • mapTests(browserId, callback): Maps over tests for a browser (or all if browserId is omitted).
    • sortTests(browserId, callback): Sorts tests.
    • eachTest(browserId, callback): Iterates over tests.
    • eachTestByVersion(browserId, callback): Iterates over tests and browser versions.
    • disableAll([browserId]): Disables all tests.
    • enableAll([browserId]): Enables all tests.
    • disableTest(fullTitle, [browserId]): Disables a specific test.
    • enableTest(fullTitle, [browserId]): Enables a specific test.
    • getRootSuite(browserId): Returns the root suite for a browser.
    • eachRootSuite(callback): Iterates over all root suites.
    • format(formatterType): Formats tests as list or tree.
    // Example: iterating over tests in a collection
    const collection = await testplane.readTests(['tests/']);
    collection.eachTest('chrome', (test, browserId) => {
        console.log(`Running test: ${test.fullTitle()}`);
    });
  2. Project structure for Storybook screenshot tests

    master

    In a Storybook-based Testplane project, screenshots are stored in directories named after the story file with a -screens suffix.

    Example structure:

    • .storybook/: Storybook configuration.
    • .testplane.conf.ts: Testplane configuration file.
    • src/stories/Button.stories.ts: The story file for a component.
    • src/stories/Button.stories.ts-screens/: The directory where screenshots for the Button component tests are stored.
    |____.storybook // storybook config
    |____.testplane.conf.ts // file with testplane configuration
    |____testplane-tests // directory with testplane test example (without storybook)
    |____src
    | |____stories // directory with your stories
    | | |____Button.stories.ts // file with a single story for the Button component
    | | |____Button.stories.ts-screens // directory where your screenshots for the Button tests will be stored
    | | |____Page.stories.ts
    | | |____Page.stories.ts-screens
  3. Use the BEFORE_FILE_READ event to inject test helpers

    master

    The BEFORE_FILE_READ event is triggered synchronously before a test file is read and parsed. It is available in both the master and worker processes. This event is primarily used to inject custom controllers (helpers) into the global testplane object, making them available for use within your test files.

    To use this, subscribe to testplane.events.BEFORE_FILE_READ and use the provided testParser object to register a controller via setController.

    testplane.on(testplane.events.BEFORE_FILE_READ, ({ file, testplane, testParser }) => {
        testParser.setController('logger', {
            log: function(prefix) {
                console.log(`${prefix}: just parsed ${this.fullTitle()} from ${file} for browser ${this.browserId}`);
            }
        });
    });
  4. Create custom helpers using Testplane events

    master

    You can extend Testplane by creating plugins that register custom helpers. This is achieved by using the testParser.setController method during the BEFORE_FILE_READ event.

    To implement a helper:

    1. Listen for the BEFORE_FILE_READ event.
    2. Use testParser.setController(name, controller) to define your helper. The controller object contains the methods that will be exposed to your test files.
    3. If running in a worker process (testplane.isWorker()), you should set the controller to a no-op (e.g., _.noop) to prevent errors, as helpers are typically used during the test discovery phase in the main process.

    This pattern allows you to inject logic that can influence test execution based on the current browser or test context.

    // Inside a plugin
    testplane.on(testplane.events.BEFORE_FILE_READ, ({ testParser }) => {
        testParser.setController('myHelper', {
            doSomething: function(arg) {
                // logic here
            }
        });
    });
  5. Understand Testplane event tags and modes

    master

    When developing Testplane plugins, event descriptions use specific tags to indicate how the event behaves and where it is triggered. Understanding these tags is essential for correctly implementing event handlers:

    • sync or async: Indicates whether the event handler is called in synchronous or asynchronous mode.
    • master: The event is available in the Testplane master process (responsible for orchestration).
    • worker: The event is available in Testplane workers (the subprocesses where tests actually execute).
    • interceptable: The event can be intercepted and modified by a plugin.
  6. Extend the Testplane CLI using the CLI event

    master

    The CLI event is triggered synchronously immediately upon startup, before Testplane parses the command-line arguments. You can subscribe to this event to add new commands, options, or extend the help documentation.

    The event handler receives a cli object of the Commander type, allowing you to use standard Commander.js methods like .option() to modify the CLI interface.

    testplane.on(testplane.events.CLI, (cli) => {
        cli.option(
            '--some-option <some-value>',
            'the full description of the option'
        );
    });
  7. How element screenshot capturing works

    master

    Element screenshot capturing in Testplane is a two-stage process designed to work across different browsers by managing scrolling and coordinate computation.

    1. Coordinates Computation

    Before capturing, the system calculates the necessary parameters:

    1. Scroll Management: Saves current scroll positions and detects the appropriate scroll element (via selectorToScroll or common scroll parents).
    2. Visibility: Scrolls to the topmost element if it lacks sufficient visibility.
    3. Spec Calculation: Computes capture specs for every selector, including full, clipped, and visible rectangles (accounting for box shadows, outlines, and pseudo-elements).
    4. Safe Area: Calculates a safeArea to avoid artifacts from sticky, fixed, or absolute elements.
    5. Ignore Areas: Computes rectangles for elements marked to be ignored.
    6. Result: Returns an object containing capture specs, scroll offset, pixelRatio, safeArea coordinates, ignoreElements coordinates, and viewport data.

    2. Screenshot Capturing

    Once specs are computed, the actual capture begins:

    1. Chunking: Captures the current viewport and registers it as an in-memory composite chunk.
    2. Iterative Scrolling: Recomputes capture specs and the safeArea after every scroll to account for layout shifts, lazy loading, or sticky elements. The system scrolls by the remaining capture height (usually near the safeArea size) and repeats the process until the area is fully captured.
    3. Compositing: Anchors all chunks in capture-area coordinates, selects safe vertical bands, fills gaps with black, and joins the pieces into the final screenshot.