DOCSEARCHbyAlgolia
Submit Docs

Reassure

repository·main·Indexed 23 days ago

https://github.com/callstack/reassure

A performance testing companion for React and React Native that allows developers to automate performance regression testing on CI or locally. It measures render characteristics and execution times of components and functions, comparing them against a stable baseline to detect regressions. It integrates with React Testing Library, React Native Testing Library, and Danger JS.

Tokens
17.9K
Snippets
41
Records
105
Agent score
80%

What's inside reassure

  1. What is Reassure and why use it?

    main

    Reassure is a performance testing library for React Native designed to automate performance regression testing on CI or local machines.

    Instead of manually profiling render patterns and memoization, Reassure allows you to write performance tests that verify your app continues to work performantly. It works by:

    1. Measuring render characteristics (duration and count) for a provided testing scenario.
    2. Repeating the scenario multiple times to mitigate environmental noise.
    3. Applying statistical analysis to determine if code changes are statistically significant.
    4. Generating human-readable reports for CI or Pull Request comments.

    Reassure is designed to reuse as much of your existing React Native Testing Library tests and setup as possible.

  2. Enable WebAssembly support in Reassure

    main

    Reassure provides support for WebAssembly (WASM).

    • In version 1.0.0, WASM support was enabled by default, running using V8 baseline compilers (sparkplug and liftoff for WASM).
    • In earlier experimental versions (0.10.2), this could be enabled via the --enable-wasm flag.

  3. Run performance measurements and comparisons

    main

    Starting from version 0.3.0, the reassure compare command was merged into reassure measure.

    • Running reassure measure will automatically generate a performance comparison if a baseline measurement file already exists.
    • To run measurements without generating a comparison, use the --no-compare option.
    • reassure is now an alias for the reassure measure command.

  4. How measurement stability and comparisons work

    main

    Reassure distinguishes between the noise of a single test and the stability of the environment:

    1. Per-Entry Noise: Use the Coefficient of Variation to understand the variability of a single Measurement Entry.
    2. Environment Stability: Use a Stability Check to compare two measurement files generated from the same code state. This helps determine if changes in results are due to the code or the machine/environment.
    3. Suite-wide Noise: To determine if a change (like a Reassure tweak) made an entire run less noisy, compare the Run Stability (the mean-duration-weighted average Coefficient of Variation) between the baseline and current measurement files, then inspect the Worst Measurement Entry.

  5. Understand Reassure Markdown report categories

    main

    Reassure generates a Markdown report that categorizes test scenarios to help you identify performance regressions:

    • Significant Changes To Duration: Statistically significant changes that indicate potential performance loss or improvement.
    • Meaningless Changes To Duration: Changes that are not statistically significant.
    • Changes To Count: Scenarios where the render or execution count changed.
    • Added Scenarios: Scenarios present in the current run but missing from the baseline.
    • Removed Scenarios: Scenarios present in the baseline but missing from the current run.

  6. Identify experimental render issues

    main

    Reassure includes an experimental feature to analyze component render patterns during the initial (warm-up) run to spot potential inefficiencies.

    Warning: This feature is experimental and its behavior may change without a major version bump.

    Supported issue types:

    • Initial updates: Detects the number of re-renders that occur synchronously immediately after the initial mount. This is often caused by useEffect hooks triggering state updates immediately. Ideally, the initial render should not trigger immediate subsequent renders.
    • Redundant updates (React Native only): Detects renders that result in the exact same host element tree as the previous render.
      • The check compares the host element structure and ignores differences in function props (like event handlers).
      • The report provides the indices of these redundant renders (where index 0 is the mount and index 1+ are updates) to assist in diagnosis.

  7. Understand the Reassure v1.x testing environment

    main

    Reassure v1.x uses Node.js's non-optimized compilation to better reflect the React Native runtime environment, replacing the previous JIT-less mode (--jitless).

    Key impacts:

    • Performance: Tests run approximately 2x faster than in v0.
    • WebAssembly: WebAssembly is now enabled by default; the --enable-wasm flag has been removed.

  8. Understand Reassure terminology and core concepts

    main

    Reassure uses specific terminology to describe performance measurement and stability. Understanding these distinctions is critical for interpreting reports and communicating results:

    • Measurement Entry: A single named performance scenario result produced by repeated runs of the same scenario.
    • Coefficient of Variation: A unitless measure of relative duration variability for a Measurement Entry, calculated as standard deviation / mean. It represents how noisy a specific scenario is.
    • Stability Check: A workflow that compares two measurement files from the same code state to assess environment or machine stability.
    • Run Stability: A summary of stability across all Measurement Entries in a single measurement file. It is calculated using a mean-duration-weighted average Coefficient of Variation.
    • Worst Measurement Entry: The specific entry in a measurement file with the highest Coefficient of Variation (the noisiest test), not necessarily the slowest test.

  9. Analyze Reassure performance results

    main

    Reassure categorizes test scenario results to help you distinguish between meaningful performance regressions and noise. When reviewing reports, look for these categories:

    • Significant Changes To Duration: Performance changes that are statistically significant. These should be investigated as they indicate potential performance losses or improvements.
    • Meaningless Changes To Duration: Performance changes that are not statistically significant (likely noise).
    • Changes To Count: Scenarios where the number of renders or executions changed.
    • Added Scenarios: New test scenarios present in the current run but missing from the baseline.
    • Removed Scenarios: Scenarios present in the baseline but missing from the current run.

  10. Write async performance tests with a scenario

    main

    If your component involves asynchronous logic or user interactions, use the scenario option in measureRenders. The scenario is an async function where you perform actions using Testing Library methods (like fireEvent).

    Note: Ensure your scenario waits for async changes to settle using findBy queries, waitFor, or waitForElementToBeRemoved.

    If using a version of React Native Testing Library older than v10.1.0, the screen helper is not available globally; instead, it is passed as the first argument to the scenario function.

    import { measureRenders } from 'reassure';
import { screen, fireEvent } from '@testing-library/react-native';
import { ComponentUnderTest } from './ComponentUnderTest';

test('Test with scenario', async () => {
  const scenario = async () => {
    fireEvent.press(screen.getByText('Go'));
    await screen.findByText('Done');
  };

  await measureRenders(<ComponentUnderTest />, { scenario });
});

  12. Create a performance testing script (`reassure-tests.sh`)

    main

    To detect performance changes, you must compare the performance of your current code against a baseline (e.g., the main branch). A common approach is to use a shell script that switches branches, installs dependencies, and runs Reassure for both states.

    Note: You must run git fetch origin on CI to allow git switch to work correctly.

    #!/usr/bin/env bash
set -e

BASELINE_BRANCH=${GITHUB_BASE_REF:="main"}

# Required for `git switch` on CI
git fetch origin

# Gather baseline perf measurements
git switch "$BASELINE_BRANCH"
yarn install
yarn reassure --baseline

# Gather current perf measurements & compare results
git switch --detach -
yarn install
yarn reassure