Loki Visual Regression Testing

Loki automatically attempts to stabilize screenshots by disabling CSS transitions/animations and requestAnimationFrame. However, certain types of animations are not covered and can cause flaky tests:

• Looped requestAnimationFrame animations
• GIFs
• SVG animations
• Native Lottie animations
• React Native Animated library

To disable Loki's automatic handling, use the chromeEnableAnimations option. To handle the unsupported animations listed above, you must manually disable them in your component logic, typically by detecting if Loki is running.

Loki supports multiple rendering targets to ensure tests are reproducible across different environments:

• Chrome in docker (Recommended for CI/CD)
• Chrome in AWS Lambda
• Local Chrome application
• iOS simulator
• Android emulator

For large test suites, you can use a remote renderer to avoid local machine bottlenecks. Currently, only AWS Lambda is supported. To set up a renderer, create a new project, install the necessary dependencies, and export a handler using createChromeAWSLambdaRenderer from @loki/renderer-aws-lambda.

mkdir loki-lambda-renderer
cd loki-lambda-renderer
yarn init -y

yarn add @loki/renderer-aws-lambda @sparticuz/chromium

// index.js
const { createChromeAWSLambdaRenderer } = require('@loki/renderer-aws-lambda');

module.exports = {
  handler: createChromeAWSLambdaRenderer(),
};

If you are running tests for the first time, you must create the baseline reference images. Run the update command to generate these images. By default, Loki stores them in a loki folder. These images should be committed to your version control system (e.g., using git-lfs).

yarn loki update

When running loki through a package manager like yarn or npm, you must prepend your arguments with -- to ensure they are passed through to the loki binary. Alternatively, you can add ./node_modules/.bin to your PATH to run loki directly.

Example using yarn:

yarn loki test -- --port 9009

To deploy your renderer to AWS, follow these steps:

1. Create a zip file: Package your production dependencies.
2. Create an IAM role: Assign a role to the Lambda. For a minimal setup, you can use a policy that denies all access.
3. Deploy the function: Use the AWS CLI to create the function. Ensure you set sufficient --memory-size (e.g., 2048) and --timeout (e.g., 120) for Chrome execution.

yarn --production
zip -r loki-lambda-renderer.zip .

aws iam create-role \
  --role-name lambda-role.loki \
  --assume-role-policy-document "arn:aws:iam::aws:policy/AWSDenyAll"

aws lambda create-function \
 --function-name loki \
 --runtime nodejs16.x \
 --role arn:aws:iam::<your role> \
 --handler index.handler \
 --memory-size 2048 \
 --timeout 120 \
 --zip-file fileb://loki-lambda-renderer.zip

If a story is only useful for development or contains content that cannot be accurately captured by a single screenshot (like an animated GIF), you can opt out of visual regression testing by setting skip: true in the loki parameter.

For Storybook storiesOf syntax: Pass the option as the third argument to .add().

For Component Story Format (CSF): Add the parameter to the story.parameters object.

// storiesOf syntax
storiesOf('MyComponent', module)
  .add('enabled story', () => <MyComponent />)
  .add('skipped story', () => <MyComponent />, { loki: { skip: true } });

// CSF syntax
export const SkippedStory = () => <MyComponent />;

SkippedStory.story = {
  parameters: {
    loki: { skip: true },
  },
};

Run Loki in test mode to compare current story screenshots against reference images.

To run a specific subset of configurations, pass a regular expression as the second argument. For example, yarn loki test laptop will run any configuration containing the string laptop.

Note: You must have your Storybook server running (e.g., yarn storybook) for Loki to access your stories. If testing against iOS or Android emulators, the respective Storybook apps must also be running.

yarn loki test
# To test a subset of configurations:
yarn loki test laptop

Custom pages are built using React components.

1. Save your React component as a .js file in website/pages/en.
2. To make the page accessible from the top navigation, add it to the headerLinks in website/siteConfig.js using the page key.

// website/siteConfig.js
{
  headerLinks: [
    ...
    { page: 'my-new-custom-page', label: 'My New Custom Page' },
    ...
  ],
  ...
}

To add a new documentation page, follow these two steps:

1. Create a new markdown file in the /docs directory (e.g., docs/newly-created-doc.md). Ensure the file includes a frontmatter block with a unique id and a title.
2. Register the new page in the sidebar by adding its id to the appropriate category in website/sidebar.json.

---
id: newly-created-doc
title: This Doc Needs To Be Edited
---

My new content here..

// Add newly-created-doc to the Getting Started category of docs in website/sidebar.json
{
  "docs": {
    "Getting Started": [
      "quick-start",
      "newly-created-doc" // new doc here
    ],
    ...
  },
  ...
}

When running Loki on a CI server, you should use the --requireReference flag. This ensures that the process fails if stories are missing reference images, preventing accidental regressions when no new images are expected to be committed. Additionally, since CI environments typically do not require an interactive server, you can point Loki directly to your built Storybook static files using the --reactUri flag instead of running Storybook in server mode.

For a reference implementation, see the loki react example project.

build-storybook && loki --requireReference --reactUri file:./storybook-static