Alchemist Golden Testing for Flutter

repository·main·Indexed 43 days ago

A Flutter package that simplifies golden testing with a declarative API. It features GoldenTestGroup and GoldenTestScenario for managing complex widget variations, handles cross-platform rendering inconsistencies via dedicated CI tests using the Ahem font, and provides advanced configuration for themes, gestures, and text scale factors through AlchemistConfig.

Tokens: 7.2K
Snippets: 21
Records: 30
Agent score: 48%

What's inside Alchemist

Explore the Alchemist Example Project
main
The Alchemist Example Project is a Flutter application designed to showcase the capabilities of the Alchemist golden testing package. It demonstrates how to implement golden tests for custom widgets with multiple variations.
Key components of the example include:
- Custom Widgets: RedButton and ContactListTile (located in lib/widgets/), each featuring various visual states.
- Testing Implementation: The test/ directory contains the actual golden test implementations for these widgets.
For a deep dive into how to use Alchemist for widget testing, refer to the main Alchemist README or examine the test files within this repository.
https://github.com/betterment/alchemist/blob/main/example/README.md
How platform tests and CI tests differ
main
Alchemist distinguishes between two types of golden tests to solve the problem of inconsistent rendering across different platforms:
1. CI Goldens: These are used for automated assertions in CI environments. They are stable and used to compare test output against reference files to ensure layout, color, and structure remain consistent.
2. Platform Goldens: These are used by developers locally to see how a specific platform (e.g., macOS, Linux, Windows) renders a component. They are useful for debugging visual issues when CI tests fail, but they are inherently unstable and should not be used for automated assertions.
Recommended File Structure:
- Test files: test/<widget_name>_golden_test.dart
- CI goldens: test/goldens/ci/<widget_name>.png (Track these in source control)
- Platform goldens: test/goldens/<platform_name>/<widget_name>.png (Do not track these in source control)
https://github.com/betterment/alchemist/blob/main/RECOMMENDED_SETUP_GUIDE.md
How platform tests vs. CI tests work in Alchemist
main
Alchemist supports two distinct types of golden tests to handle cross-platform rendering inconsistencies:
1. Platform Tests: These generate golden files with human-readable text. They are intended for local development. Because text rendering varies between operating systems (e.g., macOS vs. Linux), these tests are platform-dependent.
2. CI Tests: These are designed for CI environments to ensure platform agnosticism. Text is replaced with colored squares using the Ahem font family. This ensures that the output is identical regardless of whether the test runs on Windows, macOS, or Linux, preventing false negatives in CI.
Use Platform Tests when you want to visually inspect your widgets locally, and rely on CI Tests for your automated pipelines.
https://github.com/betterment/alchemist/blob/main/README.md
Customize image sizing and pumping behavior
main
Image Sizing
Alchemist automatically resizes images to the smallest possible size for the widgets. You can control this via the constraints argument in goldenTest.
- If constraints are tight (min == max), no resizing occurs.
- If min/max are set, the image resizes to those bounds as long as it doesn't clip widgets.
Pumping Behavior
- Before tests: Use pumpBeforeTest to prime the tree. Options include pumpOnce, pumpNTimes(n), onlyPumpAndSettle, or precacheImages to preload images.
- Pumping widgets: Provide a custom pumpWidget callback to goldenTest to wrap the widget under test in specific providers or MaterialApp instances.
https://github.com/betterment/alchemist/blob/main/README.md
Run Alchemist tests in CI
main
In your CI environment, run tests using the standard Flutter test command:
```
flutter test # Other arguments...
```
CRITICAL: Do not pass the --update-goldens flag in CI. Passing this flag will regenerate all golden files, causing all tests to pass regardless of whether the UI has actually changed, which defeats the purpose of golden testing.
https://github.com/betterment/alchemist/blob/main/RECOMMENDED_SETUP_GUIDE.md
Generate golden files with flutter test
main
After writing your golden tests, you must generate the initial golden image files. Run the standard Flutter test command with the --update-goldens flag.
```
flutter test --update-goldens
```
https://github.com/betterment/alchemist/blob/main/example/example.md

Set global configuration via flutter_test_config.dart

main

To apply an AlchemistConfig to every test in your package, create a flutter_test_config.dart file in your test/ directory and wrap the testMain function.

import 'dart:async';
import 'package:alchemist/alchemist.dart';

Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  return AlchemistConfig.runWithConfig(
    config: AlchemistConfig(
      // Configure the config here.
    ),
    run: testMain,
  );
}

Configure Alchemist using flutter_test_config.dart

main

To implement the recommended workflow, use a flutter_test_config.dart file in your package. This file is executed before every test.

You should use AlchemistConfig.runWithConfig to wrap your testMain. A key part of the configuration is setting platformGoldensConfig.enabled to false when running in a CI environment (detected via the CI environment variable) to ensure only stable CI goldens are used for assertions.

// flutter_test_config.dart

import 'dart:async';

import 'package:alchemist/alchemist.dart';
import 'package:custom_styling_package/custom_styling_package.dart';

Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  // ignore: do_not_use_environment
  const isRunningInCi = bool.fromEnvironment('CI', defaultValue: false);

  return AlchemistConfig.runWithConfig(
    config: AlchemistConfig(
      theme: CustomTheme.light(),
      platformGoldensConfig: const PlatformGoldensConfig(
        enabled: !isRunningInCi,
      ),
    ),
    run: testMain,
  );
}

Customize iOS launch screen assets
main
You can customize the iOS launch screen by replacing the image files located in the example/ios/Runner/Assets.xcassets/LaunchImage.imageset/ directory.
Alternatively, you can manage these assets through Xcode:
1. Open the iOS project using open ios/Runner.xcworkspace.
2. In the Xcode Project Navigator, select Runner/Assets.xcassets.
3. Drag and drop your desired images into the asset catalog.
```
open ios/Runner.xcworkspace
```
https://github.com/betterment/alchemist/blob/main/example/ios/Runner/Assets.xcassets/LaunchImage.imageset/README.md
Generate and run golden tests
main
Generating Golden Files
To generate new golden files (or update existing ones), run:
```
flutter test --update-goldens
```
Generated files are stored in goldens/ci/ for CI tests and goldens/<platform_name> (e.g., goldens/macos/) for platform-specific tests.
Running and Filtering Tests
All Alchemist tests are automatically tagged with golden. You can use standard Flutter test flags to filter them:
```
# Run all tests (including goldens)
flutter test

# Only run golden tests
flutter test --tags golden

# Run all tests except golden tests
flutter test --exclude-tags golden
```
https://github.com/betterment/alchemist/blob/main/README.md

Write a basic golden test with goldenTest()

main

To write a golden test, use the goldenTest function within a test file in your test/ directory. It is recommended to group related tests using a group and use GoldenTestGroup to organize multiple scenarios into a single table-like output.

Use GoldenTestScenario to define individual test cases within a group. Each scenario requires a name and a child widget.

import 'package:alchemist/alchemist.dart';
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  group('ListTile Golden Tests', () {
    goldenTest(
      'renders correctly',
      fileName: 'list_tile',
      builder: () => GoldenTestGroup(
        scenarioConstraints: const BoxConstraints(maxWidth: 600),
        children: [
          GoldenTestScenario(
            name: 'with title',
            child: ListTile(
              title: Text('ListTile.title'),
            ),
          ),
          GoldenTestScenario(
            name: 'with title and subtitle',
            child: ListTile(
              title: Text('ListTile.title'),
              subtitle: Text('ListTile.subtitle'),
            ),
          ),
          GoldenTestScenario(
            name: 'with trailing icon',
            child: ListTile(
              title: Text('ListTile.title'),
              trailing: Icon(Icons.chevron_right_rounded),
            ),
          ),
        ],
      ),
    );
  });
}

Configure .gitignore for Alchemist golden files
main
To prevent unstable platform goldens and local test failures from being committed to source control, add the following patterns to your .gitignore file. This configuration ensures that only CI goldens (located in test/**/goldens/ci/*) are tracked:
```
# Ignore non-CI golden files and failures
test/**/goldens/**/*.*
test/**/failures/**/*.*
!test/**/goldens/ci/*.*
```
https://github.com/betterment/alchemist/blob/main/RECOMMENDED_SETUP_GUIDE.md

Alchemist Golden Testing for Flutter

What's inside Alchemist

Image Sizing

Pumping Behavior

Generating Golden Files

Running and Filtering Tests