rapid

repository·master·Indexed 21 days ago

https://github.com/flyingmutant/rapid

A Go library for property-based testing that provides type-safe data generation, automatic test case minimization, and support for stateful testing. It allows developers to check high-level properties against diverse inputs using Generators and can be integrated with the standard Go fuzzer via MakeFuzz.

Tokens
7.3K
Snippets
35
Records
39
Agent score
74%

What's inside rapid

  1. How property-based testing works in rapid

    master

    Rapid operates by generating pseudo-random data based on a Generator specification.

    1. Specification: A Generator defines how data should be constructed but does not generate it immediately.
    2. Drawing: When you call .Draw(t, "name"), rapid consumes bytes from an internal random bitstream to construct the value.
    3. Tracking: Rapid tracks how the random bytes correspond to the structure of the generated value. This metadata allows rapid to perform automatic minimization: when a test fails, rapid uses this structural knowledge to find the smallest possible input that still triggers the failure.

    This differs from traditional example-based testing by checking high-level properties (e.g., decode(encode(x)) == x) against a diverse set of inputs rather than a fixed set of manual examples.

  2. Use rapid tests as fuzz targets with MakeFuzz

    master
    While property-based testing is designed for fast feedback loops, you can leverage the standard Go fuzzer for deeper coverage by using MakeFuzz. This allows any existing rapid test to be used as a target for testing.F.Fuzz, combining rapid's structured data generation with the coverage-guided feedback of the Go fuzzer.
  3. Configure rapid via CLI flags and environment variables

    master

    You can influence rapid's behavior during go test using flags prefixed with -rapid. or corresponding environment variables prefixed with RAPID_.

    CLI Flags: Pass flags directly to the test runner using -args:

    go test -args -rapid.checks=10_000

    Environment Variables: Replace the dot with an underscore and uppercase the name. Environment variables serve as defaults that can be overridden by explicit flags:

    RAPID_CHECKS=10 go test
  4. How Generator.Draw works and its logging behavior

    master

    When calling Draw(t, label), the following occurs:

    1. Grouping: The value is generated within a logical group in the random bit stream, which helps with structured generation and shrinking.
    2. Reference Checking: If t.refDraws contains a reference for the current draw index, Draw will verify that the generated value matches the expected reference using reflect.DeepEqual. If they differ, it calls t.tb.Fatalf.
    3. Logging: If the test context t has logging enabled (t.tbLog or t.rawLog != nil), the generated value is logged with the provided label. If no label is provided, it defaults to the draw index (e.g., #0).
    4. State Update: The internal draw counter t.draws is incremented.
  5. Write a basic property-based test with rapid.Check

    master

    To perform property-based testing, use rapid.Check(t, func(t *rapid.T) { ... }). Inside the callback, use rapid.Generator methods (like rapid.Int() or rapid.String()) and call .Draw(t, "name") to retrieve the generated values. If a property is violated, use t.Fatalf to signal failure. Rapid will automatically attempt to minimize the failing input to the smallest possible case.

    package rapid_test
    
    import (
    	"slices"
    	"testing"
    
    	"pgregory.net/rapid"
    )
    
    func TestSortStrings(t *testing.T) {
    	rapid.Check(t, func(t *rapid.T) {
    		s := rapid.SliceOf(rapid.String()).Draw(t, "s")
    		slices.Sort(s)
    		if !slices.IsSorted(s) {
    			t.Fatalf("unsorted after sort: %v", s)
    		}
    	})
    }
  6. Configure MakeCustom with MakeConfig

    master

    The MakeConfig struct allows you to fine-tune how MakeCustom behaves by providing overrides for different levels of the type hierarchy:

    • Types: A map from reflect.Type to a *Generator[any]. If a type is encountered during reflection, this generator will be used instead of the default one.
    • Kinds: A map from reflect.Kind to a *Generator[any]. This allows you to override generation for entire categories of types (e.g., all reflect.Int or all reflect.String).
    • Fields: A nested map map[reflect.Type]map[string]*Generator[any]. This allows you to target specific exported fields by name within a particular struct type.
    cfg := rapid.MakeConfig{
    	// Override all strings globally
    	Kinds: map[reflect.Kind]*rapid.Generator[any]{
    		reflect.String: rapid.String().AsAny(),
    	},
    	// Override a specific type
    	Types: map[reflect.Type]*rapid.Generator[any]{
    		reflect.TypeOf(MyCustomType{}): rapid.Custom(func(t *rapid.T) any { ... }).AsAny(),
    	},
    	// Override specific fields in a struct
    	Fields: map[reflect.Type]map[string]*rapid.Generator[any]{
    		reflect.TypeOf(MyStruct{}): {
    			"ID": rapid.Uint64().AsAny(),
    		},
    	},
    }
  7. Transform generators with Filter and AsAny

    master

    You can transform existing generators using the following methods:

    • Filter(fn func(V) bool) *Generator[V]: Returns a new generator that only produces values for which the provided function fn returns true. Note that if the filter is too restrictive, generation may fail.
    • AsAny() *Generator[any]: Returns a new generator that produces values of type any (the zero-interface type) instead of the specific type V.
    // Filter a generator to only even numbers
    evenGen := intGen.Filter(func(v int) bool { return v%2 == 0 })
    
    // Convert a specific generator to an any generator
    anyGen := intGen.AsAny()
  8. Generate maps from values with MapOfValues and MapOfNValues

    master

    When you want to generate a map where the keys are derived from the values themselves, use the Values variants. You must provide a keyFn that takes a value of type V and returns a key of type K.

    • MapOfValues[K comparable, V any](val *Generator[V], keyFn func(V) K) *Generator[map[K]V]: Shorthand for MapOfNValues(val, -1, -1, keyFn).
    • MapOfNValues[K comparable, V any](val *Generator[V], minLen int, maxLen int, keyFn func(V) K) *Generator[map[K]V]: Creates a map generator where keys are generated by applying keyFn to the generated values.

    Note: The generator ensures keys are unique by rejecting attempts to insert a value that would result in a duplicate key.

    // Example: Generate a map where the key is the length of the string value
    // Map[int]string
    keyFn := func(s string) int { return len(s) }
    gen := rapid.MapOfNValues(rapid.String(), 1, 5, keyFn)
  9. Generate strings with String(), StringOf(), and StringN()

    master

    The rapid package provides several shorthand functions for generating UTF-8 strings:

    • String(): A shorthand for StringOf(Rune()). Generates strings using the default rune set.
    • StringOf(elem *Generator[rune]): Generates strings where each character is produced by the provided rune generator.
    • StringN(minRunes, maxRunes, maxLen int): A shorthand for StringOfN(Rune(), minRunes, maxRunes, maxLen). Generates strings with constraints on the number of runes and the total byte length.
    • StringOfN(elem *Generator[rune], minRunes, maxRunes, maxLen int): The primary way to create constrained string generators.

    Constraints:

    • minRunes >= 0: Minimum number of runes.
    • maxRunes >= 0: Maximum number of runes.
    • maxLen >= 0: Maximum byte length (UTF-8).

    Panics:

    • StringOfN panics if maxRunes >= 0 and minRunes > maxRunes.
    • StringOfN panics if maxLen >= 0 and maxLen < maxRunes (since a rune can be up to 4 bytes, the byte length must be able to accommodate the rune count).
    // Random string using default runes
    s := rapid.String()
    
    // String with 5 to 10 runes, max 20 bytes
    s := rapid.StringN(5, 10, 20)
    
    // String using a specific rune generator (e.g., only digits)
    digitGen := rapid.RuneFrom([]rune{'0', '1', '2', '3', '4', '5', '6', '7', '8', '9'})
    s := rapid.StringOf(digitGen)
  10. Customize reflection-based generation with MakeCustom[V](cfg)

    master

    The MakeCustom[V](cfg MakeConfig) function creates a generator for type V using reflection, but allows you to override the automatic generation logic using a MakeConfig object. This is useful when you want to provide specific generators for certain types, kinds, or specific struct fields within the inferred structure.

    // Example: Overriding a specific field in a struct
    type User struct {
    	Name string
    	Age  int
    }
    
    cfg := rapid.MakeConfig{
    	Fields: map[reflect.Type]map[string]*rapid.Generator[any]{
    		reflect.TypeOf(User{}): {
    			"Age": rapid.IntRange(0, 120).AsAny(), // Force Age to be between 0 and 120
    		},
    	},
    }
    
    userGen := rapid.MakeCustom[User](cfg)
  11. Generate integers with maximum bounds

    master

    If you need to restrict the generated values to be at most a certain maximum, use the Max variants of the integer generators. These generators produce values in the range [min, max], where min is the minimum possible value for that type.

    Maximum Bound Generators

    • ByteMax(max byte)
    • IntMax(max int)
    • Int8Max(max int8)
    • Int16Max(max int16)
    • Int32Max(max int32)
    • Int64Max(max int64)
    • UintMax(max uint)
    • Uint8Max(max uint8)
    • Uint16Max(max uint16)
    • Uint32Max(max uint32)
    • Uint64Max(max uint64)
    • UintptrMax(max uintptr)
    // Example: Generate a uint32 that is at most 500
    gen := rapid.Uint32Max(500)
  12. Generate maps with MapOf and MapOfN

    master

    Use these functions to generate map[K]V types where keys and values are generated by separate generators.

    • MapOf[K comparable, V any](key *Generator[K], val *Generator[V]) *Generator[map[K]V]: Shorthand for MapOfN(key, val, -1, -1).
    • MapOfN[K comparable, V any](key *Generator[K], val *Generator[V], minLen int, maxLen int) *Generator[map[K]V]: Creates a map generator with length constraints.

    Note: MapOfN panics if maxLen >= 0 and minLen > maxLen.

    // Example: Generate a map with string keys and int values, size 1-5
    gen := rapid.MapOfN(rapid.String(), rapid.Int(), 1, 5)