DOCSEARCHbyAlgolia
Submit Docs

RapidCheck Documentation

repository·master·Indexed 22 days ago

https://github.com/emil-e/rapidcheck

A C++ framework for property-based testing that allows developers to define invariants and automatically falsify them using random test data and shrinking to find minimal counterexamples. Includes a core API with rc::check and RC_ASSERT, configuration via RC_PARAMS environment variables, and integration modules for Boost Test and Catch2.

Tokens
16.2K
Snippets
53
Records
73
Agent score
77%

What's inside RapidCheck

  1. Implement stateful testing with rc::state

    master

    Standard RapidCheck properties test simple inputs. For stateful systems (like data structures), you must test sequences of operations. RapidCheck provides the rc::state framework to handle this by requiring two components:

    1. A Model: A simple struct representing the expected state of the System Under Test (SUT). This allows RapidCheck to track state and generate valid command sequences without running the actual SUT, which is critical for efficient shrinking.
    2. Commands: Subclasses of rc::state::Command<Model, Sut> that represent operations performed on the system.

    To use this, you implement commands that define how to update the model, how to run the operation on the SUT, and what preconditions must be met.

    // Example of a simple model
struct FastKvStoreModel {
  std::map<std::string, std::string> data;
};

// Example of a command
struct Remove : rc::state::Command<FastKvStoreModel, FastKvStore> {
  std::string key;
  
  void checkPreconditions(const FastKvStoreModel &s0) const override {
    RC_PRE(s0.data.count(key) != 0);
  }

  void apply(FastKvStoreModel &s0) const override {
    s0.data.erase(key);
  }

  void run(const FastKvStoreModel &s0, FastKvStore &sut) const override {
    sut.remove(key);
    RC_ASSERT(!sut.hasKey(key));
  }

  void show(std::ostream &os) const override {
    os << "Remove(" << key << ")";
  }
};

  2. Report input data distribution using tags

    master

    RapidCheck allows you to monitor how input data is distributed across your property tests by using "tags". Tags are sequences of strings associated with each test case. After all tests complete, RapidCheck prints a summary of the distribution of these categories to the console. This helps ensure your properties are exercising the intended range of inputs.

    Key behaviors:

    • Tags are treated as ordered sequences, not sets. RC_TAG("foo", "bar") is different from RC_TAG("bar", "foo").
    • You can call multiple tagging macros within a single test case; they are cumulative.
    • Values passed to tags are converted to strings using rc::toString.

  3. How RapidCheck displays values

    master

    RapidCheck uses the rc::show(const T &, std::ostream &) template function to convert values into strings for assertion messages and counterexamples. When rc::show is called for a type T, it follows this priority order to determine how to format the output:

    1. showValue overload: If a valid overload of showValue(v, os) exists for the type, it is used.
    2. Stream insertion operator: If no showValue is found, it looks for a valid std::ostream &operator<<(std::ostream &, const T &).
    3. Fallback: If neither is available, it prints <???>.

    To enable value display for your custom types, you can either implement a standard operator<< or provide a specialized showValue function.

  4. How test case shrinking works in RapidCheck

    master

    When RapidCheck finds a case that falsifies a property, it performs shrinking. Shrinking attempts to find the smallest possible counterexample that still fails the property.

    For example, if a bug only occurs when a vector has 10 or more elements, RapidCheck will attempt to reduce the vector size and the values within it until it finds the minimal input (e.g., a vector of size 10 with specific values) that triggers the failure. This makes debugging significantly easier by providing a concise counterexample rather than a large, complex input.

  5. Understand property execution results

    master

    A RapidCheck property execution can result in one of three outcomes:

    1. Success: The property passes. By default, this requires 100 successful tests (this is configurable). Output format: OK, passed 100 tests.
    2. Failure: RapidCheck finds a counterexample. It will automatically attempt to "shrink" the inputs to find the smallest possible failing case. Output includes the number of tests run, the number of shrinks performed, the minimal counterexample (formatted as a tuple), and the failing condition.
    3. Gave Up: RapidCheck stops because too many test cases were discarded due to failing RC_PRE preconditions. The default threshold is 10 discards per successful test. This usually indicates that the property's preconditions are too restrictive for the generator being used; you should use a more specialized generator instead.

  6. Use RapidCheck assertions instead of Boost Test assertions

    master

    When writing RapidCheck properties within Boost Test, avoid using standard Boost Test assertions. RapidCheck treats exceptions as property failures, but Boost Test assertions do not use exceptions to signal failures.

    To ensure property failures are correctly caught and handled by RapidCheck's shrinking engine, use RapidCheck assertions like RC_ASSERT inside your properties.

  7. Handle non-copyable models in stateful testing

    master
    If your model does not support copy constructors or copy-assignment operators, you cannot pass the model state directly to rc::state::check. Instead, use the overload that accepts a callable which returns a fresh model state. RapidCheck will call this function to create new model states as needed during the testing process.

  8. How RapidCheck captures expression expansion

    master

    RapidCheck can often capture and print the expanded values of expressions used within assertion macros (similar to the Catch framework). This provides detailed diagnostic information when a property fails.

    Example output for a failed assertion:

    main.cpp:24:
foo == bar

Expands to:
"foo" == "bar"

  9. Understand the Shrinkable<T> abstraction

    master

    In RapidCheck, Shrinkable<T> is a fundamental template class that represents a value of type T along with a structured way to shrink it. It uses value semantics, allowing it to be copied and passed around like any other value.

    Conceptually, a Shrinkable<T> is not just a value, but a value combined with a tree of possible ways to shrink it. This tree structure allows RapidCheck to recursively search for the smallest possible value that satisfies (or fails) a property.

  10. Use `Seq<T>` to iterate over lazy sequences

    master

    Seq<T> implements a lazy sequence (often called a "stream") of values of type T. It provides a simple interface for retrieving values sequentially using the next() method.

    Key Characteristics

    • Lazy Evaluation: Values are generated or retrieved on demand.
    • Termination: The next() method returns a Maybe<T> (similar to boost::optional or Haskell's Maybe). When the sequence is exhausted, next() returns an empty Maybe to signal there are no more values.
    • Mutability: Calling next() modifies the Seq instance by advancing its internal state.
    • Value Semantics: Seq objects can be copied and passed around like any other value.

    To consume a sequence, call next() in a loop until it returns an empty Maybe.

    // Conceptual usage pattern
while (auto value = my_seq.next()) {
    // Use value
}

  11. Handle dependencies and recursion with gen::exec and gen::lazy

    master

    Dependent values with gen::exec

    Use gen::exec when the value of one generator depends on the value of another. Inside the callable, you can use operator* on a Gen<T> to pick a value. Note: gen::exec has lower compile-time and runtime performance than other combinators and may restrict shrinking capabilities. Use it only when necessary.

    Recursive types with gen::lazy

    When defining generators for recursive data structures (like trees or linked lists), use gen::lazy to wrap the recursive call. This prevents infinite recursion during generator construction.

    Example: Dependent values and Recursion

    // Dependent values
const auto name = *gen::exec([](const Address &address) {
  const auto gender = *gen::element(kMale, kFemale);
  const auto name = *genName(gender);
  return Person(name, gender, address);
});

// Recursive LinkedList
Gen<LinkedList> genLinkedList() {
  return gen::oneOf(gen::just(LinkedList()),
                    gen::construct<LinkedList>(gen::arbitrary<int>(),
                                               gen::lazy(&genLinkedList)));
}

  12. Use RC_ASSERT instead of Google Test assertions in properties

    master

    RapidCheck treats any thrown exception as a property failure. While you can use assertion mechanisms that signal failures via exceptions, you should avoid using standard Google Test assertions (like ASSERT_EQ) inside RapidCheck properties.

    Instead, use RapidCheck's own RC_ASSERT macro. This ensures proper integration with the property-based testing engine.

    RC_GTEST_PROP(MyTest, myProp, (int x)) {
  // DO NOT USE: ASSERT_EQ(x, 10);
  // USE: 
  RC_ASSERT(x == 10);
}