proptest

Proptest and QuickCheck both use random input generation and automatic shrinking to find failing test cases. However, Proptest uses explicit Strategy objects instead of relying on type-based generation.

Key advantages of Proptest's Strategy approach:

• Multiple Strategies per Type: You can define many different strategies for a single type (e.g., different integer ranges) without needing newtypes.
• Direct Constraint Expression: You can express ranges directly (e.g., 0..100) rather than relying on a global size configuration.
• Composability: You can compose complex types by creating tuples of components and using prop_map to transform them into the target struct. Shrinking is handled automatically via the input types.
• Constraint Awareness: Strategies are aware of constraints, reducing the number of rejected inputs during generation and shrinking.

Trade-off: Generating complex values in Proptest can be slower than QuickCheck because Proptest maintains intermediate states to support a richer, integrated shrinking model.

When Proptest identifies a failing test case, it automatically persists that case to a file. This ensures that subsequent test runs replay the known failing input before attempting to generate new random cases. This prevents 'flaky' test behavior where a test fails in one run but passes in the next due to different random seeds.

Key behaviors:

• Storage Location: By default, files are stored in a directory tree rooted at proptest-regressions. The file name is derived from the source file containing the failing test.
• Replay Mechanism: Proptest replays these persisted cases at the start of every test run.
• Implementation Detail: Proptest persists the seed used to generate the failing value rather than the value itself. This makes the persistence more robust to changes in the strategy, as the seed will always produce a valid value within the strategy's domain, even if the strategy has been slightly modified.

Since TestRunner::run accepts only a single Strategy, you can test functions with multiple arguments by using compound strategies. The most common method is to combine multiple strategies into a single tuple strategy. This produces a single value (a tuple) that contains all the required inputs, which you can then destructure within your test closure.

Common types of compound strategies include:

• Tuples: A tuple of strategies (e.g., (strategy_a, strategy_b)) is a strategy for tuples of the values those strategies produce.
• Fixed-size arrays: Arrays of strategies.
• Vecs: Vecs of strategies, which produce collections of values parallel to the strategy collection.
• Collection module strategies: Various other strategies provided in the proptest::collection module.

# extern crate proptest;
use proptest::test_runner::TestRunner;

fn add(a: i32, b: i32) -> i32 {
    a + b
}

#[test]
# fn dummy() {} // Doctests don't build `#[test]` functions, so we need this
fn test_add() {
    let mut runner = TestRunner::default();
    // Combine our two inputs into a strategy for one tuple. Our test
    // function then destructures the generated tuples back into separate
    // `a` and `b` variables to be passed in to `add()`.
    runner.run(&(0..1000i32, 0..1000i32), |(a, b)| {
        let sum = add(a, b);
        assert!(sum >= a);
        assert!(sum >= b);
        Ok(())
    }).unwrap();
}
# fn main() { test_add(); }

Property testing explores a randomly sampled portion of the input space rather than the entire space. Because time is finite, it is extremely unlikely to find single-value edge cases in large input spaces (e.g., finding a specific value out of $2^{64}$ possibilities).

Key Takeaways:

• Edge Cases: Property testing may miss specific, rare edge cases that exist in large domains.
• Complementary Testing: Traditional unit testing with intelligently selected, manual test cases is still necessary to cover critical edge cases that random sampling might miss.
• Strategy Design: The effectiveness of property testing depends on your Strategy. A strategy that is too broad (e.g., .{1,4096} for a parser) might produce inputs that never reach deep logic (like a code generator), making the tests ineffective.

# extern crate proptest;
use proptest::prelude::*;

proptest! {
    #[test]
    # fn dummy(0..1) {}
    fn i64_abs_is_never_negative(a: i64) {
        // This actually fails if a == i64::MIN, but randomly picking one
        // specific value out of 2⁶⁴ is overwhelmingly unlikely.
        assert!(a.abs() >= 0);
    }
}
# fn main() { i64_abs_is_never_negative() }