tasty

Use the -p|--pattern PATTERN option to restrict which tests are executed. Patterns are awk expressions evaluated against the test hierarchy.

How patterns work

When evaluating a pattern, Tasty provides the following fields:

• $0: The full test name (all groups and the test name concatenated with .).
• $1, $2, ... $NF: Individual components of the hierarchy (outermost group is $1, the test's own name is $NF).
• NF: The total number of components in the hierarchy.

Example Hierarchy

For a test defined as: testGroup "One" [ testGroup "Two" [ testCase "Three" _ ] ]

The fields are:

• $0 = "One.Two.Three"
• $1 = "One"
• $2 = "Two"
• $3 = "Three"
• NF = 3

Common Pattern Examples

• foo: Shortcut for /foo/ (matches foo anywhere in $0).
• $2 == "Two": Selects the subgroup named "Two".
• $0 !~ /skip/: Selects tests whose full name does not contain "skip".
• $NF !~ /skip/: Selects tests whose own name (not group name) does not contain "skip".
• $(NF-1) ~ /QuickCheck/: Selects tests whose immediate parent group name contains "QuickCheck".

Supported Operators and Functions

Tasty supports standard awk operators including grouping (), field reference $, logical !, &&, ||, arithmetic +, -, string concatenation, and comparison ==, !=, <, <=, >, >=.

It also supports substring matching ~ and !~ (e.g., $1 ~ /foo/), though it does not implement full regular expression matching (it performs a case-sensitive substring search).

Built-in functions:

• substr(s, m[, n]): Substring starting at position m with length n.
• tolower(s) / toupper(s): Case conversion.
• match(s, pat): Returns the 1-based position of pat in s.
• length([s]): Returns string length.

# Examples of pattern usage
./test -p foo              # Matches any test with 'foo' in the name
./test -p '$2 == "Two"'   # Selects subgroup 'Two'
./test -p '! /skip/'       # Excludes tests containing 'skip'

To create a test suite, you define a TestTree (often using testGroup to organize tests hierarchically) and pass it to defaultMain from Test.Tasty. You will typically import specific providers for the types of tests you are running (e.g., Test.Tasty.HUnit for unit tests or Test.Tasty.QuickCheck for property tests).

import Test.Tasty
import Test.Tasty.SmallCheck as SC
import Test.Tasty.QuickCheck as QC
import Test.Tasty.HUnit

import Data.List
import Data.Ord

main = defaultMain tests

tests :: TestTree
tests = testGroup "Tests" [properties, unitTests]

properties :: TestTree
properties = testGroup "Properties" [scProps, qcProps]

scProps = testGroup "(checked by SmallCheck)"
  [ SC.testProperty "sort == sort . reverse" $
      \list -> sort (list :: [Int]) == sort (reverse list)
  ]

qcProps = testGroup "(checked by QuickCheck)"
  [ QC.testProperty "sort == sort . reverse" $
      \list -> sort (list :: [Int]) == sort (reverse list)
  ]

unitTests = testGroup "Unit tests"
  [ testCase "List comparison (different length)" $
      [1, 2, 3] `compare` [1,2] @?= GT
  ]

You can customize the behavior of your test suite (e.g., mode of operation, test selection, provider parameters) using the standard console runner.

Command Line Interface

Run your test executable with the --help flag to see all available options for your specific configuration of ingredients and providers.

Common CLI options include:

• -p|--pattern PATTERN: Select tests satisfying an awk expression.
• -t|--timeout DURATION: Set timeout for individual tests (suffixes: ms, s, m, h; default: s).
• -l|--list-tests: List test names without running them.
• -j|--num-threads NUMBER: Number of threads for execution.
• -q|--quiet: No output; exit code indicates success/failure.
• --hide-successes: Only print failed tests.
• --color <never|always|auto>: Control colored output.

Environment Variables

Every option can be set via environment variables. To derive the variable name:

1. Replace hyphens - with underscores _.
2. Capitalize all letters.
3. Prepend TASTY_.

Example: --smallcheck-depth becomes TASTY_SMALLCHECK_DEPTH.

Note on Boolean Options: When using the CLI, boolean options are switches (e.g., --quickcheck-show-replay). When using environment variables, you must provide an explicit True or False value (case-insensitive), e.g., TASTY_QUICKCHECK_SHOW_REPLAY=true.

# Example of running with a timeout via CLI
./test --timeout=0.5m